% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/confusion_matrix.R
\name{confusion_matrix}
\alias{confusion_matrix}
\alias{confusion_matrix.default}
\alias{confusion_matrix.formula}
\alias{confusion_matrix.glm}
\alias{print.qwraps2_confusion_matrix}
\title{Confusion Matrices (Contingency Tables)}
\usage{
confusion_matrix(
  ...,
  thresholds = NULL,
  confint_method = "logit",
  alpha = getOption("qwraps2_alpha", 0.05)
)

\method{confusion_matrix}{default}(
  truth,
  predicted,
  ...,
  thresholds = NULL,
  confint_method = "logit",
  alpha = getOption("qwraps2_alpha", 0.05)
)

\method{confusion_matrix}{formula}(
  formula,
  data = parent.frame(),
  ...,
  thresholds = NULL,
  confint_method = "logit",
  alpha = getOption("qwraps2_alpha", 0.05)
)

\method{confusion_matrix}{glm}(
  x,
  ...,
  thresholds = NULL,
  confint_method = "logit",
  alpha = getOption("qwraps2_alpha", 0.05)
)

\method{print}{qwraps2_confusion_matrix}(x, ...)
}
\arguments{
\item{...}{pass through}

\item{thresholds}{a numeric vector of thresholds to be used to define the
confusion matrix (one threshold) or matrices (two or more thresholds).  If
\code{NULL} the unique values of \code{predicted} will be used.}

\item{confint_method}{character string denoting if the logit (default),
binomial, or Wilson Score method for deriving confidence intervals}

\item{alpha}{alpha level for 100 * (1 - alpha)\% confidence intervals}

\item{truth}{a integer vector with the values \code{0} and \code{1}, or a logical vector.
A value of \code{0} or \code{FALSE} is an indication of condition negative;
\code{1} or \code{TRUE} is an indication of condition positive.}

\item{predicted}{a numeric vector.  See Details.}

\item{formula}{column (known) ~ row (test) for building the confusion matrix}

\item{data}{environment containing the variables listed in the formula}

\item{x}{a \code{glm} object}
}
\value{
\code{confusion_matrix} returns a list with elements
\itemize{
  \item \code{cm_stats} a data.frame with columns:
  \item \code{auroc} numeric value for the area under the receiver operating
  curve
  \item \code{auroc_ci} a numeric vector of length two with the lower and
  upper bounds for a 100(1-alpha)\% confidence interval about the auroc
  \item \code{auprc} numeric value for the area under the precision recall
  curve
  \item \code{auprc_ci} a numeric vector of length two with the lower and
  upper limits for a 100(1-alpha)\% confidence interval about the auprc
  \item \code{confint_method} a character string reporting the method used to
  build the \code{auroc_ci} and \code{auprc_ci}
  \item \code{alpha} the alpha level of the confidence intervals
  \item \code{prevalence} the proportion of the input of positive cases, that
  is (TP + FN) / (TP + FN + FP + TN) = P / (P + N)
}
}
\description{
Construction of confusion matrices, accuracy, sensitivity,
specificity, confidence intervals (Wilson's method and (optional
bootstrapping)).
}
\details{
The confusion matrix:

\tabular{lccc}{
                    \tab      \tab True \tab Condition \cr
                    \tab      \tab +    \tab -         \cr
Predicted Condition \tab +    \tab TP   \tab FP        \cr
Predicted Condition \tab -    \tab FN   \tab TN        \cr
}
where
\itemize{
  \item FN: False Negative = truth = 1 & prediction < threshold,
  \item FP: False Positive = truth = 0 & prediction >= threshold,
  \item TN: True Negative  = truth = 0 & prediction < threshold, and
  \item TP: True Positive  = truth = 1 & prediction >= threshold.
}

The statistics returned in the \code{cm_stats} element are:
\itemize{
  \item accuracy    = (TP + TN) / (TP + TN + FP + FN)
  \item sensitivity, aka true positive rate or recall = TP / (TP + FN)
  \item specificity, aka true negative rate = TN / (TN + FP)
  \item positive predictive value (PPV), aka precision = TP / (TP + FP)
  \item negative predictive value (NPV) = TN / (TN + FN)
  \item false negative rate (FNR) = 1 - Sensitivity
  \item false positive rate (FPR) = 1 - Specificity
  \item false discovery rate (FDR) = 1 - PPV
  \item false omission rate (FOR) = 1 - NPV
  \item F1 score
  \item Matthews Correlation Coefficient (MCC) =
    ((TP * TN) - (FP * FN)) / sqrt((TP + FP) (TP+FN) (TN+FP) (TN+FN))
}

Synonyms for the statistics:
\itemize{
\item Sensitivity: true positive rate (TPR), recall, hit rate
\item Specificity: true negative rate (TNR), selectivity
\item PPV: precision
\item FNR: miss rate
}

Sensitivity and PPV could, in some cases, be indeterminate due to division by
zero.  To address this we use the GERBIL convention from the DICE group
\url{https://github.com/dice-group/gerbil/wiki/Precision,-Recall-and-F1-measure}:
If TP, FP, and FN are all 0, then PPV, sensitivity, and F1 will be defined to
be 1.  If TP are 0 and FP + FN > 0, then PPV, sensitivity, and F1 are all
defined to be 0.
}
\examples{

# Example 1: known truth and prediction status
df <-
  data.frame(
      truth = c(1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0)
    , pred  = c(1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 0, 0)
  )

confusion_matrix(df$truth, df$pred, thresholds = 1)

# Example 2: Use with a logistic regression model
mod <- glm(
  formula = spam ~ word_freq_our + word_freq_over + capital_run_length_total
, data = spambase
, family = binomial()
)

confusion_matrix(mod)
confusion_matrix(mod, thresholds = 0.5)

}
