% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/mutualinfo.R
\name{mutualinfo}
\alias{mutualinfo}
\title{Calculate mutual information between groups of joint variates}
\usage{
mutualinfo(
  Y1names,
  Y2names = NULL,
  X = NULL,
  learnt,
  tails = NULL,
  n = NULL,
  unit = "Sh",
  parallel = TRUE,
  verbose = FALSE
)
}
\arguments{
\item{Y1names}{Character vector: first group of joint variates}

\item{Y2names}{Character vector or \code{NULL}: second group of joint variates}

\item{X}{Matrix or data.frame or \code{NULL}: values of some variates conditional on which we want the probabilities.}

\item{learnt}{Either a character with the name of a directory or full path
for an 'learnt.rds' object, or such an object itself.}

\item{tails}{Named vector or list, or \code{NULL} (default). The names must match some or all of the variates in arguments \code{X}. For variates in this list, the probability conditional is understood in an semi-open interval sense: \eqn{X \le x} or \eqn{X \ge x}, an so on. See analogous argument in \code{\link[=Pr]{Pr()}}.}

\item{n}{Integer or \code{NULL} (default): number of samples from which to approximately calculate the mutual information. Default as many as Monte Carlo samples in \code{learnt}.}

\item{unit}{Either one of 'Sh' for \emph{shannon} (default), 'Hart' for \emph{hartley}, 'nat' for \emph{natural unit}, or a positive real indicating the base of the logarithms to be used.}

\item{parallel}{Logical or positive integer or cluster object. \code{TRUE} (default): use roughly half of available cores; \code{FALSE}: use serial computation; integer: use this many cores. It can also be a cluster object previously created with \code{\link[parallel:makeCluster]{parallel::makeCluster()}}; in this case the parallel computation will use this object.}

\item{verbose}{Logical, default \code{FALSE}: give messages about parallel processing?}
}
\value{
A list consisting of the following elements:
\itemize{
\item \code{MI}, a vector of \code{value} and \code{accuracy}: the mutual information between (joint) variates \code{Y1names} and (joint) variates \code{Y2names}.
\item \code{CondEn12}, \code{CondEn21}, vectors of \code{value} and \code{accuracy}: the conditional entropy of the first variate given the second, and vice versa.
\item \code{En1}, \code{En2}, vectors of \code{value} and \code{accuracy}: the (differential) entropies of the first and second variates.
\item \code{MI.rGauss}, a vector of \code{value} and \code{accuracy}: the absolute value of the Pearson correlation coefficient \eqn{r} of a \emph{multivariate Gaussian distribution} having mutual information \code{MI}; the two are related by \eqn{\mathrm{MI} = -\ln(1 - r^2)/2}. It may provide a vague intuition for the \code{MI} value for people more familiar with Pearson's correlation, but should be taken with a grain of salt.
\item \code{unit}, \code{Y1names}, \code{Y1names}: same as the input arguments, included for the user's convenience.
}
}
\description{
This function calculates various entropic information measures between two grops of joint variates: the mutual information, the conditional entropies, and the entropies.
}
\details{
If \eqn{Y_1} and \eqn{Y_2} are two variates, each of which can be a joint variate such as \eqn{Y_1 = (Y_{1,1}, Y_{1,2}, \dotsc)}, and \eqn{X} a third, also possibly join, variate, then the mutual information \eqn{\mathit{MI}} between \eqn{Y_1} and \eqn{Y_2}, conditional on \eqn{X = x}, is given by
\deqn{\mathit{MI}(Y_1, Y_2 \vert X = x) \mathrel{:=}
\sum_{y_1, y_2}
\mathrm{Pr}(Y_1 = y_1, Y_2 = y_2 \vert X = x, \text{data})
\log_2\frac{
\mathrm{Pr}(Y_1 = y_1, Y_2 = y_2 \vert X = x, \text{data})
}{
\mathrm{Pr}(Y_1 = y_1 \vert X = x, \text{data})
\cdot
\mathrm{Pr}(Y_2 = y_2 \vert X = x, \text{data})
} \, \mathrm{Sh}
}
an expression which can also be written in several other equivalent ways. It is an information-theoretic measure of association that is model-free, that is, does not depend on assumptions such as linearity, gaussianity, and similar. See \code{vignette('mutualinfo')} for discussion and example uses, and also the "References" section.  If \eqn{Y_1, Y_2} are \emph{jointly gaussian variates}, then there is a mathematical correspondence between their mutual information and their Pearson correlation coefficient; see output \code{MI.rGauss} in the "Value" section.

The conditional entropy of \eqn{Y_1} with respect to \eqn{Y_2}, conditional on \eqn{X = x}, is given by
\deqn{\mathit{CondEn12}(Y_1, Y_2 \vert X = x) \mathrel{:=}
-\sum_{y_1, y_2}
\mathrm{Pr}(Y_1 = y_1 \vert Y_2 = y_2, X = x, \text{data})
\log_2
\mathrm{Pr}(Y_1 = y_1 \vert Y_2 = y_2, X = x, \text{data})
\cdot
\mathrm{Pr}(Y_2 = y_2 \vert X = x, \text{data})
\, \mathrm{Sh}
}

The (differential) entropy of \eqn{Y_1}, conditional on \eqn{X = x}, is given by
\deqn{\mathit{En1}(Y_1 \vert X = x) \mathrel{:=}
-\sum_{y_1}
\mathrm{Pr}(Y_1 = y_1 \vert X = x, \text{data})
\log_2
\mathrm{Pr}(Y_1 = y_1 \vert  X = x, \text{data})
\, \mathrm{Sh}
}

see "References" section for discussions about entropy and conditional entropy.

The function \code{mutualinfo()} calculates the quantities above for the joint variates specified in the arguments \code{Y1names} and \code{Y2names}, conditional on the values of the variates specified in the data frame \code{X}. If \code{X} is omitted or \code{NULL}, then the posterior probabilities \eqn{\mathrm{Pr}(Y_1 | \text{data})} etc. are used. Each variate in the argument \code{X} can be specified either as a point-value \eqn{X = x} or as a left-open interval \eqn{X \le x} or as a right-open interval \eqn{X \ge x}, through the argument \code{tails}.

The computation of these quantities is done via Monte Carlo integration, using the samples produced by the \code{\link[=learn]{learn()}} function. The present function also output the numerical error associated with this computation.
}
\examples{
## Load the example `learnt` object calculated from the "penguins" dataset;
## variates: 'species' and 'bill_len'
learnt <- learntExample

## mutual information between variates 'species' and 'bill_len'
MI <- mutualinfo(Y1names = 'species', Y2names = 'bill_len',
  learnt = learnt, parallel = 1)

paste0(MI$MI, ' ', MI$unit, collapse = ' +/- ')

## Shannon entropy of variate 'species'
paste0(MI$En1, ' ', MI$unit, collapse = ' +/- ')


\donttest{
## Shannon entropy of variate 'species',
## conditional on a bill length of 30 mm:
entr <- mutualinfo(
  Y1names = 'species',
  X = data.frame(bill_len = 30),
  learnt = learnt, parallel = 1
)

paste0(entr$En1, ' ', entr$unit, collapse = ' +/- ')

## the entropy is now lower; indeed a penguin with a short bill length
## is most probably of the 'Adelie' species:
probs <- Pr(
  Y = data.frame(species = c('Adelie', 'Gentoo', 'Chinstrap')),
  X = data.frame(bill_len = 30),
  learnt = learnt, parallel = 1
)

print(probs)
}

}
\seealso{
\code{\link[=Pr]{Pr()}} to calculate probabilities and their variability.

\code{\link[=learn]{learn()}}, which generates the \code{learnt} objects required by \code{mutualinfo()}.
}
