% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/loo_compare.R, R/loo_compare.psis_loo_ss_list.R
\name{loo_compare}
\alias{loo_compare}
\alias{loo_compare.default}
\alias{print.compare.loo}
\alias{print.compare.loo_ss}
\title{Model comparison}
\usage{
loo_compare(x, ...)

\method{loo_compare}{default}(x, ...)

\method{print}{compare.loo}(x, ..., digits = 1, simplify = TRUE)

\method{print}{compare.loo_ss}(x, ..., digits = 1, simplify = TRUE)
}
\arguments{
\item{x}{An object of class \code{"loo"} or a list of such objects.}

\item{...}{Additional objects of class \code{"loo"}.}

\item{digits}{For the print method only, the number of digits to use when
printing.}

\item{simplify}{For the print method only, should only the essential columns
of the summary matrix be printed? The entire matrix is always returned, but
by default only the most important columns are printed.}
}
\value{
A matrix with class \code{"compare.loo"} that has its own
print method. See the \strong{Details} section.
}
\description{
Compare fitted models based on \link[=loo-glossary]{ELPD}.

By default the print method shows only the most important information. Use
\code{print(..., simplify=FALSE)} to print a more detailed summary.
}
\details{
When comparing two fitted models, we can estimate the difference in their
expected predictive accuracy by the difference in \code{elpd_loo} or
\code{elpd_waic} (or multiplied by \eqn{-2}, if desired, to be on the
deviance scale).

When using \code{loo_compare()}, the returned matrix will have one row per
model and several columns of estimates. The values in the \code{elpd_diff}
and \code{se_diff} columns of the returned matrix are computed by making
pairwise comparisons between each model and the model with the largest ELPD
(the model in the first row). For this reason the \code{elpd_diff} column
will always have the value \code{0} in the first row (i.e., the difference
between the preferred model and itself) and negative values in subsequent
rows for the remaining models.

To compute the standard error of the difference in ELPD --- which should
not be expected to equal the difference of the standard errors --- we use a
paired estimate to take advantage of the fact that the same set of \eqn{N}
data points was used to fit both models. These calculations should be most
useful when \eqn{N} is large, because then non-normality of the
distribution is not such an issue when estimating the uncertainty in these
sums. These standard errors, for all their flaws, should give a better
sense of uncertainty than what is obtained using the current standard
approach of comparing differences of deviances to a Chi-squared
distribution, a practice derived for Gaussian linear models or
asymptotically, and which only applies to nested models in any case.
}
\examples{
# very artificial example, just for demonstration!
LL <- example_loglik_array()
loo1 <- loo(LL, r_eff = NA)     # should be worst model when compared
loo2 <- loo(LL + 1, r_eff = NA) # should be second best model when compared
loo3 <- loo(LL + 2, r_eff = NA) # should be best model when compared

comp <- loo_compare(loo1, loo2, loo3)
print(comp, digits = 2)

# show more details with simplify=FALSE
# (will be the same for all models in this artificial example)
print(comp, simplify = FALSE, digits = 3)

# can use a list of objects
loo_compare(x = list(loo1, loo2, loo3))

\dontrun{
# works for waic (and kfold) too
loo_compare(waic(LL), waic(LL - 10))
}

}
\references{
Vehtari, A., Gelman, A., and Gabry, J. (2017a). Practical Bayesian model
evaluation using leave-one-out cross-validation and WAIC.
\emph{Statistics and Computing}. 27(5), 1413--1432. doi:10.1007/s11222-016-9696-4
(\href{https://link.springer.com/article/10.1007/s11222-016-9696-4}{journal version},
\href{https://arxiv.org/abs/1507.04544}{preprint arXiv:1507.04544}).

Vehtari, A., Simpson, D., Gelman, A., Yao, Y., and Gabry, J. (2019).
Pareto smoothed importance sampling.
\href{https://arxiv.org/abs/1507.02646}{preprint arXiv:1507.02646}
}
