% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/msm.R
\name{msm}
\alias{msm}
\title{Multi-state Markov and hidden Markov models in continuous time}
\usage{
msm(
  formula,
  subject = NULL,
  data = list(),
  qmatrix,
  gen.inits = FALSE,
  ematrix = NULL,
  hmodel = NULL,
  obstype = NULL,
  obstrue = NULL,
  covariates = NULL,
  covinits = NULL,
  constraint = NULL,
  misccovariates = NULL,
  misccovinits = NULL,
  miscconstraint = NULL,
  hcovariates = NULL,
  hcovinits = NULL,
  hconstraint = NULL,
  hranges = NULL,
  qconstraint = NULL,
  econstraint = NULL,
  initprobs = NULL,
  est.initprobs = FALSE,
  initcovariates = NULL,
  initcovinits = NULL,
  deathexact = NULL,
  death = NULL,
  exacttimes = FALSE,
  censor = NULL,
  censor.states = NULL,
  pci = NULL,
  phase.states = NULL,
  phase.inits = NULL,
  subject.weights = NULL,
  cl = 0.95,
  fixedpars = NULL,
  center = TRUE,
  opt.method = "optim",
  hessian = NULL,
  use.deriv = TRUE,
  use.expm = TRUE,
  analyticp = TRUE,
  na.action = na.omit,
  ...
)
}
\arguments{
\item{formula}{A formula giving the vectors containing the observed states
and the corresponding observation times. For example,

\code{state ~ time}

Observed states should be numeric variables in the set \code{1, \dots{}, n},
where \code{n} is the number of states.  Factors are allowed only if their
levels are called \code{"1", \dots{}, "n"}.

The times can indicate different types of observation scheme, so be careful
to choose the correct \code{obstype}.

For hidden Markov models, \code{state} refers to the outcome variable, which
need not be a discrete state.  It may also be a matrix, giving multiple
observations at each time (see \code{\link{hmmMV}}).}

\item{subject}{Vector of subject identification numbers for the data
specified by \code{formula}. If missing, then all observations are assumed
to be on the same subject. These must be sorted so that all observations on
the same subject are adjacent.}

\item{data}{Optional data frame in which to interpret the variables supplied
in \code{formula}, \code{subject}, \code{covariates}, \code{misccovariates},
\code{hcovariates}, \code{obstype} and \code{obstrue}.}

\item{qmatrix}{Matrix which indicates the allowed transitions in the
continuous-time Markov chain, and optionally also the initial values of
those transitions. If an instantaneous transition is not allowed from state
\eqn{r} to state \eqn{s}, then \code{qmatrix} should have \eqn{(r,s)} entry
0, otherwise it should be non-zero.

If supplying initial values yourself, then the non-zero entries should be
those values.  If using \code{gen.inits=TRUE} then the non-zero entries can
be anything you like (conventionally 1).  Any diagonal entry of
\code{qmatrix} is ignored, as it is constrained to be equal to minus the sum
of the rest of the row.

For example,\cr

\code{ rbind( c( 0, 0.1, 0.01 ), c( 0.1, 0, 0.2 ), c( 0, 0, 0 ) ) }\cr

represents a 'health - disease - death' model, with initial transition
intensities 0.1 from health to disease, 0.01 from health to death, 0.1 from
disease to health, and 0.2 from disease to death.

If the states represent ordered levels of severity of a disease, then this
matrix should usually only allow transitions between adjacent states.  For
example, if someone was observed in state 1 ("mild") at their first
observation, followed by state 3 ("severe") at their second observation,
they are assumed to have passed through state 2 ("moderate") in between, and
the 1,3 entry of \code{qmatrix} should be zero.

The initial intensities given here are with any covariates set to their
means in the data (or set to zero, if \code{center = FALSE}). If any
intensities are constrained to be equal using \code{qconstraint}, then the
initial value is taken from the first of these (reading across rows).}

\item{gen.inits}{If \code{TRUE}, then initial values for the transition
intensities are generated automatically using the method in
\code{\link{crudeinits.msm}}. The non-zero entries of the supplied
\code{qmatrix} are assumed to indicate the allowed transitions of the model.
This is not available for hidden Markov models, including models with
misclassified states.}

\item{ematrix}{If misclassification between states is to be modelled, this
should be a matrix of initial values for the misclassification
probabilities.  The rows represent underlying states, and the columns
represent observed states.  If an observation of state \eqn{s} is not
possible when the subject occupies underlying state \eqn{r}, then
\code{ematrix} should have \eqn{(r,s)} entry 0.  Otherwise \code{ematrix}
should have \eqn{(r,s)} entry corresponding to the probability of observing
\eqn{s} conditionally on occupying true state \eqn{r}. The diagonal of
\code{ematrix} is ignored, as rows are constrained to sum to 1.  For
example, \cr

\code{ rbind( c( 0, 0.1, 0 ), c( 0.1, 0, 0.1 ), c( 0, 0.1, 0 ) ) }\cr

represents a model in which misclassifications are only permitted between
adjacent states.

If any probabilities are constrained to be equal using \code{econstraint},
then the initial value is taken from the first of these (reading across
rows).

For an alternative way of specifying misclassification models, see
\code{hmodel}.}

\item{hmodel}{Specification of the hidden Markov model (HMM).  This should
be a list of return values from HMM constructor functions.  Each element of
the list corresponds to the outcome model conditionally on the corresponding
underlying state.  Univariate constructors are described in
the\code{\link{hmm-dists}} help page.  These may also be grouped together to
specify a multivariate HMM with a set of conditionally independent
univariate outcomes at each time, as described in \code{\link{hmmMV}}.

For example, consider a three-state hidden Markov model.  Suppose the
observations in underlying state 1 are generated from a Normal distribution
with mean 100 and standard deviation 16, while observations in underlying
state 2 are Normal with mean 54 and standard deviation 18. Observations in
state 3, representing death, are exactly observed, and coded as 999 in the
data.  This model is specified as

\code{hmodel = list(hmmNorm(mean=100, sd=16), hmmNorm(mean=54, sd=18),
hmmIdent(999))}

The mean and standard deviation parameters are estimated starting from these
initial values. If multiple parameters are constrained to be equal using
\code{hconstraint}, then the initial value is taken from the value given on
the first occasion that parameter appears in \code{hmodel}.

See the \code{\link{hmm-dists}} help page for details of the constructor
functions for each univariate distribution.

A misclassification model, that is, a hidden Markov model where the outcomes
are misclassified observations of the underlying states, can either be
specified using a list of \code{\link{hmmCat}} or \code{\link{hmmIdent}}
objects, or by using an \code{ematrix}.

For example, \cr

\code{ ematrix = rbind( c( 0, 0.1, 0, 0 ), c( 0.1, 0, 0.1, 0 ), c( 0, 0.1,
0, 0), c( 0, 0, 0, 0) ) }\cr

is equivalent to \cr

\code{hmodel = list( hmmCat(prob=c(0.9, 0.1, 0, 0)), hmmCat(prob=c(0.1, 0.8,
0.1, 0)), hmmCat(prob=c(0, 0.1, 0.9, 0)), hmmIdent()) }\cr}

\item{obstype}{A vector specifying the observation scheme for each row of
the data. This can be included in the data frame \code{data} along with the
state, time, subject IDs and covariates.  Its elements should be either 1, 2
or 3, meaning as follows:

\describe{ \item{1}{An observation of the process at an arbitrary time (a
"snapshot" of the process, or "panel-observed" data). The states are unknown
between observation times.} \item{2}{An exact transition time, with the
state at the previous observation retained until the current observation.
An observation may represent a transition to a different state or a repeated
observation of the same state (e.g. at the end of follow-up).  Note that if
all transition times are known, more flexible models could be fitted with
packages other than \pkg{msm} - see the note under \code{exacttimes}.

Note also that if the previous state was censored using \code{censor}, for
example known only to be state 1 or state 2, then \code{obstype} 2 means
that either state 1 is retained or state 2 is retained until the current
observation - this does not allow for a change of state in the middle of the
observation interval.  } \item{3}{An exact transition time, but the state at
the instant before entering this state is unknown. A common example is death
times in studies of chronic diseases.} } If \code{obstype} is not specified,
this defaults to all 1. If \code{obstype} is a single number, all
observations are assumed to be of this type.  The obstype value for the
first observation from each subject is not used.

This is a generalisation of the \code{deathexact} and \code{exacttimes}
arguments to allow different schemes per observation.  \code{obstype}
overrides both \code{deathexact} and \code{exacttimes}.

\code{exacttimes=TRUE} specifies that all observations are of obstype 2.

\code{deathexact = death.states} specifies that all observations of
\code{death.states} are of type 3.  \code{deathexact = TRUE} specifies that
all observations in the final absorbing state are of type 3.}

\item{obstrue}{In misclassification models specified with \code{ematrix},
\code{obstrue} is a vector of logicals (\code{TRUE} or \code{FALSE}) or
numerics (1 or 0) specifying which observations (\code{TRUE}, 1) are
observations of the underlying state without error, and which (\code{FALSE},
0) are realisations of a hidden Markov model.

In HMMs specified with \code{hmodel}, where the hidden state is known at
some times, if \code{obstrue} is supplied it is assumed to contain the
actual true state data.  Elements of \code{obstrue} at times when the hidden
state is unknown are set to \code{NA}.  This allows the information from HMM
outcomes generated conditionally on the known state to be included in the
model, thus improving the estimation of the HMM outcome distributions.

HMMs where the true state is known to be within a specific set at specific
times can be defined with a combination of \code{censor} and \code{obstrue}.
In these models, a code is defined for the \code{state} outcome (see
\code{censor}), and \code{obstrue} is set to 1 for observations where the
true state is known to be one of the elements of \code{censor.states} at the
corresponding time.}

\item{covariates}{A formula or a list of formulae representing the
covariates on the transition intensities via a log-linear model. If a single
formula is supplied, like

\code{covariates = ~ age + sex + treatment}

then these covariates are assumed to apply to all intensities.  If a named
list is supplied, then this defines a potentially different model for each
named intensity.  For example,

\code{covariates = list("1-2" = ~ age, "2-3" = ~ age + treatment)}

specifies an age effect on the state 1 - state 2 transition, additive age
and treatment effects on the state 2 - state 3 transition, but no covariates
on any other transitions that are allowed by the \code{qmatrix}.

If covariates are time dependent, they are assumed to be constant in between
the times they are observed, and the transition probability between a pair
of times \eqn{(t1, t2)} is assumed to depend on the covariate value at
\eqn{t1}.}

\item{covinits}{Initial values for log-linear effects of covariates on the
transition intensities. This should be a named list with each element
corresponding to a covariate.  A single element contains the initial values
for that covariate on each transition intensity, reading across the rows in
order.  For a pair of effects constrained to be equal, the initial value for
the first of the two effects is used.

For example, for a model with the above \code{qmatrix} and age and sex
covariates, the following initialises all covariate effects to zero apart
from the age effect on the 2-1 transition, and the sex effect on the 1-3
transition.  \code{ covinits = list(sex=c(0, 0, 0.1, 0), age=c(0, 0.1, 0,
0))}

For factor covariates, name each level by concatenating the name of the
covariate with the level name, quoting if necessary. For example, for a
covariate \code{agegroup} with three levels \code{0-15, 15-60, 60-}, use
something like

\code{ covinits = list("agegroup15-60"=c(0, 0.1, 0, 0), "agegroup60-"=c(0.1,
0.1, 0, 0))}

If not specified or wrongly specified, initial values are assumed to be
zero.}

\item{constraint}{A list of one numeric vector for each named covariate. The
vector indicates which covariate effects on intensities are constrained to
be equal. Take, for example, a model with five transition intensities and
two covariates. Specifying\cr

\code{constraint = list (age = c(1,1,1,2,2), treatment = c(1,2,3,4,5))}\cr

constrains the effect of age to be equal for the first three intensities,
and equal for the fourth and fifth. The effect of treatment is assumed to be
different for each intensity. Any vector of increasing numbers can be used
as indicators. The intensity parameters are assumed to be ordered by reading
across the rows of the transition matrix, starting at the first row,
ignoring the diagonals.

Negative elements of the vector can be used to indicate that particular
covariate effects are constrained to be equal to minus some other effects.
For example:

\code{constraint = list (age = c(-1,1,1,2,-2), treatment = c(1,2,3,4,5))
}\cr

constrains the second and third age effects to be equal, the first effect to
be minus the second, and the fifth age effect to be minus the fourth.  For
example, it may be realisitic that the effect of a covariate on the
"reverse" transition rate from state 2 to state 1 is minus the effect on the
"forward" transition rate, state 1 to state 2.  Note that it is not possible
to specify exactly which of the covariate effects are constrained to be
positive and which negative.  The maximum likelihood estimation chooses the
combination of signs which has the higher likelihood.

For categorical covariates, defined as factors, specify constraints as
follows:\cr

\code{list(..., covnameVALUE1 = c(...), covnameVALUE2 = c(...), ...)}\cr

where \code{covname} is the name of the factor, and \code{VALUE1},
\code{VALUE2}, ... are the labels of the factor levels (usually excluding
the baseline, if using the default contrasts).

Make sure the \code{contrasts} option is set appropriately, for example, the
default

\code{options(contrasts=c(contr.treatment, contr.poly))}

sets the first (baseline) level of unordered factors to zero, then the
baseline level is ignored in this specification.

To assume no covariate effect on a certain transition, use the
\code{fixedpars} argument to fix it at its initial value (which is zero by
default) during the optimisation.}

\item{misccovariates}{A formula representing the covariates on the
misclassification probabilities, analogously to \code{covariates}, via
multinomial logistic regression. Only used if the model is specified using
\code{ematrix}, rather than \code{hmodel}.

This must be a single formula - lists are not supported, unlike
\code{covariates}.  If a different model on each probability is required,
include all covariates in this formula, and use \code{fixedpars} to fix some
of their effects (for particular probabilities) at their default initial
values of zero.}

\item{misccovinits}{Initial values for the covariates on the
misclassification probabilities, defined in the same way as \code{covinits}.
Only used if the model is specified using \code{ematrix}.}

\item{miscconstraint}{A list of one vector for each named covariate on
misclassification probabilities. The vector indicates which covariate
effects on misclassification probabilities are constrained to be equal,
analogously to \code{constraint}.  Only used if the model is specified using
\code{ematrix}.}

\item{hcovariates}{List of formulae the same length as \code{hmodel},
defining any covariates governing the hidden Markov outcome models.  The
covariates operate on a suitably link-transformed linear scale, for example,
log scale for a Poisson outcome model. If there are no covariates for a
certain hidden state, then insert a NULL in the corresponding place in the
list.  For example, \code{hcovariates = list(~acute + age, ~acute, NULL).}}

\item{hcovinits}{Initial values for the hidden Markov model covariate
effects. A list of the same length as \code{hcovariates}. Each element is a
vector with initial values for the effect of each covariate on that state.
For example, the above \code{hcovariates} can be initialised with
\code{hcovariates = list(c(-8, 0), -8, NULL)}. Initial values must be given
for all or no covariates, if none are given these are all set to zero.  The
initial value given in the \code{hmodel} constructor function for the
corresponding baseline parameter is interpreted as the value of that
parameter with any covariates fixed to their means in the data.  If multiple
effects are constrained to be equal using \code{hconstraint}, then the
initial value is taken from the first of the multiple initial values
supplied.}

\item{hconstraint}{A named list. Each element is a vector of constraints on
the named hidden Markov model parameter. The vector has length equal to the
number of times that class of parameter appears in the whole model.

For example consider the three-state hidden Markov model described above,
with normally-distributed outcomes for states 1 and 2.  To constrain the
outcome variance to be equal for states 1 and 2, and to also constrain the
effect of \code{acute} on the outcome mean to be equal for states 1 and 2,
specify

\code{hconstraint = list(sd = c(1,1), acute=c(1,1))}

Note this excludes initial state occupancy probabilities and covariate
effects on those probabilities, which cannot be constrained.}

\item{hranges}{Range constraints for hidden Markov model parameters.
Supplied as a named list, with each element corresponding to the named
hidden Markov model parameter.  This element is itself a list with two
elements, vectors named "lower" and "upper".  These vectors each have length
equal to the number of times that class of parameter appears in the whole
model, and give the corresponding mininum amd maximum allowable values for
that parameter.  Maximum likelihood estimation is performed with these
parameters constrained in these ranges (through a log or logit-type
transformation).  Lower bounds of \code{-Inf} and upper bounds of \code{Inf}
can be given if the parameter is unbounded above or below.

For example, in the three-state model above, to constrain the mean for state
1 to be between 0 and 6, and the mean of state 2 to be between 7 and 12,
supply

\code{hranges=list(mean=list(lower=c(0, 7), upper=c(6, 12)))}

These default to the natural ranges, e.g. the positive real line for
variance parameters, and [0,1] for probabilities.  Therefore \code{hranges}
need not be specified for such parameters unless an even stricter constraint
is desired.  If only one limit is supplied for a parameter, only the first
occurrence of that parameter is constrained.

Initial values should be strictly within any ranges, and not on the range
boundary, otherwise optimisation will fail with a "non-finite value" error.}

\item{qconstraint}{A vector of indicators specifying which baseline
transition intensities are equal. For example,

\code{qconstraint = c(1,2,3,3)}

constrains the third and fourth intensities to be equal, in a model with
four allowed instantaneous transitions.  When there are covariates on the
intensities and \code{center=TRUE} (the default), \code{qconstraint} is
applied to the intensities with covariates taking the values of the means in
the data.  When \code{center=FALSE}, \code{qconstraint} is applied to the
intensities with covariates set to zero.}

\item{econstraint}{A similar vector of indicators specifying which baseline
misclassification probabilities are constrained to be equal.  Only used if
the model is specified using \code{ematrix}, rather than \code{hmodel}.}

\item{initprobs}{Only used in hidden Markov models.  Underlying state
occupancy probabilities at each subject's first observation.  Can either be
a vector of \eqn{nstates} elements with common probabilities to all
subjects, or a \eqn{nsubjects} by \eqn{nstates} matrix of subject-specific
probabilities.  This refers to observations after missing data and subjects
with only one observation have been excluded.

If these are estimated (see \code{est.initprobs}), then this represents an
initial value, and defaults to equal probability for each state.  Otherwise
this defaults to \code{c(1, rep(0, nstates-1))}, that is, in state 1 with a
probability of 1.  Scaled to sum to 1 if necessary.  The state 1 occupancy
probability should be non-zero.}

\item{est.initprobs}{Only used in hidden Markov models.  If \code{TRUE},
then the underlying state occupancy probabilities at the first observation
will be estimated, starting from a vector of initial values supplied in the
\code{initprobs} argument.  Structural zeroes are allowed: if any of these
initial values are zero they will be fixed at zero during optimisation, even
if \code{est.initprobs=TRUE}, and no covariate effects on them are
estimated.  The exception is state 1, which should have non-zero occupancy
probability.

Note that the free parameters during this estimation exclude the state 1
occupancy probability, which is fixed at one minus the sum of the other
probabilities.}

\item{initcovariates}{Formula representing covariates on the initial state
occupancy probabilities, via multinomial logistic regression.  The linear
effects of these covariates, observed at the individual's first observation
time, operate on the log ratio of the state \eqn{r} occupancy probability to
the state 1 occupancy probability, for each \eqn{r = 2} to the number of
states.  Thus the state 1 occupancy probability should be non-zero. If
\code{est.initprobs} is \code{TRUE}, these effects are estimated starting
from their initial values.  If \code{est.initprobs} is \code{FALSE}, these
effects are fixed at theit initial values.}

\item{initcovinits}{Initial values for the covariate effects
\code{initcovariates}.  A named list with each element corresponding to a
covariate, as in \code{covinits}. Each element is a vector with (1 - number
of states) elements, containing the initial values for the linear effect of
that covariate on the log odds of that state relative to state 1, from state
2 to the final state.  If \code{initcovinits} is not specified, all
covariate effects are initialised to zero.}

\item{deathexact}{Vector of indices of absorbing states whose time of entry
is known exactly, but the individual is assumed to be in an unknown
transient state ("alive") at the previous instant.  This is the usual
situation for times of death in chronic disease monitoring data.  For
example, if you specify \code{deathexact = c(4, 5)} then states 4 and 5 are
assumed to be exactly-observed death states.

See the \code{obstype} argument.  States of this kind correspond to
\code{obstype=3}. \code{deathexact = TRUE} indicates that the final
absorbing state is of this kind, and \code{deathexact = FALSE} or
\code{deathexact = NULL} (the default) indicates that there is no state of
this kind.

The \code{deathexact} argument is overridden by \code{obstype} or
\code{exacttimes}.

Note that you do not always supply a \code{deathexact} argument, even if
there are states that correspond to deaths, because they do not necessarily
have \code{obstype=3}.  If the state is known between the time of death and
the previous observation, then you should specify \code{obstype=2} for the
death times, or \code{exacttimes=TRUE} if the state is known at all times,
and the \code{deathexact} argument is ignored.}

\item{death}{Old name for the \code{deathexact} argument.  Overridden by
\code{deathexact} if both are supplied. Deprecated.}

\item{exacttimes}{By default, the transitions of the Markov process are
assumed to take place at unknown occasions in between the observation times.
If \code{exacttimes} is set to \code{TRUE}, then the observation times are
assumed to represent the exact times of transition of the process.  The
subject is assumed to be in the same state between these times.  An
observation may represent a transition to a different state or a repeated
observation of the same state (e.g. at the end of follow-up).  This is
equivalent to every row of the data having \code{obstype = 2}.  See the
\code{obstype} argument.  If both \code{obstype} and \code{exacttimes} are
specified then \code{exacttimes} is ignored.

Note that the complete history of the multi-state process is known with this
type of data.  The models which \pkg{msm} fits have the strong assumption of
constant (or piecewise-constant) transition rates.  Knowing the exact
transition times allows more realistic models to be fitted with other
packages.  For example parametric models with sojourn distributions more
flexible than the exponential can be fitted with the \pkg{flexsurv} package,
or semi-parametric models can be implemented with \pkg{survival} in
conjunction with \pkg{mstate}.}

\item{censor}{A state, or vector of states, which indicates censoring.
Censoring means that the observed state is known only to be one of a
particular set of states. For example, \code{censor=999} indicates that all
observations of \code{999} in the vector of observed states are censored
states.  By default, this means that the true state could have been any of
the transient (non-absorbing) states. To specify corresponding true states
explicitly, use a \code{censor.states} argument.

Note that in contrast to the usual terminology of survival analysis, here it
is the \emph{state} which is considered to be censored, rather than the
\emph{event time}.  If at the end of a study, an individual has not died,
but their true state is \emph{known}, then \code{censor} is unnecessary,
since the standard multi-state model likelihood is applicable.  Also a
"censored" state here can be at any time, not just at the end.

For hidden Markov models, censoring may indicate either a set of possible
observed states, or a set of (hidden) true states. The later case is
specified by setting the relevant elements of \code{obstrue} to 1 (and
\code{NA} otherwise).

Note in particular that general time-inhomogeneous Markov models with
piecewise constant transition intensities can be constructed using the
\code{censor} facility. If the true state is unknown on occasions when a
piecewise constant covariate is known to change, then censored states can be
inserted in the data on those occasions.  The covariate may represent time
itself, in which case the \code{pci} option to msm can be used to perform
this trick automatically, or some other time-dependent variable.

Not supported for multivariate hidden Markov models specified with
\code{\link{hmmMV}}.}

\item{censor.states}{Specifies the underlying states which censored
observations can represent. If \code{censor} is a single number (the
default) this can be a vector, or a list with one element.  If \code{censor}
is a vector with more than one element, this should be a list, with each
element a vector corresponding to the equivalent element of \code{censor}.
For example

\code{censor = c(99, 999), censor.states = list(c(2,3), c(3,4))}

means that observations coded 99 represent either state 2 or state 3, while
observations coded 999 are really either state 3 or state 4.}

\item{pci}{Model for piecewise-constant intensities.  Vector of cut points
defining the times, since the start of the process, at which intensities
change for all subjects.  For example

\code{pci = c(5, 10)}

specifies that the intensity changes at time points 5 and 10.  This will
automatically construct a model with a categorical (factor) covariate called
\code{timeperiod}, with levels \code{"[-Inf,5)"}, \code{"[5,10)"} and
\code{"[10,Inf)"}, where the first level is the baseline.  This covariate
defines the time period in which the observation was made.  Initial values
and constraints on covariate effects are specified the same way as for a
model with a covariate of this name, for example,

\code{covinits = list("timeperiod[5,10)"=c(0.1,0.1),
"timeperiod[10,Inf)"=c(0.1,0.1))}

Thus if \code{pci} is supplied, you cannot have a previously-existing
variable called \code{timeperiod} as a covariate in any part of a \code{msm}
model.

To assume piecewise constant intensities for some transitions but not others
with \code{pci}, use the \code{fixedpars} argument to fix the appropriate
covariate effects at their default initial values of zero.

Internally, this works by inserting censored observations in the data at
times when the intensity changes but the state is not observed.

If the supplied times are outside the range of the time variable in the
data, \code{pci} is ignored and a time-homogeneous model is fitted.

After fitting a time-inhomogeneous model, \code{\link{qmatrix.msm}} can be
used to obtain the fitted intensity matrices for each time period, for
example,

\code{qmatrix.msm(example.msm, covariates=list(timeperiod="[5,Inf)"))}

This facility does not support interactions between time and other
covariates.  Such models need to be specified "by hand", using a state
variable with censored observations inserted.  Note that the \code{data}
component of the \code{msm} object returned from a call to \code{msm} with
\code{pci} supplied contains the states with inserted censored observations
and time period indicators.  These can be used to construct such models.

Note that you do not need to use \code{pci} in order to model the effect of
a time-dependent covariate in the data.  \code{msm} will automatically
assume that covariates are piecewise-constant and change at the times when
they are observed.  \code{pci} is for when you want all intensities to
change at the same pre-specified times for all subjects.

\code{pci} is not supported for multivariate hidden Markov models specified with
\code{\link{hmmMV}}.   An approximate equivalent can be constructed by
creating a variable in the data to represent the time period, and treating
that as a covariate using the \code{covariates} argument to \code{msm}.
This will assume that the value of this variable is constant between
observations.}

\item{phase.states}{Indices of states which have a two-phase sojourn
distribution.  This defines a semi-Markov model, in which the hazard of an
onward transition depends on the time spent in the state.

This uses the technique described by Titman and Sharples (2009).  A hidden
Markov model is automatically constructed on an expanded state space, where
the phases correspond to the hidden states.  The "tau" proportionality
constraint described in this paper is currently not supported.

Covariates, constraints, \code{deathexact} and \code{censor} are expressed
with respect to the expanded state space.  If not supplied by hand,
\code{initprobs} is defined automatically so that subjects are assumed to
begin in the first of the two phases.

Hidden Markov models can additionally be given phased states.  The user
supplies an outcome distribution for each original state using
\code{hmodel}, which is expanded internally so that it is assumed to be the
same within each of the phased states.  \code{initprobs} is interpreted on
the expanded state space.  Misclassification models defined using
\code{ematrix} are not supported, and these must be defined using
\code{hmmCat} or \code{hmmIdent} constructors, as described in the
\code{hmodel} section of this help page.  Or the HMM on the expanded state
space can be defined by hand.

Output functions are presented as it were a hidden Markov model on the
expanded state space, for example, transition probabilities between states,
covariate effects on transition rates, or prevalence counts, are not
aggregated over the hidden phases.

Numerical estimation will be unstable when there is weak evidence for a
two-phase sojourn distribution, that is, if the model is close to Markov.

See \code{\link{d2phase}} for the definition of the two-phase distribution
and the interpretation of its parameters.

This is an experimental feature, and some functions are not implemented.
Please report any experiences of using this feature to the author!}

\item{phase.inits}{Initial values for phase-type models.  A list with one
component for each "two-phased" state.  Each component is itself a list of
two elements.  The first of these elements is a scalar defining the
transition intensity from phase 1 to phase 2.  The second element is a
matrix, with one row for each potential destination state from the
two-phased state, and two columns.  The first column is the transition rate
from phase 1 to the destination state, and the second column is the
transition rate from phase 2 to the destination state.  If there is only one
destination state, then this may be supplied as a vector.

In phase type models, the initial values for transition rates out of
non-phased states are taken from the \code{qmatrix} supplied to msm, and
entries of this matrix corresponding to transitions out of phased states are
ignored.}

\item{subject.weights}{Name of a variable in the data (unquoted) giving
  weights to apply to each subject in the data
  when calculating the log-likelihood as a weighted sum over
  subjects.  These are taken from the first observation for each
  subject, and any weights supplied for subsequent observations are
  not used.

 Weights at the observation level are not supported.}

\item{cl}{Width of symmetric confidence intervals for maximum likelihood
estimates, by default 0.95.}

\item{fixedpars}{Vector of indices of parameters whose values will be fixed
at their initial values during the optimisation. These are given in the
order: transition intensities (reading across rows of the transition
matrix), covariates on intensities (ordered by intensities within
covariates), hidden Markov model parameters, including misclassification
probabilities or parameters of HMM outcome distributions (ordered by
parameters within states), hidden Markov model covariate parameters (ordered
by covariates within parameters within states), initial state occupancy
probabilities (excluding the first probability, which is fixed at one minus
the sum of the others).

If there are equality constraints on certain parameters, then
\code{fixedpars} indexes the set of unique parameters, excluding those which
are constrained to be equal to previous parameters.

To fix all parameters, specify \code{fixedpars = TRUE}.

This can be useful for profiling likelihoods, and building complex models
stage by stage.}

\item{center}{If \code{TRUE} (the default, unless \code{fixedpars=TRUE})
then covariates are centered at their means during the maximum likelihood
estimation. This usually improves stability of the numerical optimisation.}

\item{opt.method}{If "optim", "nlm" or "bobyqa", then the corresponding R
function will be used for maximum likelihood estimation.
\code{\link{optim}} is the default.  "bobyqa" requires the package
\pkg{minqa} to be installed.  See the help of these functions for further
details.  Advanced users can also add their own optimisation methods, see
the source for \code{optim.R} in msm for some examples.

If "fisher", then a specialised Fisher scoring method is used (Kalbfleisch
and Lawless, 1985) which can be faster than the generic methods, though less
robust.  This is only available for Markov models with panel data
(\code{obstype=1}), that is, not for models with censored states, hidden
Markov models, exact observation or exact death times (\code{obstype=2,3}).}

\item{hessian}{If \code{TRUE} then standard errors and confidence intervals
are obtained from a numerical estimate of the Hessian (the observed
information matrix).  This is the default when maximum likelihood estimation
is performed.  If all parameters are fixed at their initial values and no
optimisation is performed, then this defaults to \code{FALSE}.  If
requested, the actual Hessian is returned in \code{x$paramdata$opt$hessian},
where \code{x} is the fitted model object.

If \code{hessian} is set to \code{FALSE}, then standard errors and
confidence intervals are obtained from the Fisher (expected) information
matrix, if this is available.  This may be preferable if the numerical
estimation of the Hessian is computationally intensive, or if the resulting
estimate is non-invertible or not positive definite.}

\item{use.deriv}{If \code{TRUE} then analytic first derivatives are used in
the optimisation of the likelihood, where available and an appropriate
quasi-Newton optimisation method, such as BFGS, is being used.  Analytic
derivatives are not available for all models.}

\item{use.expm}{If \code{TRUE} then any matrix exponentiation needed to
calculate the likelihood is done using the \pkg{expm} package.  Otherwise
the original routines used in \pkg{msm} 1.2.4 and earlier are used.  Set to
\code{FALSE} for backward compatibility, and let the package maintainer know
if this gives any substantive differences.}

\item{analyticp}{By default, the likelihood for certain simpler 3, 4 and 5
state models is calculated using an analytic expression for the transition
probability (P) matrix. For all other models, matrix exponentiation is used
to obtain P. To revert to the original method of using the matrix
exponential for all models, specify \code{analyticp=FALSE}. See the PDF
manual for a list of the models for which analytic P matrices are
implemented.}

\item{na.action}{What to do with missing data: either \code{na.omit} to drop
it and carry on, or \code{na.fail} to stop with an error.  Missing data
includes all NAs in the states, times, \code{subject} or \code{obstrue}, all
NAs at the first observation for a subject for covariates in
\code{initcovariates}, all NAs in other covariates (excluding the last
observation for a subject), all NAs in \code{obstype} (excluding the first
observation for a subject), and any subjects with only one observation (thus
no observed transitions).}

\item{...}{Optional arguments to the general-purpose optimisation routine,
\code{\link{optim}} by default. For example \code{method="Nelder-Mead"} to
change the optimisation algorithm from the \code{"BFGS"} method that msm
calls by default.

It is often worthwhile to normalize the optimisation using
\code{control=list(fnscale = a)}, where \code{a} is the a number of the
order of magnitude of the -2 log likelihood.

If 'false' convergence is reported and the standard errors cannot be
calculated due to a non-positive-definite Hessian, then consider tightening
the tolerance criteria for convergence. If the optimisation takes a long
time, intermediate steps can be printed using the \code{trace} argument of
the control list. See \code{\link{optim}} for details.

For the Fisher scoring method, a \code{control} list can be supplied in the
same way, but the only supported options are \code{reltol}, \code{trace} and
\code{damp}.  The first two are used in the same way as for
\code{\link{optim}}.  If the algorithm fails with a singular information
matrix, adjust \code{damp} from the default of zero (to, e.g. 1).  This adds
a constant identity matrix multiplied by \code{damp} to the information
matrix during optimisation.}
}
\value{
To obtain summary information from models fitted by the
\code{\link{msm}} function, it is recommended to use extractor functions
such as \code{\link{qmatrix.msm}}, \code{\link{pmatrix.msm}},
\code{\link{sojourn.msm}}, \code{\link{msm.form.qoutput}}.  These provide
estimates and confidence intervals for quantities such as transition
probabilities for given covariate values.

For advanced use, it may be necessary to directly use information stored in
the object returned by \code{\link{msm}}.  This is documented in the help
page \code{\link{msm.object}}.

Printing a \code{msm} object by typing the object's name at the command line
implicitly invokes \code{\link{print.msm}}.  This formats and prints the
important information in the model fit, and also returns that information in
an R object.  This includes estimates and confidence intervals for the
transition intensities and (log) hazard ratios for the corresponding
covariates. When there is a hidden Markov model, the chief information in
the \code{hmodel} component is also formatted and printed. This includes
estimates and confidence intervals for each parameter.
}
\description{
Fit a continuous-time Markov or hidden Markov multi-state model by maximum
likelihood. Observations of the process can be made at arbitrary times, or
the exact times of transition between states can be known.  Covariates can
be fitted to the Markov chain transition intensities or to the hidden Markov
observation process.
}
\details{
For full details about the methodology behind the \pkg{msm} package, refer
to the PDF manual \file{msm-manual.pdf} in the \file{doc} subdirectory of
the package. This includes a tutorial in the typical use of \pkg{msm}.  The
paper by Jackson (2011) in Journal of Statistical Software presents the
material in this manual in a more concise form.

\pkg{msm} was designed for fitting \emph{continuous-time} Markov models,
processes where transitions can occur at any time.  These models are defined
by \emph{intensities}, which govern both the time spent in the current state
and the probabilities of the next state.  In \emph{discrete-time models},
transitions are known in advance to only occur at multiples of some time
unit, and the model is purely governed by the probability distributions of
the state at the next time point, conditionally on the state at the current
time. These can also be fitted in \pkg{msm}, assuming that there is a
continuous-time process underlying the data.  Then the fitted transition
probability matrix over one time period, as returned by
\code{pmatrix.msm(...,t=1)} is equivalent to the matrix that governs the
discrete-time model. However, these can be fitted more efficiently using
multinomial logistic regression, for example, using \code{multinom} from the
R package \pkg{nnet} (Venables and Ripley, 2002).

For simple continuous-time multi-state Markov models, the likelihood is
calculated in terms of the transition intensity matrix \eqn{Q}. When the
data consist of observations of the Markov process at arbitrary times, the
exact transition times are not known.  Then the likelihood is calculated
using the transition probability matrix \eqn{P(t) = \exp(tQ)}{P(t) =
exp(tQ)}, where \eqn{\exp}{exp} is the matrix exponential.  If state \eqn{i}
is observed at time \eqn{t} and state \eqn{j} is observed at time \eqn{u},
then the contribution to the likelihood from this pair of observations is
the \eqn{i,j} element of \eqn{P(u - t)}. See, for example, Kalbfleisch and
Lawless (1985), Kay (1986), or Gentleman \emph{et al.} (1994).

For hidden Markov models, the likelihood for an individual with \eqn{k}
observations is calculated directly by summing over the unknown state at
each time, producing a product of \eqn{k} matrices. The calculation is a
generalisation of the method described by Satten and Longini (1996), and
also by Jackson and Sharples (2002), and Jackson \emph{et al.} (2003).

There must be enough information in the data on each state to estimate each
transition rate, otherwise the likelihood will be flat and the maximum will
not be found.  It may be appropriate to reduce the number of states in the
model, the number of allowed transitions, or the number of covariate
effects, to ensure convergence.  Hidden Markov models, and situations where
the value of the process is only known at a series of snapshots, are
particularly susceptible to non-identifiability, especially when combined
with a complex transition matrix.  Choosing an appropriate set of initial
values for the optimisation can also be important.  For flat likelihoods,
'informative' initial values will often be required.  See the PDF manual for
other tips.
}
\examples{

### Heart transplant data
### For further details and background to this example, see
### Jackson (2011) or the PDF manual in the doc directory.
print(cav[1:10,])
twoway4.q <- rbind(c(-0.5, 0.25, 0, 0.25), c(0.166, -0.498, 0.166, 0.166),
c(0, 0.25, -0.5, 0.25), c(0, 0, 0, 0))
statetable.msm(state, PTNUM, data=cav)
crudeinits.msm(state ~ years, PTNUM, data=cav, qmatrix=twoway4.q)
cav.msm <- msm( state ~ years, subject=PTNUM, data = cav,
                 qmatrix = twoway4.q, deathexact = 4, 
                 control = list ( trace = 2, REPORT = 1 )  )
cav.msm
qmatrix.msm(cav.msm)
pmatrix.msm(cav.msm, t=10)
sojourn.msm(cav.msm)

}
\references{
Jackson, C.H. (2011). Multi-State Models for Panel Data: The msm
Package for R., Journal of Statistical Software, 38(8), 1-29. URL
http://www.jstatsoft.org/v38/i08/.

Kalbfleisch, J., Lawless, J.F., The analysis of panel data under a Markov
assumption \emph{Journal of the Americal Statistical Association} (1985)
80(392): 863--871.

Kay, R.  A Markov model for analysing cancer markers and disease states in
survival studies.  \emph{Biometrics} (1986) 42: 855--865.

Gentleman, R.C., Lawless, J.F., Lindsey, J.C. and Yan, P.  Multi-state
Markov models for analysing incomplete disease history data with
illustrations for HIV disease.  \emph{Statistics in Medicine} (1994) 13(3):
805--821.

Satten, G.A. and Longini, I.M.  Markov chains with measurement error:
estimating the 'true' course of a marker of the progression of human
immunodeficiency virus disease (with discussion) \emph{Applied Statistics}
45(3): 275-309 (1996)

Jackson, C.H. and Sharples, L.D. Hidden Markov models for the onset and
progression of bronchiolitis obliterans syndrome in lung transplant
recipients \emph{Statistics in Medicine}, 21(1): 113--128 (2002).

Jackson, C.H., Sharples, L.D., Thompson, S.G. and Duffy, S.W. and Couto, E.
Multi-state Markov models for disease progression with classification error.
\emph{The Statistician}, 52(2): 193--209 (2003)

Titman, A.C. and Sharples, L.D.  Semi-Markov models with phase-type sojourn
distributions.  \emph{Biometrics} 66, 742-752 (2009).

Venables, W.N. and Ripley, B.D. (2002) \emph{Modern Applied Statistics with
S}, second edition. Springer.
}
\seealso{
\code{\link{simmulti.msm}}, \code{\link{plot.msm}},
\code{\link{summary.msm}}, \code{\link{qmatrix.msm}},
\code{\link{pmatrix.msm}}, \code{\link{sojourn.msm}}.
}
\author{
C. H. Jackson \email{chris.jackson@mrc-bsu.cam.ac.uk}
}
\keyword{models}
