WIAS Preprint No. 1907, (2013)

Forward-reverse EM algorithm for Markov chains


  • Bayer, Christian
    ORCID: 0000-0002-9116-0039
  • Mai, Hilmar
  • Schoenmakers, John G. M.
    ORCID: 0000-0002-4389-8266

2010 Mathematics Subject Classification

  • 65C05 60J20


  • forward-reverse representations, EM algorithm, Monte Carlo simulation, maximum likelihood estimation, Markov chain estimation




We develop an EM algorithm for estimating parameters that determine the dynamics of a discrete time Markov chain evolving through a certain measurable state space. As a key tool for the construction of the EM method we also develop forward-reverse representations for Markov chains conditioned on a certain terminal state. These representations may be considered as an extension of the earlier work of Bayer and Schoenmakers (2013) on conditional diffusions. We present several experiments and consider the convergence of the new EM algorithm.

Appeared in

  • Adv. Appl. Probab., 2 (2018), pp. 621--644 , under the new title: Forward-reverse expectation-maximization algorithm for Markov chains: Convergence and numerical analysis, DOI 10.1017/apr.2018.27 .

Download Documents