Stationary distributions of continuous-time Markov chains: a review of theory and truncation-based approximations
File(s)M128962.pdf (887.95 KB)
Working paper
Author(s)
Kuntz, J
Thomas, P
Stan, G-B
Barahona, M
Type
Working Paper
Abstract
Computing the stationary distributions of a continuous-time Markov chain involves solving a set of linear equations. In most cases of interest, the number of equations is infinite or too large, and cannot be solved analytically or numerically. Several approximation schemes overcome this issue by truncating the state space to a manageable size. In this review, we first give a comprehensive theoretical account of the stationary distributions and their relation to the long-term behaviour of the Markov chain, which is readily accessible to non-experts and free of irreducibility assumptions made in standard texts. We then review truncation-based approximation schemes paying particular attention to their convergence and to the errors they introduce, and we illustrate their performance with an example of a stochastic reaction network of relevance in biology and chemistry. We conclude by elaborating on computational trade-offs associated with error control and some open questions.
Date Issued
2019-09-12
Online Publication Date
2020-08-21T14:41:55Z
ISSN
0036-1445
Publisher
arXiv
Copyright Statement
© 2020 The Author(s)
Sponsor
Engineering & Physical Science Research Council (EPSRC)
Identifier
http://arxiv.org/abs/1909.05794v1
Grant Number
EP/N014529/1
Subjects
math.PR
math.PR
cond-mat.stat-mech
math.OC
q-bio.MN
q-bio.PE
60J27 (Primary), 60J22, 65C40, 90C05, 90C90 (Secondary)
math.PR
math.PR
cond-mat.stat-mech
math.OC
q-bio.MN
q-bio.PE
60J27 (Primary), 60J22, 65C40, 90C05, 90C90 (Secondary)
Numerical & Computational Mathematics
0102 Applied Mathematics
0906 Electrical and Electronic Engineering