Deterministic approximation schemes with computable errors for the distributions of Markov chains

Abstract

This thesis is a monograph on Markov chains and deterministic approximation schemes that enable the quantitative analysis thereof. We present schemes that yield approximations of the time-varying law of a Markov chain, of its stationary distributions, and of the exit distributions and occupation measures associated with its exit times. In practice, our schemes reduce to solving systems of linear ordinary differential equations, linear programs, and semidefinite pro- grams. We focus on the theoretical aspects of these schemes, proving convergence and providing computable error bounds for most of them. To a lesser extent, we study their practical use, applying them to a variety of examples and discussing the numerical issues that arise in their implementation.