This thesis is a monograph on Markov chains and deterministic approximation schemes that
enable the quantitative analysis thereof. We present schemes that yield approximations of the
time-varying law of a Markov chain, of its stationary distributions, and of the exit distributions
and occupation measures associated with its exit times. In practice, our schemes reduce to
solving systems of linear ordinary differential equations, linear programs, and semidefinite pro-
grams. We focus on the theoretical aspects of these schemes, proving convergence and providing
computable error bounds for most of them. To a lesser extent, we study their practical use,
applying them to a variety of examples and discussing the numerical issues that arise in their
implementation.