On inference for ABC approximations of time series models

Abstract

Markov chain Monte Carlo (MCMC) and Sequential Monte Carlo (SMC) are well-studied simulation based methods that have been very successful for a variety of complex inference problems. However, standard MCMC and SMC cannot deal with problems where the corresponding likelihood is intractable. By intractable we mean that the density of the likelihood cannot be calculated analytically or it is too computationally expensive to do so. This motivated the use of Approximate Bayesian Computation (ABC) as far back as thirty years ago, but only in the last ten years has its popularity surged due to the higher computational power available in modern computers. The focus of this thesis is to use simulation to estimate the density of the intractable likelihoods as in ABC methods and combine with MCMC and SMC methods. The flexibility and accuracy of the latter can be used to address general issues regarding inference for time series and ABC techniques can be used to deal with the intractable conditional observation likelihoods. The times series we will investigate in this thesis are Hidden Markov Models (HMMs) and observation driven time series models. SMC and ABC methods have been used to perform filtering and smoothing for intractable HMMs [Jasra et al., 2012]. We will present a method that combines SMC and ABC, and uses gradient ascent to deliver maximum likelihood estimates for the model parameters. The accuracy of the method will be illustrated by theoretical results and challenging numerical examples. In regard to estimating model parameters for other time series such as intractable observation driven models, we consider methods that use MCMC together with ABC approximations. These methods can be used to perform Bayesian estimation of the model parameters. However applying ideas from ABC in this context naively will prevent the exploration of difficult regions in the parameter space. Difficult regions arise, for example, when attempting ABC approximations very close to the true model. Based on recent ideas from [Lee, 2012], we identify a new unbiased estimator for the ABC likelihood which is more robust and able to explore the difficult state-space regions. The advantage of the resulting algorithm is that it is capable of producing very accurate inference. Often one is interested in estimating the initial condition of dynamic systems based on noisy observations of its evolution. These types of inverse problems are often formulated as Bayesian inference problems. The prior used in Bayesian inference can be interpreted also as a regularization term to ensure the problem is well-posed. We show how SMC together with ABC approximations can be used for estimating the initial condition of deterministic dynamical systems that are observed with additive noise whose density is intractable. We present a method which uses SMC samplers as in [Chopin, 2002, Del Moral et al., 2006] with ABC approximations, and is applied to challenging numerical examples from data assimilation.