Ensemble transport adaptive importance sampling
File(s)Ensemble-Transport-Adaptive-Importance-Sampling.pdf (729.26 KB)
Published version
Author(s)
Cotter, C
Cotter, S
Russell, P
Type
Journal Article
Abstract
Markov chain Monte Carlo methods are a powerful and commonly used family ofnumerical methods for sampling from complex probability distributions. As applications of thesemethods increase in size and complexity, the need for efficient methods increases. In this paper, wepresent a particle ensemble algorithm. At each iteration, an importance sampling proposal distri-bution is formed using an ensemble of particles. A stratified sample is taken from this distributionand weighted under the posterior, a state-of-the-art ensemble transport resampling method is thenused to create an evenly weighted sample ready for the next iteration. We demonstrate that thisensemble transport adaptive importance sampling (ETAIS) method outperforms MCMC methodswith equivalent proposal distributions for low dimensional problems, and in fact shows better thanlinear improvements in convergence rates with respect to the number of ensemble members. We alsointroduce a new resampling strategy, multinomial transformation (MT), which while not as accurateas the ensemble transport resampler, is substantially less costly for large ensemble sizes, and canthen be used in conjunction with ETAIS for complex problems. We also focus on how algorithmicparameters regarding the mixture proposal can be quickly tuned to optimise performance. In partic-ular, we demonstrate this methodology’s superior sampling for multimodal problems, such as thosearising from inference for mixture models, and for problems with expensive likelihoods requiring thesolution of a differential equation, for which speed-ups of orders of magnitude are demonstrated.Likelihood evaluations of the ensemble could be computed in a distributed manner, suggesting thatthis methodology is a good candidate for parallel Bayesian computations.
Date Issued
2019-04-25
Date Acceptance
2019-01-30
ISSN
2166-2525
Publisher
Society for Industrial and Applied Mathematics
Start Page
444
End Page
471
Journal / Book Title
SIAM/ASA Journal on Uncertainty Quantification
Volume
7
Issue
2
Copyright Statement
© 2019 SIAM and ASA. Published by SIAM and ASA under the terms
of the Creative Commons 4.0 license (http://creativecommons.org/licenses/by/4.0/).
of the Creative Commons 4.0 license (http://creativecommons.org/licenses/by/4.0/).
Subjects
Science & Technology
Physical Sciences
Mathematics, Interdisciplinary Applications
Physics, Mathematical
Mathematics
Physics
MCMC
importance sampling
Bayesian
inverse problems
ensemble
resampling
ERGODICITY
MIXTURES
LANGEVIN
CHAINS
math.NA
math.NA
stat.CO
Publication Status
Published
Date Publish Online
2019-04-25