Deep gaussian processes with importance-weighted variational inference
File(s)
Working paper
Author(s)
Salimbeni, Hugh
Dutordoir, Vincent
Hensman, James
Deisenroth, Marc Peter
Type
Working Paper
Abstract
Deep Gaussian processes (DGPs) can model complex marginal densities as well
as complex mappings. Non-Gaussian marginals are essential for modelling
real-world data, and can be generated from the DGP by incorporating
uncorrelated variables to the model. Previous work on DGP models has introduced
noise additively and used variational inference with a combination of sparse
Gaussian processes and mean-field Gaussians for the approximate posterior.
Additive noise attenuates the signal, and the Gaussian form of variational
distribution may lead to an inaccurate posterior. We instead incorporate noisy
variables as latent covariates, and propose a novel importance-weighted
objective, which leverages analytic results and provides a mechanism to trade
off computation for improved accuracy. Our results demonstrate that the
importance-weighted objective works well in practice and consistently
outperforms classical variational inference, especially for deeper models.
as complex mappings. Non-Gaussian marginals are essential for modelling
real-world data, and can be generated from the DGP by incorporating
uncorrelated variables to the model. Previous work on DGP models has introduced
noise additively and used variational inference with a combination of sparse
Gaussian processes and mean-field Gaussians for the approximate posterior.
Additive noise attenuates the signal, and the Gaussian form of variational
distribution may lead to an inaccurate posterior. We instead incorporate noisy
variables as latent covariates, and propose a novel importance-weighted
objective, which leverages analytic results and provides a mechanism to trade
off computation for improved accuracy. Our results demonstrate that the
importance-weighted objective works well in practice and consistently
outperforms classical variational inference, especially for deeper models.
Date Issued
2019-05-14
Citation
2019
Publisher
arXiv
Copyright Statement
© 2019 The Authors.
Identifier
http://arxiv.org/abs/1905.05435v1
Subjects
stat.ML
stat.ML
cs.LG
Notes
Appearing ICML 2019
Publication Status
Published