Structured uncertainty in the observation space of variational autoencoders
File(s)
Published version
Author(s)
Langley, James
Monteiro, Miguel
Jones, Charles
Pawlowski, Nick
Glocker, Ben
Type
Journal Article
Abstract
Variational autoencoders (VAEs) are a popular class of deep generative models with many variants and a wide range of applications. Improvements upon the standard VAE mostly focus on the modelling of the posterior distribution over the latent space and the properties of the neural network decoder. In contrast, improving the model for the observational distribution is rarely considered and typically defaults to a pixel-wise independent categorical or normal distribution. In image synthesis, sampling from such distributions produces spatially-incoherent results with uncorrelated pixel noise, resulting in only the sample mean being somewhat useful as an output prediction. In this paper, we aim to stay true to VAE theory by improving the samples from the observational distribution. We propose SOS-VAE, an alternative model for the observation space, encoding spatial dependencies via a low-rank parameterisation. We demonstrate that this new observational distribution has the ability to capture relevant covariance between pixels, resulting in spatially-coherent samples. In contrast to pixel-wise independent distributions, our samples seem to contain semantically-meaningful variations from the mean allowing the prediction of multiple plausible outputs with a single forward pass.
Date Issued
2022
Date Acceptance
2022-10-01
Citation
Transactions on Machine Learning Research, 2022, 2022
ISSN
2835-8856
Publisher
Transactions on Machine Learning Research
Journal / Book Title
Transactions on Machine Learning Research
Volume
2022
Copyright Statement
Creative Commons Attribution 4.0 International (CC BY 4.0)
License URL
Identifier
https://openreview.net/forum?id=cxp7n9q5c4
Publication Status
Published
Article Number
201
Date Publish Online
2022-10-30