Logistic variational Bayes revisited
File(s)
Accepted version
Author(s)
Komodromos, Michael
Evangelou, Marina
Filippi, Sarah
Type
Conference Paper
Abstract
Variational logistic regression is a popular method for approximate Bayesian inference seeing wide-spread use in many areas of machine learning including: Bayesian optimization, reinforcement learning and multi-instance learning to name a few. However, due to the intractability of the Evidence Lower Bound, authors have turned to the use of Monte Carlo, quadrature or bounds to perform inference, methods which are costly or give poor approximations to the true posterior.
In this paper we introduce a new bound for the expectation of softplus function and subsequently show how this can be applied to variational logistic regression and Gaussian process classification. Unlike other bounds, our proposal does not rely on extending the variational family, or introducing additional parameters to ensure the bound is tight. In fact, we show that this bound is tighter than the state-of-the-art, and that the resulting variational posterior achieves state-of-the-art performance, whilst being significantly faster to compute than Monte-Carlo methods.
In this paper we introduce a new bound for the expectation of softplus function and subsequently show how this can be applied to variational logistic regression and Gaussian process classification. Unlike other bounds, our proposal does not rely on extending the variational family, or introducing additional parameters to ensure the bound is tight. In fact, we show that this bound is tighter than the state-of-the-art, and that the resulting variational posterior achieves state-of-the-art performance, whilst being significantly faster to compute than Monte-Carlo methods.
Date Acceptance
2024-05-02
Publisher
ICML
Identifier
https://arxiv.org/abs/2406.00713
Source
International Conference on Machine Learning (ICML 2024)
Publication Status
Accepted
Start Date
2024-07-21
Finish Date
2024-07-27
Coverage Spatial
Vienna, Austria
Date Publish Online
2024-07-03