Dear Spiral user, We’re pleased to introduce Helix, https://helix.imperial.ac.uk/ Imperial’s new repository for research datasets. Please use Helix for all future dataset deposits. There is no change to Spiral - continue depositing as usual. For queries, contact the Research Data Management team at rdm-enquiries@imperial.ac.uk. Kind regards, the Spiral Team. The Spiral team
On the geometry of Stein variational gradient descent
File(s)
20-602.pdf (915.32 KB)
Published version
Author(s)
Duncan, A
Nusken, N
Szpruch, L
Type
Journal Article
Abstract
Bayesian inference problems require sampling or approximating high-dimensional probability distributions. The focus of this paper is on the recently introduced Stein variational gradient descent methodology, a class of algorithms that rely on iterated steepest descent steps with respect to a reproducing kernel Hilbert space norm. This construction leads to interacting particle systems, the mean field limit of which is a gradient flow on the space of probability distributions equipped with a certain geometrical structure. We leverage this viewpoint to shed some light on the convergence properties of the algorithm, in particular addressing the problem of choosing a suitable positive definite kernel function. Our analysis leads us to considering certain nondifferentiable kernels with adjusted tails. We demonstrate significant performance gains of these in various numerical experiments.
Date Issued
2023
Date Acceptance
2023-01-01
Citation
Journal of Machine Learning Research, 2023, 24 (56), pp.1-39
URI
URL
ISSN
1532-4435
Publisher
Microtome Publishing
Start Page
1
End Page
39
Journal / Book Title
Journal of Machine Learning Research
Volume
24
Issue
56
Copyright Statement
©2023 Andrew Duncan, Nikolas N¨usken and Lukasz Szpruch.
License: CC-BY 4.0, see https://creativecommons.org/licenses/by/4.0/. Attribution requirements are provided at
http://jmlr.org/papers/v24/20-602.html.
License URL
Identifier
https://jmlr.org/papers/v24/20-602.html
Subjects
Automation & Control Systems
Bayesian inference
Computer Science
Computer Science, Artificial Intelligence
CONVEXITY
DIFFUSION
DYNAMICS
EQUATIONS
EULERIAN CALCULUS
geometry of optimal transport
gradient flows
LIMIT
PRINCIPLE
reproducing kernel Hilbert spaces
Science & Technology
Stein's method
SYSTEMS
Technology
TRANSPORTATION
Publication Status
Published
Date Publish Online
2023-01-01