IRUS Total

Minimum Stein discrepancy estimators

File Description SizeFormat 
9457-minimum-stein-discrepancy-estimators.pdfPublished version485.33 kBAdobe PDFView/Open
Title: Minimum Stein discrepancy estimators
Authors: Barp, A
Briol, FX
Duncan, A
Girolami, M
Mackey, L
Item Type: Conference Paper
Abstract: When maximum likelihood estimation is infeasible, one often turns to score matching, contrastive divergence, or minimum probability flow to obtain tractable parameter estimates. We provide a unifying perspective of these techniques as minimum Stein discrepancy estimators, and use this lens to design new diffusion kernel Stein discrepancy (DKSD) and diffusion score matching (DSM) estimators with complementary strengths. We establish the consistency, asymptotic normality, and robustness of DKSD and DSM estimators, then derive stochastic Riemannian gradient descent algorithms for their efficient optimisation. The main strength of our methodology is its flexibility, which allows us to design estimators with desirable properties for specific models at hand by carefully selecting a Stein discrepancy. We illustrate this advantage for several challenging problems for score matching, such as non-smooth, heavy-tailed or light-tailed densities.
Issue Date: 8-Dec-2019
Date of Acceptance: 4-Sep-2019
URI: http://hdl.handle.net/10044/1/77552
Publisher: Neural Information Processing Systems Foundation, Inc.
Journal / Book Title: NIPS Proceedings
Volume: 32
Copyright Statement: © 2019 Neural Information Processing Systems Foundation, Inc.
Sponsor/Funder: The Alan Turing Institute
Funder's Grant Number: ATI PO 000002890 R/LRF/AD1
Conference Name: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019)
Start Date: 2019-12-08
Finish Date: 2019-12-14
Conference Place: Vancouver, ON, Canada
Appears in Collections:Mathematics