Learning in deep factor graphs with Gaussian belief propagation
File(s)
Published version
OA Location
Author(s)
Nabarro, S
van der Wilk, M
Davison, AJ
Type
Conference Paper
Abstract
We propose an approach to do learning in Gaussian factor graphs. We treat all relevant quantities (inputs, outputs, parameters, activations) as random variables in a graphical model, and view training and prediction as inference problems with different observed nodes. Our experiments show that these problems can be efficiently solved with belief propagation (BP), whose updates are inherently local, presenting exciting opportunities for distributed and asynchronous training. Our approach can be scaled to deep networks and provides a natural means to do continual learning: use the BP-estimated posterior of the current task as a prior for the next. On a video denoising task we demonstrate the benefit of learnable parameters over a classical factor graph approach and we show encouraging performance of deep factor graphs for continual image classification.
Date Issued
2024-07-21
Date Acceptance
2024-07-01
Citation
Proceedings of Machine Learning Research, 2024, 235, pp.37141-37163
ISSN
2640-3498
Publisher
MLResearchPress
Start Page
37141
End Page
37163
Journal / Book Title
Proceedings of Machine Learning Research
Volume
235
Copyright Statement
Copyright © The authors and PMLR 2024.
Sponsor
Engineering & Physical Science Research Council (EPSRC)
Grant Number
EP/S036636/1
Source
41st International Conference on Machine Learning
Publication Status
Published
Start Date
2024-07-21
Finish Date
2024-07-27
Coverage Spatial
Vienna, Austria