Learning predictive cognitive maps with spiking neurons during behaviour and replays
File(s)
Published version
Author(s)
Bono, Jacopo
Zannone, Sara
Pedrosa, Victor
Clopath, Claudia
Type
Journal Article
Abstract
The hippocampus has been proposed to encode environments using a representation that contains predictive information about likely future states, called the successor representation. However, it is not clear how such a representation could be learned in the hippocampal circuit. Here, we propose a plasticity rule that can learn this predictive map of the environment using a spiking neural network. We connect this biologically plausible plasticity rule to reinforcement learning, mathematically and numerically showing that it implements the TD-lambda algorithm. By spanning these different levels, we show how our framework naturally encompasses behavioral activity and replays, smoothly moving from rate to temporal coding, and allows learning over behavioral timescales with a plasticity rule acting on a timescale of milliseconds. We discuss how biological parameters such as dwelling times at states, neuronal firing rates and neuromodulation relate to the delay discounting parameter of the TD algorithm, and how they influence the learned representation. We also find that, in agreement with psychological studies and contrary to reinforcement learning theory, the discount factor decreases hyperbolically with time. Finally, our framework suggests a role for replays, in both aiding learning in novel environments and finding shortcut trajectories that were not experienced during behavior, in agreement with experimental data.
Date Issued
2023-03-16
Date Acceptance
2023-01-12
Citation
eLife, 2023, 12
ISSN
2050-084X
Publisher
eLife Sciences Publications Ltd
Journal / Book Title
eLife
Volume
12
Copyright Statement
© 2023, Bono, Zannone et al. This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
License URL
Publication Status
Published
Article Number
ARTN e80671