The dynamics of continuous-time recurrent neural networks and their relevance to episodic memory

Abstract

Continuous-time recurrent neural networks (CTRNNs) are both plausible models of cortical circuits and practically relevant for biologically inspired machine learning. To date, their dynamics are not well understood, especially when they are driven by external signals, and are hence typically treated as a “black box”. This work builds on previous attempts to better understand the complex dynamics of CTRNNs. A series of computational experiments are presented whose aim is to reveal, through both visualisation and analysis, the dynamical transitions exhibited by these networks when driven by external periodic signals. On the applications front, the suitability of CTRNNs for episodic-like memory is explored, using the reservoir computing paradigm. This paradigm has emerged as a simple approach to overcoming important limitations of the gradient-based methods conventionally used to train recurrent neural networks. To explore their applicability to episodic-like memory, CTRNNs are trained to memorise and reconstruct sequences of high-dimensional images, representing episodic experiences. The various parameters affecting network performance are investigated, revealing a strong dependence on the dynamic regime the networks operate in. Finally, a simple computational experiment is presented using CTRNNs with modular, small-world connectivity. The aim is to visualise information broadcast in complex networks, as well as to provide computational support for the plausibility of the “connective core hypothesis”, which posits a particular relationship between connectivity and information flow in the brain. The experiments and findings presented in this work offer a deeper and more intuitive understanding of the dynamics of continuous-time recurrent neural networks, and open up new avenues for future research in both their theory and application.