Graph expansions of neural networks: theory and applications for time-series analysis
File(s)
Author(s)
Muça Cirone, Nicola
Type
Thesis
Abstract
The study of scaling limits in neural networks, particularly as the number of parameters grows large, provides crucial insights into their generalization properties, training stability, and information processing efficiency, making it an area of significant empirical interest.
However, existing approaches often lack mathematical rigor and are specific to certain architectures and hyper-parameters.
Our research introduces a novel mathematical framework for analyzing the scaling limits of arbitrary architectures.
By utilizing genus expansion techniques from random matrix theory, we represent neural networks as Taylor-like series of operator graphs, separating the effects of activation functions from the network’s random weights.
We then leverage these expansions to study the statistical properties of Feed Forward Neural Networks, State Space Models, and Neural Controlled Differential Equations, discovering new expressive kernels on path spaces.
However, existing approaches often lack mathematical rigor and are specific to certain architectures and hyper-parameters.
Our research introduces a novel mathematical framework for analyzing the scaling limits of arbitrary architectures.
By utilizing genus expansion techniques from random matrix theory, we represent neural networks as Taylor-like series of operator graphs, separating the effects of activation functions from the network’s random weights.
We then leverage these expansions to study the statistical properties of Feed Forward Neural Networks, State Space Models, and Neural Controlled Differential Equations, discovering new expressive kernels on path spaces.
Version
Open Access
Date Issued
2025-09-14
Date Awarded
01/12/2025
License URL
Advisor
Salvi, Cristopher
Publisher Department
Department of Mathematics
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)