Structured neural learning for autonomous systems
File(s)
Author(s)
Sandoval, Ilya Orson
Type
Thesis
Abstract
This thesis investigates the application of structured neural networks for modelling and controlling complex dynamical systems, prioritising interpretability, efficiency, and practical utility. Two complementary approaches are investigated at the inter-section of neural networks, non-linear control theory, and reinforcement learning. Part one explores the use of Neural Ordinary Differential Equations (Neural ODEs) as continuous-time feedback policies for non-linear optimal control. This framework directly incorporates state and control constraints, a critical aspect often challenging for traditional methods. The effectiveness of these constrained neural controllers was demonstrated on benchmark systems, including the Van der Pol oscillator and a bioreactor control problem.
Part two extends the principle of exploiting inherent structure to reinforcement learning (RL) for autonomous cyber defence. To address the limitation that existing RL agents often overlook the critical graph structure of computer networks, agents based on Graph Attention Networks (GATs) were developed. Operating within a custom simulation environment where network states are represented as graphs, these GAT-based policies are able to leverage topological information. This results in defensive strategies demonstrating enhanced robustness to dynamic network changes, better generalisation across network size variations with similar structures, and more interpretable actions, marking a step towards more capable autonomous cyber defence systems.
In summary, this thesis contributes novel methodologies enabled by differentiable programming, showcasing how strategically structuring neural networks, ranging from continuous-time Neural ODEs for optimal control to graph-aware RL policies for cyber defence with GATs, yields effective and efficient solutions by exploiting the inherent structure of the problem domains.
Part two extends the principle of exploiting inherent structure to reinforcement learning (RL) for autonomous cyber defence. To address the limitation that existing RL agents often overlook the critical graph structure of computer networks, agents based on Graph Attention Networks (GATs) were developed. Operating within a custom simulation environment where network states are represented as graphs, these GAT-based policies are able to leverage topological information. This results in defensive strategies demonstrating enhanced robustness to dynamic network changes, better generalisation across network size variations with similar structures, and more interpretable actions, marking a step towards more capable autonomous cyber defence systems.
In summary, this thesis contributes novel methodologies enabled by differentiable programming, showcasing how strategically structuring neural networks, ranging from continuous-time Neural ODEs for optimal control to graph-aware RL policies for cyber defence with GATs, yields effective and efficient solutions by exploiting the inherent structure of the problem domains.
Version
Open Access
Date Issued
2025-07-05
Date Awarded
2026-04-01
Copyright Statement
Attribution-NonCommercial 4.0 International Licence (CC BY-NC)
License URL
Advisor
del Rio Chanona, Ehecatl Antonio
Publisher Department
Department of Chemical Engineering
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)