On the theory and applications of stochastic gradient descent in continuous time

Abstract

Stochastic optimisation problems are ubiquitous across machine learning, engineering, the natural sciences, economics, and operational research. One of the most popular and widely used methods for solving such problems is stochastic gradient descent. In this thesis, we study the theoretical properties and the applications of stochastic gradient descent in continuous time.

We begin by analysing the asymptotic properties of two-timescale stochastic gradient descent in continuous time, extending well known results in discrete time. The proposed algorithm, which arises naturally in the context of stochastic bilevel optimisation problems, consists of two coupled stochastic recursions which evolve on different timescales. Under weak and classical assumptions, we establish the almost sure convergence of this algorithm, and obtain an asymptotic convergence rate.

We next illustrate how the proposed algorithm can be applied to an important problem arising in continuous-time state-space models: joint online parameter estimation and optimal sensor placement. Under suitable conditions on the process consisting of the latent signal process, the filter, and the filter derivatives, we establish almost sure convergence of the online parameter estimates and optimal sensor placements generated by our algorithm to the stationary points of the asymptotic log-likelihood of the observations, and the asymptotic covariance of the state estimate, respectively. We also provide extensive numerical results illustrating the performance of our approach in the case that the hidden signal is governed by the two-dimensional stochastic advection-diffusion equation, a model arising in many meteorological and environmental monitoring applications.

In the final part of this thesis, we introduce a continuous-time stochastic gradient descent algorithm for recursive estimation of the parameters of a stochastic McKean-Vlasov equation equation, and the associated system of interacting particles. Such models arise in a variety of applications, including statistical physics, mathematical biology, and the social sciences. We prove that our estimator converges in L1 to the stationary points of the asymptotic log-likelihood of the McKean-Vlasov SDE in the joint limit as t and the number of particles N go to infinity, under suitable conditions which guarantee ergodicity and uniform-in-time propagation of chaos. We also establish, assuming also strong concavity for the asymptotic log-likelihood, an L2 convergence rate to the unique maximiser of this asymptotic log-likelihood function. Our theoretical results are demonstrated via a range of numerical examples, including a stochastic Kuramoto model and a stochastic opinion dynamics model.