Reinforcement learning controls of functional electrical stimulation for movement restoration
File(s)
Author(s)
Wannawas, Nat
Type
Thesis
Abstract
Hope prevails for the restoration of movement in paralysed individuals, spurred by advancements in neural engineering and AI. Functional Electrical Stimulation (FES) stands out as a non-invasive avenue to restore lost motor functions. However, despite over a century of research, FES has yet to fully realise its primary potential to restore general movements. Challenges in controls and translation into practical use, especially in the absence of technical assistance, persist. Reinforcement Learning (RL) emerges as a general method for controlling FES, capable of tailoring itself to users. Yet, applying RL to human-in-the-loop systems entails key challenges including the need for large data quantity and ethical concerns. This thesis explores RL applications in FES controls, seeking to illustrate its capability and address the key challenges. This work lays the foundation to harness RL's potential in this domain, aspiring to make FES control genuinely effective in the daily lives of those who need it. This has led to contributions within the areas of FES control and machine learning. Four detailed neuromechanical models and a muscle fatigue model for FES control applications are proposed. Next, two RL setups for controlling cycling speed and finding stimulation patterns are introduced, followed by RL setups for controlling arbitrary arm movements and the investigation of their learning behaviours and performances. This is followed by the introduction of Gaussian state space model that allows RL to better deal with muscle fatigue. Next, two data-efficient probabilistic modelling methods that leverage both physics knowledge and data are proposed. These models substantially help improve data efficiency of dyna-RL. Finally, the real-world deployment of RL is outlined, considering both ethical concerns and system designs, followed by experimental setups and the results. The successes in real-world deployment present the feasibility of RL and unlock further developments which are outlined in the conclusion.
Version
Open Access
Date Issued
2024-01
Date Awarded
2024-08
Copyright Statement
Creative Commons Attribution NonCommercial Licence
License URL
Advisor
Faisal, Aldo
Sponsor
Thailand
Publisher Department
Bioengineering
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)