Data-driven reverse engineering of visuomotor mechanisms
File(s)
Author(s)
Harston, John
Type
Thesis or dissertation
Abstract
The visuomotor system is critical to every interaction we make with our environments, yet much remains unknown, even superficially, about the natural dynamics and mechanisms of freely-moving visuomotor behaviour. Whilst the relationship between vision and action in real world tasks is highly complex and nonlinear, recent advances in machine learning and computer vision are beginning to facilitate investigation of this relationship. Visual and motor behaviours in humans are tightly spatiotemporally coupled, yet relatively few studies of freely-moving gaze dynamics account for the kinematics of the body, thereby ignoring a fundamental component of the perception-action visuomotor loop.
In this thesis I take a data-driven approach to analyse visuomotor behaviour at scale, investigating both the structure of natural behaviour in real-world tasks and the complex dynamics of the visuomotor repertoire. I analyse both the inherent internal dynamics of behaviour, and the effect of extrinsic environmental context (gaze-objects). I also showcase the tooling I have begun to build in this space, to improve the software infrastructure available to the scientific community for real-world visuomotor behaviour recordings. I showcase early evidence of the existence of a predictive relationship between gaze behaviour and whole-body kinematics, and dissect this further. The work detailed in this thesis contributes to a better data-driven understanding of behaviour, and hopefully helps us in beginning to answer some of these difficult questions. This work aids both a basic understanding of visuomotor structure, and offers insight that may help researchers to build translational neurotechnological applications for those with motor disabilities.
In this thesis I take a data-driven approach to analyse visuomotor behaviour at scale, investigating both the structure of natural behaviour in real-world tasks and the complex dynamics of the visuomotor repertoire. I analyse both the inherent internal dynamics of behaviour, and the effect of extrinsic environmental context (gaze-objects). I also showcase the tooling I have begun to build in this space, to improve the software infrastructure available to the scientific community for real-world visuomotor behaviour recordings. I showcase early evidence of the existence of a predictive relationship between gaze behaviour and whole-body kinematics, and dissect this further. The work detailed in this thesis contributes to a better data-driven understanding of behaviour, and hopefully helps us in beginning to answer some of these difficult questions. This work aids both a basic understanding of visuomotor structure, and offers insight that may help researchers to build translational neurotechnological applications for those with motor disabilities.
Version
Open Access
Date Issued
2022-12-13
Date Awarded
2023-03-01
Copyright Statement
Attribution-NonCommercial 4.0 International Licence (CC BY-NC)
Advisor
Faisal, A. Aldo
Sponsor
Engineering and Physical Sciences Research Council
Publisher Department
Department of Bioengineering
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)