Understanding the hand: robotics, virtual reality and the neuroscience of control
File(s)
Author(s)
Konnaris, Charalambos
Type
Thesis or dissertation
Abstract
Our primary interaction with the world is through the spontaneous generation of movement. In particular, the hand is a critical tool allowing humans to perform a myriad of dexterous manipulations. The brain's ability to control and coordinate movement across the multiple joints of the hand seems effortless and it remains unrivalled by any other artificial system when confronted with such a high dimensional-structure. Thus, replicating the naturalistic control functionality in a prosthetic hand poses a major challenge. While previous research focused on studying the hand in constrained laboratory conditions, this thesis is focused on analysing hand movements obtained from free behaviour.
Using various statistical analyses, the kinematic structure of hand movements is examined across participants. By exploiting the kinematic correlation of natural hand movements, we demonstrated the capability of a data driven method to efficiently reconstruct missing digits using information from intact fingers. Subsequently, to overcome the laborious process required to effectively label free behaviour, a semi-supervised approach was developed allowing us to accurately measure the frequency and duration of various grasps throughout everyday life. Through further analysis, we found no significant differences in terms of movement complexity between dominant and non-dominant hand. Additionally, we found that a small number of kinematic synergies are consisted across participants yet the rest of their kinematic spaces seemed to be dominated by individuality.
To address these findings, we developed an ecologically valid Virtual Reality (VR) platform (EthoPlatform) that directly involves the human in the control loop. Using EthoPlatform, subjects' performances were objectively quantified while their joint angles were manipulated through dimensionality reduction algorithms and reconstructed on the VR hand in real-time. Using this paradigm, we tested the ability of personalised models versus models trained on databases of aggregate data. We found that models from dominant and non-dominant hand produce similar performances as opposed to models trained on an aggregate of data. Subsequently, we examined the impact of non-linear dimensionality reduction algorithms on human performance. From this experiment, we found that reconstruction error is directly related to subjects' performances. These findings are of high importance for the development of prosthetic hand controllers, as we now can create the most tailored model to control the prosthesis via the dataset acquired from the non-amputated hand. Furthermore, we can deploy non-linear models that can better capture the kinematic structure of hand movements while requiring less information.
To validate the results obtained from the VR experiments as well as to close the gap between experimental paradigms and real world applications, a dexterous artificial hand (EthoHand) was developed. Contrary to other robot hands that focus on grasping, EthoHand was developed to facilitate complex in-hand manipulations. For this reason, an appropriate thumb articulation was implemented allowing EthoHand to go beyond grasping and to perform multi-planar thumb movements similar to the real hand. Lastly, the ability of dimensionality reduction algorithms to successfully control a dexterous artificial hand was demonstrated on the physical hand.
The findings throughout this thesis demonstrate a coherent approach for analysing the complex structure of natural hand movements. Furthermore, an alternative avenue towards developing superior prosthetic hand controllers is provided through personalised models.
Using various statistical analyses, the kinematic structure of hand movements is examined across participants. By exploiting the kinematic correlation of natural hand movements, we demonstrated the capability of a data driven method to efficiently reconstruct missing digits using information from intact fingers. Subsequently, to overcome the laborious process required to effectively label free behaviour, a semi-supervised approach was developed allowing us to accurately measure the frequency and duration of various grasps throughout everyday life. Through further analysis, we found no significant differences in terms of movement complexity between dominant and non-dominant hand. Additionally, we found that a small number of kinematic synergies are consisted across participants yet the rest of their kinematic spaces seemed to be dominated by individuality.
To address these findings, we developed an ecologically valid Virtual Reality (VR) platform (EthoPlatform) that directly involves the human in the control loop. Using EthoPlatform, subjects' performances were objectively quantified while their joint angles were manipulated through dimensionality reduction algorithms and reconstructed on the VR hand in real-time. Using this paradigm, we tested the ability of personalised models versus models trained on databases of aggregate data. We found that models from dominant and non-dominant hand produce similar performances as opposed to models trained on an aggregate of data. Subsequently, we examined the impact of non-linear dimensionality reduction algorithms on human performance. From this experiment, we found that reconstruction error is directly related to subjects' performances. These findings are of high importance for the development of prosthetic hand controllers, as we now can create the most tailored model to control the prosthesis via the dataset acquired from the non-amputated hand. Furthermore, we can deploy non-linear models that can better capture the kinematic structure of hand movements while requiring less information.
To validate the results obtained from the VR experiments as well as to close the gap between experimental paradigms and real world applications, a dexterous artificial hand (EthoHand) was developed. Contrary to other robot hands that focus on grasping, EthoHand was developed to facilitate complex in-hand manipulations. For this reason, an appropriate thumb articulation was implemented allowing EthoHand to go beyond grasping and to perform multi-planar thumb movements similar to the real hand. Lastly, the ability of dimensionality reduction algorithms to successfully control a dexterous artificial hand was demonstrated on the physical hand.
The findings throughout this thesis demonstrate a coherent approach for analysing the complex structure of natural hand movements. Furthermore, an alternative avenue towards developing superior prosthetic hand controllers is provided through personalised models.
Version
Open Access
Date Issued
2019-09
Date Awarded
2020-08
Copyright Statement
Creative Commons Attribution NonCommercial Licence
Advisor
Faisal, Aldo
Publisher Department
Bioengineering
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)