A Heterogeneous Sensor Suite for Reducing the Cognitive Burden of Upper Limb Robotic Control

Abstract

A Human machine interface (HMI) acts like a bridge between motor function and the brain. Bypassing these natural pathways allows disabled individuals to perform actions that might otherwise be too difficult or impossible. The loss or impairment of voluntary muscle control can have a detrimental effect on an individual’s quality of life. By utilising natural physiological motion with sensor fusion techniques, grasp intention can be predicted, leading to an assistive HMI concept that can reduce the cognitive load that traditional HMIs impose on the user. This thesis investigates a novel design concept for a heterogeneous sensor suite, by fusing mechanomyogram (MMG) sensors for muscle activation, computer vision for object recognition, and inertial measurement sensors for predicting grasp intention. The developed architecture focuses on the prediction of intentional grasp activity of 1 amputee and 10 healthy subjects, using the natural physiological motion of the arm when reaching to grasp 3 objects with up to 3 different grasp patterns. 84 motion features are extracted and used as a classification tool for predicting intention, yielding an average grasp classification accuracy of 100%, 82.5% and 88.9% for bottle, lid and box objects across all subjects. The novel heterogeneous sensor suite is applied to automate the grasp control of a myoelectric hand prosthesis. Real-time task-based experiments evaluated the performance of the proposed system, comparing it against conventional control using MMG sensors, yielding an 8.5% average faster completion time, as well as a reduction in overall cognitive and physical burden. The results of this research provide excellent potential for the use of natural motion to replace discrete muscle input as a selection and intention prediction tool, where the emphasis is on reducing the cognitive load imposed by assistive technologies.