Adaptive gesture recognition for human-robot interface using mechanomyography (MMG)

Abstract

In the past decades, hand gesture recognition (HGR) has been a widely studied topic, with researchers aiming to improve information extracted from human muscle activities. However, the majority of studies have focused on using electromyogram (EMG) as a main acquisition method, in which the sensors can face disadvantages from impedance changes during non-laboratory usage. On the other hand, interest in utilizing alternative sensors has recently increased to address the disadvantages of mainstream EMG. Among these alternative sensors, mechanomyogram (MMG) has shown promising results due to its information quality and robustness to non-laboratory environment factors.

This research presents a novel hand gesture recognition system, integrating mechanomyogram (MMG) sensors and an Inertial Measurement Unit into an upper limb wearable. MMG, which detects mechanical vibrations from muscle activities, can provide descriptive information during gesture contractions and is robust to undesired real-world noises, such as perspiration. By developing a portable system embedded with these sensors and its tailored gesture recognition pipeline, this research can offer improvements to HGR applications, including prosthesis hand control for upper-limb amputees.

In this research, the initial algorithm tailored for forearm MMG was developed and tested within a controlled environment. Explorations on forearm MMG signals were conducted along with optimization processes under various real-world usage constraints. Afterward, the study was extended to non-laboratory environment usages by incorporating unsupervised domain adaptation techniques into the pipeline to achieve adaptation on dynamic factors. The changes in muscle activities from different arm positions were used as the study topic. The developed pipeline can adapt to these changes and achieve improved prediction accuracy within limited training information.

By providing insights and practical solutions for MMG-tailored HGR developments, this work could provide a foundation for efficient and versatile systems capable of accurately recognizing hand gestures in diverse environments and for a wide range of users.