Hand gesture recognition for minimally invasive surgery.

Abstract

Laparoscopic surgery, a form of Minimally Invasive Surgery (MIS), presents many challenges to surgeons in terms of instrument control, manual dexterity, and hand-eye coordination. The current training and assessment regime is subjective and there is an increasing demand for objective measurement of surgical skills. Existing research into laparoscopic surgery reveals many psychomotor challenges to the surgeon, highlighting the importance of adequate training and assessment of laparoscopic skills. This thesis addresses the problem of objective assessment of surgical competency by developing a Body Sensor Network (BSN) Sensor Glove (BSG) for laparoscopic gesture recognition and surgical skills evaluation. An examination of hand gestures related to surgical manoeuvres performed by surgeons during laparoscopic procedures is performed. Following an initial exploration into gesture recognition for laparoscopic surgery, supervised and unsupervised methods of gesture recognition have been attempted. Hidden Markov Models (HMMs) are then used to classify these gestures. A novel HMM Sensor Selection Framework (HSSF) has been presented for the design of the BSG and to tackle the issue of optimal sensor placement for reducing the overall system complexity. By using the framework, sensors providing the maximal amount of information and the least correlating data related to a series of laparoscopic gestures can be selected. Both phantom and tissue experiments have been performed in laboratory settings and analysis methods have been developed for observing performance changes during training. The results presented in this thesis suggest that the proposed BSG is non-intrusive and easy to use. Furthermore, the proposed HSSF can successfully extract the optimal sensor channels, thus reducing the number of sensors required while maintaining the classification accuracy of the gestures. Further miniaturisation of the device will make the BSN sensor glove a valuable tool for objective assessment of MIS skills in clinical settings.