3D Reconstruction from Stereo and Photometric Cues in Minimally Invasive Surgery

Abstract

During the last decade, an increasing number of advanced platforms for robotic assisted Minimally Invasive Surgery (MIS) have been translated from research concepts to practical systems used in routine clinical settings. Such systems have defined their own niche in the market place and are gaining increasing acceptance by the surgical community. More importantly, they improve surgeons' dexterity and intra-operative visualisation together with the benefits of improved procedural time, safety, consistency and patient outcome. Some of the new technical features integrated within current robotic systems include higher degrees of freedom for more agile manoeuvring, tremor filtering and haptic feedback. Where the next frontier lies in terms of novel means of surgical assistance is in navigation systems that would enable intra-operative surgical guidance, imposing dynamic active constraints and high fidelity augmented reality environments. At the foundation of all navigation systems lies an accurate 3D reconstruction algorithm. To overcome the current limitations of feature-based matching and reconstruction systems, this thesis investigates a method to integrate depth information from conventional stereo 3D reconstruction with photometric cues from Shape-from-Shading techniques. This is further enhanced with a gaze-contingent framework suitable for intra-operative navigation during MIS. The resulting system is suitable for applications with existing MIS platforms without the need of additional hardware, ensuring increased accuracy in areas without salient anatomical landmarks or artificially introduced fiducials. Furthermore, a multi-scale feature detector is proposed for intra-operative stereo telestration required for telementoring and remote collaboration of robotic assisted MIS. An improved formulation of the current state-of-the-art in Shape-from-Shading is integrated with intra-operative scenes for accurate metric depth recovery. The potential clinical application of the technique is demonstrated for autonomous navigation of wireless capsule endoscopes. Future research directions and preliminary results of integrating photometric stereo constraints for reliable reconstruction of featureless, non- Lambertian deforming tissue surfaces are provided and all methods proposed have been validated with either in vivo or phantom experiments to justify their potential clinical values in MIS.