From macro to micro: autonomous multiscale image fusion for robotic surgery
File(s)07924395.pdf (4.39 MB) RAM_macro_micro_auto_scan_V2.pdf (1.38 MB)
Published version
Accepted version
Author(s)
Type
Journal Article
Abstract
In recent years, minimally invasive robotic surgery has shown great promises for enhancing surgical precision and improving patient outcomes. Despite these advances, intraoperative tissue characterisation (such as the identification of cancerous tissue) still relies on traditional biopsy and histology, a process that is time-consuming and often disrupts the normal surgical workflow. In order to provide effective intra-operative decision-making, emerging optical biopsy techniques, such as probe based confocal laser endomicroscopy (pCLE) and optical coherence tomography (OCT), have been developed to provide real-time in vivo, in situ assessment of tissue micro-structures. Clinical deployment of these techniques, however, requires large area surveillance, from macro (mm/cm) to micro (µm) coverage in order to differentiate underlying tissue structures. This article provides a real-time multi-scale fusion scheme for robotic surgery. It demonstrates how the da Vinci surgical robot, used together with the da Vinci Research Kit, can be used for automated 2D scanning of pCLE/OCT probes, providing large area tissue surveillance by image stitching. Open-loop control of the robot provides insufficient precision for probe scanning, and therefore the motion is visually servoed using the live pCLE images (for lateral position) and OCT images (for axial position). The resulting tissue maps can then be fused in real-time with a stereo reconstruction from the laparoscopic video, providing the surgeon with a multi-scale 3D view of the operating site.
Date Issued
2017-05-10
Date Acceptance
2017-02-16
Citation
IEEE Robotics & Automation Magazine, 2017, 24 (2), pp.63-72
ISSN
1070-9932
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Start Page
63
End Page
72
Journal / Book Title
IEEE Robotics & Automation Magazine
Volume
24
Issue
2
Copyright Statement
This is an open access article freely available at http://ieeexplore.ieee.org/document/7924395/
Sponsor
Engineering & Physical Science Research Council (EPSRC)
Engineering & Physical Science Research Council (EPSRC)
Grant Number
EP/N022521/1
EP/N019318/1
Subjects
Science & Technology
Technology
Automation & Control Systems
Robotics
OPTICAL COHERENCE TOMOGRAPHY
ENDOMICROSCOPY
PROBE
0801 Artificial Intelligence And Image Processing
0906 Electrical And Electronic Engineering
0913 Mechanical Engineering
Industrial Engineering & Automation
Publication Status
Published