Online surgical tool tracking for active constraints in robotic surgery with a novel hands-on robot
File(s)
Author(s)
Souipas, Spyridon
Type
Thesis or dissertation
Abstract
Robotic-assisted surgery has advanced significantly over the past decades, tackling different types of surgery. Following a comprehensive review of commercial systems, orthopaedic robots were predominantly categorised into two configurations, namely bulky robots providing full haptic feedback or miniaturised, non-haptic devices. Furthermore, haptic feedback is principally achieved through preoperative imaging, without allowing for intraoperative Region of Interest (RoI) definition.
To facilitate RoI definition, detection and localisation of surgical tools in the Operating Theatre (OR) were explored. A camera-based neural network was developed, achieving 3D pose estimation of conventional surgical tools. The network was trained and tested using a novel dataset, exhibiting a position and orientation error of 5.5mm ± 6.6mm and 3.3° ± 3.1° respectively, with real-time deployment at 4.5 frames per second.
To address the absence of compact surgical robots with haptic feedback, a novel, hands-on, miniaturised platform was designed. This system achieved sub-millimetre accuracy, while also providing haptic and visual feedback. A study with untrained volunteers demonstrated improved performance under feedback compared to freehand operation.
Combining the tool tracking network with the haptic robot led to a novel intraoperative haptic feedback workflow. Safety volumes were generated by tracking a surgical tool and employed in two haptic experiments. In the first, the user was impeded from exiting the volume; in the second, the user was prevented from penetrating the volume. Results confirm the viability of the concept and highlight key performance metrics for the prototype in robotic-assisted orthopaedic surgery.
This thesis outlines a technique for intraoperatively establishing RoI integrated within a haptic feedback workflow, featuring a novel collaborative robot and a deep learning network. While the pipeline is efficient and independent of preoperative imaging, future research is required to develop gravity compensation for the robot and expand generalisation capabilities for the network, with the eventual aim of clinical translation.
To facilitate RoI definition, detection and localisation of surgical tools in the Operating Theatre (OR) were explored. A camera-based neural network was developed, achieving 3D pose estimation of conventional surgical tools. The network was trained and tested using a novel dataset, exhibiting a position and orientation error of 5.5mm ± 6.6mm and 3.3° ± 3.1° respectively, with real-time deployment at 4.5 frames per second.
To address the absence of compact surgical robots with haptic feedback, a novel, hands-on, miniaturised platform was designed. This system achieved sub-millimetre accuracy, while also providing haptic and visual feedback. A study with untrained volunteers demonstrated improved performance under feedback compared to freehand operation.
Combining the tool tracking network with the haptic robot led to a novel intraoperative haptic feedback workflow. Safety volumes were generated by tracking a surgical tool and employed in two haptic experiments. In the first, the user was impeded from exiting the volume; in the second, the user was prevented from penetrating the volume. Results confirm the viability of the concept and highlight key performance metrics for the prototype in robotic-assisted orthopaedic surgery.
This thesis outlines a technique for intraoperatively establishing RoI integrated within a haptic feedback workflow, featuring a novel collaborative robot and a deep learning network. While the pipeline is efficient and independent of preoperative imaging, future research is required to develop gravity compensation for the robot and expand generalisation capabilities for the network, with the eventual aim of clinical translation.
Version
Open Access
Date Issued
2024-03
Date Awarded
2024-08
Copyright Statement
Creative Commons Attribution NonCommercial Licence
Advisor
Rodriguez Y Baena, Ferdinando
Davies, Brian
Nguyen, Anh
Sponsor
Signature Robot
Publisher Department
Mechanical Engineering
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)