Occlusion-robust markerless surgical instrument pose estimation
Author(s)
Xu, Haozheng
Giannarou, Stamatia
Type
Journal Article
Abstract
The estimation of the pose of surgical instruments is important in Robot-assisted Minimally Invasive Surgery (RMIS) to assist surgical
navigation and enable autonomous robotic task execution. The performance of current instrument pose estimation methods deteriorates
significantly in the presence of partial tool visibility, occlusions, and changes in the surgical scene. In this work, a vision-based framework
is proposed for markerless estimation of the 6DoF pose of surgical instruments. To deal with partial instrument visibility, a keypoint object
representation is used and stable and accurate instrument poses are computed using a PnP solver. To boost the learning process of the model
under occlusion, a new mask-based data augmentation approach has been proposed. To validate our model, a dataset for instrument pose
estimation with highly accurate ground truth data has been generated using different surgical robotic instruments. The proposed network can
achieve submillimeter accuracy and our experimental results verify its generalisability to different shapes of occlusion.
navigation and enable autonomous robotic task execution. The performance of current instrument pose estimation methods deteriorates
significantly in the presence of partial tool visibility, occlusions, and changes in the surgical scene. In this work, a vision-based framework
is proposed for markerless estimation of the 6DoF pose of surgical instruments. To deal with partial instrument visibility, a keypoint object
representation is used and stable and accurate instrument poses are computed using a PnP solver. To boost the learning process of the model
under occlusion, a new mask-based data augmentation approach has been proposed. To validate our model, a dataset for instrument pose
estimation with highly accurate ground truth data has been generated using different surgical robotic instruments. The proposed network can
achieve submillimeter accuracy and our experimental results verify its generalisability to different shapes of occlusion.
Date Issued
2024-12
Date Acceptance
2024-11-11
Citation
Healthcare Technology Letters, 2024, 11 (6), pp.327-335
ISSN
2053-3713
Publisher
Wiley
Start Page
327
End Page
335
Journal / Book Title
Healthcare Technology Letters
Volume
11
Issue
6
Copyright Statement
© 2024 The Author(s). Healthcare Technology Letters published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology.
This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.
This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.
License URL
Identifier
https://ietresearch.onlinelibrary.wiley.com/doi/full/10.1049/htl2.12100
Publication Status
Published
Date Publish Online
2024-11-27