Self-supervised monocular depth estimation with 3-D displacement module for laparoscopic images
Author(s)
Xu, Chi
Huang, Baoru
Elson, Daniel S
Type
Journal Article
Abstract
We present a novel self-supervised training framework with 3D displacement (3DD) module for accurately estimating per-pixel depth maps from single laparoscopic images. Recently, several self-supervised learning based monocular depth estimation models have achieved good results on the KITTI dataset, under the hypothesis that the camera is dynamic and the objects are stationary, however this hypothesis is often reversed in the surgical setting (laparoscope is stationary, the surgical instruments and tissues are dynamic). Therefore, a 3DD module is proposed to establish the relation between frames instead of ego-motion estimation. In the 3DD module, a convolutional neural network (CNN) analyses source and target frames to predict the 3D displacement of a 3D point cloud from a target frame to a source frame in the coordinates of the camera. Since it is difficult to constrain the depth displacement from two 2D images, a novel depth consistency module is proposed to maintain depth consistency between displacement-updated depth and model-estimated depth to constrain 3D displacement effectively. Our proposed method achieves remarkable performance for monocular depth estimation on the Hamlyn surgical dataset and acquired ground truth depth maps, outperforming monodepth, monodepth2 and packnet models.
Date Issued
2022-05-01
Date Acceptance
2022-04-14
Citation
IEEE Transactions on Medical Robotics and Bionics, 2022, 4 (2), pp.331-334
ISSN
2576-3202
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Start Page
331
End Page
334
Journal / Book Title
IEEE Transactions on Medical Robotics and Bionics
Volume
4
Issue
2
Copyright Statement
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Publication Status
Published
Date Publish Online
2022-04-27