Multitask weakly supervised generative network for MR-US registration
File(s)
Author(s)
Type
Journal Article
Abstract
Registering pre-operative modalities, such as magnetic resonance imaging or computed tomography, to ultrasound images is crucial for guiding clinicians during surgeries and biopsies. Recently, deep-learning approaches have been proposed to increase the speed and accuracy of this registration problem. However, all of these approaches need expensive supervision from the ultrasound domain. In this work, we propose a multitask generative framework that needs weak supervision only from the pre-operative imaging domain during training. To perform a deformable registration, the proposed framework translates a magnetic resonance image to the ultrasound domain while preserving the structural content. To demonstrate the efficacy of the proposed method, we tackle the registration problem of pre-operative 3D MR to transrectal ultrasonography images as necessary for targeted prostate biopsies. We use an in-house dataset of 600 patients, divided into 540 for training, 30 for validation, and the remaining for testing. An expert manually segmented the prostate in both modalities for validation and test sets to assess the performance of our framework. The proposed framework achieves a 3.58 mm target registration error on the expert-selected landmarks, 89.2% in the Dice score, and 1.81 mm 95th percentile Hausdorff distance on the prostate masks in the test set. Our experiments demonstrate that the proposed generative model successfully translates magnetic resonance images into the ultrasound domain. The translated image contains the structural content and fine details due to an ultrasound-specific two-path design of the generative model. The proposed framework enables training learning-based registration methods while only weak supervision from the pre-operative domain is available.
Date Issued
2024-11-01
Date Acceptance
2024-05-05
Citation
IEEE Transactions on Medical Imaging, 2024, 43 (11), pp.3780-3793
ISSN
0278-0062
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Start Page
3780
End Page
3793
Journal / Book Title
IEEE Transactions on Medical Imaging
Volume
43
Issue
11
Copyright Statement
© 2024 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
License URL
Identifier
https://www.ncbi.nlm.nih.gov/pubmed/38829753
Subjects
Science & Technology
Technology
Life Sciences & Biomedicine
Computer Science, Interdisciplinary Applications
Engineering, Biomedical
Engineering, Electrical & Electronic
Imaging Science & Photographic Technology
Radiology, Nuclear Medicine & Medical Imaging
Computer Science
Engineering
Ultrasonic imaging
Training
Image segmentation
Measurement
Biopsy
Multitasking
Deep learning
Convolutional neural network
medical image registration
multi-task learning
unpaired image-to-image translation
US-MR registration
ULTRASOUND REGISTRATION
IMAGE REGISTRATION
LEARNING FRAMEWORK
SEGMENTATION
Publication Status
Published
Coverage Spatial
United States
Date Publish Online
2024-06-03