Less is more: unsupervised mask-guided annotated CT image synthesis with minimum manual segmentations

Publication available at:
Title: Less is more: unsupervised mask-guided annotated CT image synthesis with minimum manual segmentations
Authors: Xing, X
Papanastasiou, G
Walsh, S
Yang, G
Item Type: Journal Article
Abstract: As a pragmatic data augmentation tool, data synthesis has generally returned dividends in performance for deep learning based medical image analysis. However, generating corresponding segmentation masks for synthetic medical images is laborious and subjective. To obtain paired synthetic medical images and segmentations, conditional generative models that use segmentation masks as synthesis conditions were proposed. However, these segmentation mask-conditioned generative models still relied on large, varied, and labeled training datasets, and they could only provide limited constraints on human anatomical structures, leading to unrealistic image features. Moreover, the invariant pixel-level conditions could reduce the variety of synthetic lesions and thus reduce the efficacy of data augmentation. To address these issues, in this work, we propose a novel strategy for medical image synthesis, namely Unsupervised Mask (UM)-guided synthesis, to obtain both synthetic images and segmentations using limited manual segmentation labels. We first develop a superpixel based algorithm to generate unsupervised structural guidance and then design a conditional generative model to synthesize images and annotations simultaneously from those unsupervised masks in a semi-supervised multi-task setting. In addition, we devise a multi-scale multi-task Fréchet Inception Distance (MM-FID) and multi-scale multi-task standard deviation (MM-STD) to harness both fidelity and variety evaluations of synthetic CT images. With multiple analyses on different scales, we could produce stable image quality measurements with high reproducibility. Compared with the segmentation mask guided synthesis, our UM-guided synthesis provided high-quality synthetic images with significantly higher fidelity, variety, and utility (p < 0.05 by Wilcoxon Signed Ranked test).
Issue Date: 1-Sep-2023
Date of Acceptance: 18-Mar-2023
DOI: 10.1109/TMI.2023.3260169
ISSN: 0278-0062
Publisher: Institute of Electrical and Electronics Engineers
Start Page: 2566
End Page: 2576
Journal / Book Title: IEEE Transactions on Medical Imaging
Volume: 42
Issue: 9
Copyright Statement: Copyright © 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Publication Status: Published
Open Access location:
Online Publication Date: 2023-03-21
Appears in Collections:Bioengineering
National Heart and Lung Institute
Faculty of Medicine
Faculty of Engineering