Image synthesis with class-aware semantic diffusion models for surgical scene segmentation
Author(s)
Zhou, Yihang
Towning, Rebecca
Awad, Zaid
Giannarou, Stamatia
Type
Journal Article
Abstract
Surgical scene segmentation is essential for enhancing surgical precision, yet it is frequently compromised by the scarcity and imbalance of available data. To address these challenges, semantic image synthesis methods based on generative adversarial networks and diffusion models have been developed. However, these models often yield non-diverse images and fail to capture small, critical tissue classes, limiting their effectiveness. In response, a class-aware semantic diffusion model (CASDM), a novel approach which utilizes segmentation maps as conditions for image synthesis to tackle data scarcity and imbalance is proposed. Novel class-aware mean squared error and class-aware self-perceptual loss functions have been defined to prioritize critical, less visible classes, thereby enhancing image quality and relevance. Furthermore, to the authors' knowledge, they are the first to generate multi-class segmentation maps using text prompts in a novel fashion to specify their contents. These maps are then used by CASDM to generate surgical scene images, enhancing datasets for training and validating segmentation models. This evaluation assesses both image quality and downstream segmentation performance, demonstrates the strong effectiveness and generalisability of CASDM in producing realistic image-map pairs, significantly advancing surgical scene segmentation across diverse and challenging datasets.
Date Issued
2025-01-31
Date Acceptance
2024-11-11
Citation
Healthcare Technology Letters, 2025, 12 (1)
ISSN
2053-3713
Publisher
Wiley
Journal / Book Title
Healthcare Technology Letters
Volume
12
Issue
1
Copyright Statement
© 2025 The Author(s). Healthcare Technology Letters published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.
License URL
Identifier
10.1049/htl2.70003
Publication Status
Published
Article Number
e70003
Date Publish Online
2025-01-31