Style-extracting diffusion models for semi-supervised histopathology segmentation
File(s)
Accepted version
Author(s)
Type
Conference Paper
Abstract
Deep learning-based image generation has seen significant advancements with diffusion models, notably improving the quality of generated images. Despite these developments, generating images with unseen characteristics beneficial for downstream tasks has received limited attention. To bridge this gap, we propose Style-Extracting Diffusion Models, featuring two conditioning mechanisms. Specifically, we utilize 1) a style conditioning mechanism which allows to inject style information of previously unseen images during image generation and 2) a content conditioning which can be targeted to a downstream task, e.g., layout for segmentation. We introduce a trainable style encoder to extract style information from images, and an aggregation block that merges style information from multiple style inputs. This architecture enables the generation of images with unseen styles in a zero-shot manner, by leveraging styles from unseen images, resulting in more diverse generations. In this work, we use the image layout as target condition and first show the capability of our method on a natural image dataset as a proof-of-concept. We further demonstrate its versatility in histopathology, where we combine prior knowledge about tissue composition and unannotated data to create diverse synthetic images with known layouts. This allows us to generate additional synthetic data to train a segmentation network in a semi-supervised fashion. We verify the added value of the generated images by showing improved segmentation results and lower performance variability between patients when synthetic images are included during segmentation training. The code of the method is publicly available at https://github.com/OettlM/STEDM.
Editor(s)
Leonardis, A
Ricci, E
Roth, S
Russakovsky, O
Sattler, T
Varol, G
Date Issued
2025-11-01
Date Acceptance
2024-09-01
Citation
Computer Vision - ECCV 2024, 2025, 15133, pp.236-252
ISBN
978-3-031-73225-6
ISSN
0302-9743
Publisher
Springer International Publishing AG
Start Page
236
End Page
252
Journal / Book Title
Computer Vision - ECCV 2024
Volume
15133
Copyright Statement
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG. This is the author’s accepted manuscript made available under a CC-BY licence in accordance with Imperial’s Research Publications Open Access policy (www.imperial.ac.uk/oa-policy)
License URL
Source
18th European Conference on Computer Vision (ECCV)
Subjects
Computer Science
Computer Science, Artificial Intelligence
Computer Science, Interdisciplinary Applications
Computer Science, Theory & Methods
Science & Technology
Technology
Publication Status
Published
Start Date
2024-09-29
Finish Date
2024-10-04
Coverage Spatial
Milan, Italy