Data-efficient generation for dataset distillation
OA Location
Author(s)
Li, Zhe
Zhang, Weitong
Cechnicka, Sarah
Kainz, Bernhard
Type
Chapter
Abstract
While deep learning techniques have proven successful in image-related tasks, the exponentially increased data storage and computation costs become a significant challenge. Dataset distillation addresses these challenges by synthesizing only a few images for each class that encapsulate all essential information. Most current methods focus on matching. The problems lie in the synthetic images not being human-readable and the dataset performance being insufficient for downstream learning tasks. Moreover, the distillation time can quickly get out of bounds when the number of synthetic images per class increases even slightly. To address this, we train a class conditional latent diffusion model capable of generating realistic synthetic images with labels. The sampling time can be reduced to several tens of images per seconds. We demonstrate that models can be effectively trained using only a small set of synthetic images and evaluated on a large real test set. Our approach achieved rank in The First Dataset Distillation Challenge at ECCV 2024 on the CIFAR100 and TinyImageNet datasets.
Editor(s)
DelBue, A
Date Issued
2025-05-20
Citation
Computer Vision – ECCV 2024 Workshops, 2025, 15641, pp.68-82
ISBN
978-3-031-93805-4
Publisher
Springer Nature Switzerland AG
Start Page
68
End Page
82
Journal / Book Title
Computer Vision – ECCV 2024 Workshops
Lecture Notes in Computer Science
Volume
15641
Copyright Statement
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG.
Subjects
Computer Science
Publication Status
Published
Date Publish Online
2025-05-20