Fine-tuning a small vision language model using synthetic data for explaining bacterial skin disease images
File(s)
Published version
Author(s)
Zhang, Shiwan
Yilmaz, Abdurrahim
Gencoglan, Gulsum
Temelkuran, Burak
Type
Journal Article
Abstract
Background/Objectives: Vision language models (VLMs) show strong potential for medical image understanding, but their large scale often limits practical deployment. This study investigates whether a compact VLM can be effectively adapted for dermatology, with a focus on explaining bacterial skin disease images. Methods: We curate a dataset derived from PMC-OA using the BIOMEDICA dataset and construct PMC-derma-VQA-bacteria by pairing images with inherited figure captions and synthetically generated question–answer (QA) supervision produced by Google’s Gemini model. SmolVLM is fine-tuned under three supervision settings: QA-only, caption-only, and a combined QA+caption strategy. The models are evaluated on a held-out test set for both text-generation quality and diagnostic classification performance. Results: QA-only supervision yields the best report-generation performance, while the combined QA+caption setting achieves the highest classification accuracy (70.20%). Conclusions: Synthetic QA supervision can meaningfully enhance compact VLMs for medical image understanding and diagnostic support in dermatology.
Date Issued
2025-02-01
Date Acceptance
2026-02-09
Citation
Diagnostics, 2025, 16 (4)
ISSN
2075-4418
Publisher
MDPI AG
Journal / Book Title
Diagnostics
Volume
16
Issue
4
Copyright Statement
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
License URL
Identifier
10.3390/diagnostics16040603
Subjects
vision language models
fine-tuning
bacterial skin diseases
dermatology imaging
visual question answering
synthetic data
medical AI
Publication Status
Published
Article Number
603
Date Publish Online
2026-02-18