Performance of vision-language models compared with 252 medical students on text-only and image-based dermatology examinations
File(s) s41598-026-68943-3.pdf (1.89 MB)
Published version
Author(s)
Type
Journal Article
Abstract
Vision–language models (VLMs) are increasingly evaluated in medical education, yet their performance on visually intensive assessments remains incompletely understood. We compared four state-of-the-art VLMs, GPT-4o and GPT-5 (both accessed via ChatGPT), Gemini 2.5 Flash, and Gemini 3 Pro, with fifth-year medical students on ten consecutive dermatology clerkship examinations administered between September 2023 and January 2025. Examinations combined text-only questions (multiple-choice, multiple-select, and matching; 60% of total score) with image-based, structured open-ended questions (40%) spanning seven dermatologic sub-domains. Model outputs were evaluated using expert-validated answer keys and grading rubrics, with repeated runs to assess output variability. All VLMs significantly outperformed medical students on text-only examinations (mean scores > 95 vs. 84.9; p < 0.001), showing minimal sensitivity to exam difficulty. In contrast, image-based performance was heterogeneous: Gemini 3 Pro and GPT-5 achieved higher scores than students, whereas students significantly outperformed GPT-4o and Gemini 2.5 Flash. Medical students showed the smallest difference between text-only and image-based scores within this examination setting. Among VLMs, sub-domain analyses suggested that strong visual description and diagnostic performance did not uniformly translate into etiological, differential diagnostic, or treatment-related reasoning. Within the constraints of this single-center, examination-based benchmark, these findings indicate that VLMs demonstrate strong text-based dermatologic knowledge, whereas multimodal performance remains uneven and model-dependent. These results should not be interpreted as evidence of standalone clinical competence, but rather as support for further controlled evaluation of VLMs as complementary, human-supervised tools in dermatology education.
Date Issued
2026-09-29
Date Acceptance
2026-08-25
Citation
Scientific Reports, 2026, 16
ISSN
2045-2322
Publisher
Nature Portfolio
Journal / Book Title
Scientific Reports
Volume
16
Copyright Statement
© The Author(s) 2026. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
License URL
Identifier
10.1038/s41598-026-68943-3
Subjects
Vision-language models
Medical education
Dermatology
Medical students
Medical examinations
Large language models
Publication Status
Published
Article Number
ARTN 30214
Date Publish Online
2026-09-05
