Validation of artificial intelligence spirometry diagnostic support software in primary care: a blinded diagnostic accuracy study
File(s)
Published version
Author(s)
Type
Journal Article
Abstract
Objective and design
The objective of the present study was to assess the discriminative accuracy of artificial intelligence (AI) software to identify COPD and other chronic respiratory diseases from primary care spirometry. This was a diagnostic study with blinded analysis.
Methods
Retrospective hand-held spirometry data from consecutive patients attending primary care clinics in Hillingdon (London, UK) between September 2015 and March 2019 were used. The index diagnosis was the “preferred” diagnosis determined by AI software (highest probability) using supervised random-forest machine learning to interpret raw spirometry data and basic demographics. The reference diagnosis was based on the consensus of expert pulmonologists with access to primary and secondary care medical notes and results of relevant investigations. Cross-tabulation of the index test results by the results of the reference standard for COPD and other respiratory disease categories provided the main outcome measures.
Results
In this primary care spirometry dataset from 1113 patients, 543 (48.8%) had a reference diagnosis of COPD. AI preferred diagnosis detected 456, achieving a sensitivity of 84.0% (95% CI 80.6–87.0%), specificity of 86.8% (83.8–89.5%), accuracy of 85.4% (83.2–87.5%) with area under curve (AUC) of 0.914 (0.896–0.930). AI preferred diagnosis identified 187 out of 249 patients with reference diagnosis of interstitial lung disease and 59 out of 107 patients with asthma, with AUCs of 0.900 (0.880–0.916) and 0.814 (0.790–0.836), respectively.
Conclusion
AI software achieved high sensitivity and specificity in identifying COPD using spirometry and basic demographic data and may support accurate diagnosis of COPD in primary care. AI software performed less well for other chronic respiratory disease categories.
The objective of the present study was to assess the discriminative accuracy of artificial intelligence (AI) software to identify COPD and other chronic respiratory diseases from primary care spirometry. This was a diagnostic study with blinded analysis.
Methods
Retrospective hand-held spirometry data from consecutive patients attending primary care clinics in Hillingdon (London, UK) between September 2015 and March 2019 were used. The index diagnosis was the “preferred” diagnosis determined by AI software (highest probability) using supervised random-forest machine learning to interpret raw spirometry data and basic demographics. The reference diagnosis was based on the consensus of expert pulmonologists with access to primary and secondary care medical notes and results of relevant investigations. Cross-tabulation of the index test results by the results of the reference standard for COPD and other respiratory disease categories provided the main outcome measures.
Results
In this primary care spirometry dataset from 1113 patients, 543 (48.8%) had a reference diagnosis of COPD. AI preferred diagnosis detected 456, achieving a sensitivity of 84.0% (95% CI 80.6–87.0%), specificity of 86.8% (83.8–89.5%), accuracy of 85.4% (83.2–87.5%) with area under curve (AUC) of 0.914 (0.896–0.930). AI preferred diagnosis identified 187 out of 249 patients with reference diagnosis of interstitial lung disease and 59 out of 107 patients with asthma, with AUCs of 0.900 (0.880–0.916) and 0.814 (0.790–0.836), respectively.
Conclusion
AI software achieved high sensitivity and specificity in identifying COPD using spirometry and basic demographic data and may support accurate diagnosis of COPD in primary care. AI software performed less well for other chronic respiratory disease categories.
Date Issued
2025-09-01
Date Acceptance
2025-02-21
Citation
ERJ Open Research, 2025, 11 (5)
ISSN
2312-0541
Publisher
European Respiratory Society
Journal / Book Title
ERJ Open Research
Volume
11
Issue
5
Copyright Statement
Copyright ©The authors 2025 This version is distributed under the terms of the Creative Commons Attribution NonCommercial Licence 4.0. For commercial reproduction rights and permissions contact permissions@ersnet.org
License URL
Identifier
https://www.ncbi.nlm.nih.gov/pubmed/41031099
PII: 00116-2025
Publication Status
Published
Coverage Spatial
England
Article Number
00116-2025
Date Publish Online
2025-04-03