Collective intelligence increases diagnostic accuracy in a general practice setting
Author(s)
Type
Journal Article
Abstract
Background
General practitioners (GPs) work in an ill-defined environment where diagnostic errors are prevalent. Previous research indicates that aggregating independent diagnoses can improve diagnostic accuracy in a range of settings. We examined whether aggregating independent diagnoses can also improve diagnostic accuracy for GP decision making. In addition, we investigated the potential benefit of such an approach in combination with a decision support system (DSS).
Methods
We simulated virtual groups using data sets from 2 previously published studies. In study 1, 260 GPs independently diagnosed 9 patient cases in a vignette-based study. In study 2, 30 GPs independently diagnosed 12 patient actors in a patient-facing study. In both data sets, GPs provided diagnoses in a control condition and/or DSS condition(s). Each GP’s diagnosis, confidence rating, and years of experience were entered into a computer simulation. Virtual groups of varying sizes (range: 3–9) were created, and different collective intelligence rules (plurality, confidence, and seniority) were applied to determine each group’s final diagnosis. Diagnostic accuracy was used as the performance measure.
Results
Aggregating independent diagnoses by weighing them equally (i.e., the plurality rule) substantially outperformed average individual accuracy, and this effect increased with increasing group size. Selecting diagnoses based on confidence only led to marginal improvements, while selecting based on seniority reduced accuracy. Combining the plurality rule with a DSS further boosted performance.
Discussion
Combining independent diagnoses may substantially improve a GP’s diagnostic accuracy and subsequent patient outcomes. This approach did, however, not improve accuracy in all patient cases. Therefore, future work should focus on uncovering the conditions under which collective intelligence is most beneficial in general practice.
Highlights
•
We examined whether aggregating independent diagnoses of GPs can improve diagnostic accuracy.
•
Using data sets of 2 previously published studies, we composed virtual groups of GPs and combined their independent diagnoses using 3 collective intelligence rules (plurality, confidence, and seniority).
•
Aggregating independent diagnoses by weighing them equally substantially outperformed average individual GP accuracy, and this effect increased with increasing group size.
•
Combining independent diagnoses may substantially improve GP’s diagnostic accuracy and subsequent patient outcomes.
General practitioners (GPs) work in an ill-defined environment where diagnostic errors are prevalent. Previous research indicates that aggregating independent diagnoses can improve diagnostic accuracy in a range of settings. We examined whether aggregating independent diagnoses can also improve diagnostic accuracy for GP decision making. In addition, we investigated the potential benefit of such an approach in combination with a decision support system (DSS).
Methods
We simulated virtual groups using data sets from 2 previously published studies. In study 1, 260 GPs independently diagnosed 9 patient cases in a vignette-based study. In study 2, 30 GPs independently diagnosed 12 patient actors in a patient-facing study. In both data sets, GPs provided diagnoses in a control condition and/or DSS condition(s). Each GP’s diagnosis, confidence rating, and years of experience were entered into a computer simulation. Virtual groups of varying sizes (range: 3–9) were created, and different collective intelligence rules (plurality, confidence, and seniority) were applied to determine each group’s final diagnosis. Diagnostic accuracy was used as the performance measure.
Results
Aggregating independent diagnoses by weighing them equally (i.e., the plurality rule) substantially outperformed average individual accuracy, and this effect increased with increasing group size. Selecting diagnoses based on confidence only led to marginal improvements, while selecting based on seniority reduced accuracy. Combining the plurality rule with a DSS further boosted performance.
Discussion
Combining independent diagnoses may substantially improve a GP’s diagnostic accuracy and subsequent patient outcomes. This approach did, however, not improve accuracy in all patient cases. Therefore, future work should focus on uncovering the conditions under which collective intelligence is most beneficial in general practice.
Highlights
•
We examined whether aggregating independent diagnoses of GPs can improve diagnostic accuracy.
•
Using data sets of 2 previously published studies, we composed virtual groups of GPs and combined their independent diagnoses using 3 collective intelligence rules (plurality, confidence, and seniority).
•
Aggregating independent diagnoses by weighing them equally substantially outperformed average individual GP accuracy, and this effect increased with increasing group size.
•
Combining independent diagnoses may substantially improve GP’s diagnostic accuracy and subsequent patient outcomes.
Date Issued
2024-05
Date Acceptance
2024-02-28
Citation
Medical Decision Making, 2024, 44 (4), pp.451-462
ISSN
0272-989X
Publisher
SAGE Publications
Start Page
451
End Page
462
Journal / Book Title
Medical Decision Making
Volume
44
Issue
4
Copyright Statement
© The Author(s) 2024.
Creative Commons License (CC BY 4.0)
This article is distributed under the terms of the Creative Commons Attribution 4.0 License (https://creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).
Creative Commons License (CC BY 4.0)
This article is distributed under the terms of the Creative Commons Attribution 4.0 License (https://creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).
License URL
Identifier
https://journals.sagepub.com/doi/10.1177/0272989X241241001
Subjects
CHOICE
collective intelligence
decision support systems
diagnostic accuracy
ERROR
general practice
GPS
GROUP JUDGMENTS
Health Care Sciences & Services
Health Policy & Services
INFORMATION DISTORTION
INTERVENTION
Life Sciences & Biomedicine
medical diagnostics
Medical Informatics
PERFORMANCE
Science & Technology
SUPPORT
wisdom of crowds
Publication Status
Published
Date Publish Online
2024-04-12