Return on investment from quality improvement programmes in mental healthcare; a Delphi study on conceptual uncertainty and ambiguity
File(s)
Published version
Author(s)
Thusini, S’thembile
Soukup, Tayana
Henderson, Claire
Type
Journal Article
Abstract
Introduction
Return on Investment (ROI) is a recommended and thus legitimatised approach to assessing the value of healthcare programmes, including in Quality Improvement (QI). In its accounting origins, ROI estimates monetary benefits against investments. In health economics, ROI estimates a monetary value of healthcare related benefits against costs. In a recent single site study, we explored the metaphorical use of ROI as a concept of benefits from QI in-order to develop a QI-ROI conceptual framework. We found that in QI, both monetary and non-monetary benefits are deemed as legitimate ROI by mental healthcare leaders. However, various ambiguities and uncertainties associated with QI-ROI conceptualisation were also apparent. As such, in the current study, we explored consensus on the QI-ROI concept with a wider group of mental healthcare leaders across multiple sites. We then assessed the potential impact of the consensus levels on the stability of our QI-ROI conceptual framework.
Methods
We ran two rounds of Delphi through Qualtrics online platform. Public sector mental health leaders were approached UK-wide (N = > 100). Only leaders from England participated (N = 23). This included board members (n = 15), directors (n = 2), and QI leaders (n = 6). Sixty-seven items were rated, including patient outcomes, development (e.g., culture), external benefits (e.g., socio-economic), incentives (e.g., reputation), implementation outcomes (e.g., sustainability), and monetised benefits. Consensus was measured using interquartile range and median. We also collected qualitative data to help explain participant responses.
Results
There was consensus on 45 of 67 (67%) of the items in our existing QI-ROI conceptual framework. There was dissensus on 22 of 67 (33%) items. Positive consensus (item acceptance) was on 33 of 45 (73.3%), negative consensus (item rejection) was on 5 of 45 (11.1%) items, and indecision was on 7 of 45 (15.6%). Patient outcomes were rated as most relevant, followed by developmental benefits, and benefits for external stakeholders. External incentives and monetised outcomes were rated as least relevant and legitimate as QI-ROI. Areas of indecision and dissensus were largely on implementation outcomes, measurement, and monetisation of QI benefits. Qualitative data indicated that indecision and dissensus was likely related to organisational needs, or the novelty of some related benefits.
Conclusion
Mental healthcare leaders value various QI benefits depending on organisational needs. Thus, although there is consensus on most aspects, some ambiguity and uncertainty may remain. Regardless, QI-ROI is viewed as both monetary and non-monetary benefits. This has implications for similar organisations and policies relating to the assessment of the value of QI.
Return on Investment (ROI) is a recommended and thus legitimatised approach to assessing the value of healthcare programmes, including in Quality Improvement (QI). In its accounting origins, ROI estimates monetary benefits against investments. In health economics, ROI estimates a monetary value of healthcare related benefits against costs. In a recent single site study, we explored the metaphorical use of ROI as a concept of benefits from QI in-order to develop a QI-ROI conceptual framework. We found that in QI, both monetary and non-monetary benefits are deemed as legitimate ROI by mental healthcare leaders. However, various ambiguities and uncertainties associated with QI-ROI conceptualisation were also apparent. As such, in the current study, we explored consensus on the QI-ROI concept with a wider group of mental healthcare leaders across multiple sites. We then assessed the potential impact of the consensus levels on the stability of our QI-ROI conceptual framework.
Methods
We ran two rounds of Delphi through Qualtrics online platform. Public sector mental health leaders were approached UK-wide (N = > 100). Only leaders from England participated (N = 23). This included board members (n = 15), directors (n = 2), and QI leaders (n = 6). Sixty-seven items were rated, including patient outcomes, development (e.g., culture), external benefits (e.g., socio-economic), incentives (e.g., reputation), implementation outcomes (e.g., sustainability), and monetised benefits. Consensus was measured using interquartile range and median. We also collected qualitative data to help explain participant responses.
Results
There was consensus on 45 of 67 (67%) of the items in our existing QI-ROI conceptual framework. There was dissensus on 22 of 67 (33%) items. Positive consensus (item acceptance) was on 33 of 45 (73.3%), negative consensus (item rejection) was on 5 of 45 (11.1%) items, and indecision was on 7 of 45 (15.6%). Patient outcomes were rated as most relevant, followed by developmental benefits, and benefits for external stakeholders. External incentives and monetised outcomes were rated as least relevant and legitimate as QI-ROI. Areas of indecision and dissensus were largely on implementation outcomes, measurement, and monetisation of QI benefits. Qualitative data indicated that indecision and dissensus was likely related to organisational needs, or the novelty of some related benefits.
Conclusion
Mental healthcare leaders value various QI benefits depending on organisational needs. Thus, although there is consensus on most aspects, some ambiguity and uncertainty may remain. Regardless, QI-ROI is viewed as both monetary and non-monetary benefits. This has implications for similar organisations and policies relating to the assessment of the value of QI.
Date Issued
2025-10-08
Date Acceptance
2025-09-05
Citation
BMC Health Services Research, 2025, 25
ISSN
1472-6963
Publisher
BMC
Journal / Book Title
BMC Health Services Research
Volume
25
Issue
1
Copyright Statement
© The Author(s) 2025. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
License URL
Identifier
https://www.ncbi.nlm.nih.gov/pubmed/41063081
PII: 10.1186/s12913-025-13484-0
Subjects
Consensus
Dissensus
QI
Quality improvement
ROI
Return on investment
Delphi Technique
Humans
Quality Improvement
Mental Health Services
Uncertainty
Cost-Benefit Analysis
England
Consensus
Publication Status
Published
Coverage Spatial
England
Article Number
1330
Date Publish Online
2025-10-08