Devising a quality framework for the development and evaluation of digital health apps

Abstract

Smartphones, tablet devices, and apps have become ubiquitous technologies in the developed world and much interest has arisen around their uses in the healthcare context. There are currently more than 165,000 digital health apps (DHapps) available through app repositories. The potential for these technologies to disrupt and transform healthcare delivery models is frequently cited. It has been suggested however, that minimal barriers to entry and a lack of regulation and evaluation have resulted in an overcrowded marketplace, housing a large proportion of poor quality DHapps. This in turn has undermined confidence in the digital health field, and strengthened calls for higher quality DHapp development and evaluation.

The overarching aim of this Thesis is to devise a quality framework for DHapp development and evaluation. In doing so, earlier chapters of the Thesis make the case for such a framework, highlighting the need and demand for it, and form the premise for later chapters which define its requirements. An introduction to the digital health field is first presented, outlining the potential benefits and hazards, and provides the theoretical basis for this work (Chapter 1). A systematic review exploring uses of these technologies in surgery and the types of clinical studies used to evaluate them is presented (Chapter 2), underlining the potential for DHapps to transform many aspects of surgical practice, but that clinical trials are insufficient and poorly suited on their own to evaluating DHapps holistically. The empirical work of this Thesis is then reported. Three survey studies with healthcare professionals (HCPs) spanning the community, primary, and secondary care settings are presented (Chapter 3), demonstrating ubiquitous ownership and widespread use of these technologies in clinical practice, including potentially high risk interventions without any form of quality or risk assessment prior to use, with large majorities indicating that they would be more likely to use and recommend digital tools if they were formally evaluated. This is followed by an app repository review focussing on breast disease related apps (Chapter 4), demonstrating substantial deficiencies in quality among the majority including the potential for some to cause user harm. The findings from Chapters 1 through 4 make a compelling case for a framework for DHapp development and evaluation – the premise for chapters that follow. A large-scale international two-phase Delphi study with 61 digital health experts is then presented, consisting of an initial interview phase (Chapter 5 – split into three subchapters) followed by a consensus survey phase (Chapter 6), designed to identify and define the requirements of such a framework. Through the initial interview phase themes emerged around DHapp quality and risk (Subchapter 5a), adoption (Subchapter 5b), and evaluation (Subchapter 5c). Important outputs from Subchapters 5a and 5b include a proposed high-level quality framework for DHapp development to guide higher quality app production, a consumer-facing risk interrogation toolbox designed to enable end-users to assess the risks posed by DHapps prior to use, a series of recommendations to minimise the risks posed by the current DHapp marketplace, and a series of key recommendations to facilitate DHapp adoption. Findings from subchapter 5c, which pertain to the requirements of app evaluation specifically, are used to inform the content of surveys for the consensus survey stage of the Delphi study presented in Chapter 6. Through expert consensus, guidelines are devised defining the criteria that should be evaluated as part of an evaluation framework, evaluation models that should be utilised, and the wider requirements of any devised framework. In the concluding chapter (Chapter 7) we present our proposed framework for DHapp evaluation, devised according to the expert-consensus derived guidelines of the preceding chapter, and, to a lesser extent, insights from earlier chapters. A general discussion of the work undertaken in this Thesis is also provided, reflecting on the findings and the wider evidence base. Implications of the work and areas for future research are explored.