Background
The artificial neural network (ANN) is an increasingly important tool in the context of solving complex medical classification problems. However, one of the principal challenges in leveraging AI technology in the healthcare setting has been the relative inability to translate models into clinician workflow. Here we demonstrate the development of a COVID-19 outcome prediction application which utilises an ANN and assesses its usability in the clinical setting.
Methods
Usability assessment was conducted on the application followed by a semi-structured end-user interview. Usability was specified by effectiveness, efficiency, and satisfaction measures. These data were reported with descriptive statistics. The end-user interview data were analysed using the thematic framework method, which allowed for the development of themes from the interview narratives.
Participants
Thirty-one Nation Health Service (NHS) physicians at a West London teaching hospital, including foundation doctors, senior house officers, registrars, and consultants.
Results
All participants were able to complete the assessment, with a mean time to complete separate patient vignettes of 59.35 seconds (standard deviation (SD) = 10.35). Mean system usability scale (SUS) score was 91.94 (SD = 8.54), which corresponds with an adjective rating of “Excellent”. The clinicians found the application intuitive and easy to use, with the majority describing its predictions as a useful adjunct to their clinical practice. The main concern related to use of the application in isolation as opposed to in conjunction with other clinical parameters. However, most clinicians felt that the application could positively reinforce or validate their clinical judgement.
Conclusion
Translating AI technologies into the clinical setting remains an important but challenging task. We demonstrate the effectiveness, efficiency, and system usability of a web application designed to predict COVID-19 patient outcomes from an ANN.