The assessment of non-technical skills in ENT surgery: a multidisciplinary simulation programme to improve patient safety

Abstract

Surgical patients are at particular risk of harm, with 41% of all adverse events in hospital occurring in the operating theatre. Failures in Human factors are the leading cause. Despite recognition of the importance of human factors training to patient safety, there is a lack of theatre ENT crisis management simulation, and no formal assessment of the requisite skills. Aims: To Develop a psychometrically robust assessment tool for assessing Non-technical skills in the ENT theatre – to be termed ENT-NOTECHS. To Develop and validate an ENT themed multidisciplinary simulation programme for the assessment and feedback of non-technical skills. Methods: A multimodal method approach was used to create a novel behavioural marker tool to capture non-technical skills in the ENT theatre environment. Alongside this, a prospective, observational study involving a multidisciplinary team training day in ENT and airway themed crisis’ in a high fidelity simulated theatre environment was designed. Teams undertook 6 high fidelity simulation scenarios and non-technical skills were assessed using the ENT-NOTECHS tool. The ENT-NOTECHS tool was assessed for its psychometric robustness; reliability and construct validity. Candidate feedback was obtained to determine overall effectiveness of training.

Results: We successfully designed and delivered a novel multidisciplinary team ENT themed training day. Over 15 months, 74 trainees (surgeons, anaesthetists and nurses) participated in 6 MDT simulation days, totalling 54 hours of simulation training and 210 assessments. Excellent Face and content validity was demonstrated. 100% of participants reported improved confidence in managing ENT crisis scenarios and demonstrated an improvement in non-technical skills (ENT-NOTECHS). The ENT-NOTECHS tool demonstrated excellent psychometric robustness. Good inter-rater reliability scores (cronbachs >0.7) were shown and the tool discriminated between novice and expert trainees (p<0.001).

Conclusion: Multidisciplinary team training in ENT-themed crisis is feasible and well received training intervention. The simulated operating theatre serves as an excellent environment for the assessment and training of non-technical skills. ENT -NOTECHS is a novel assessment tool with evidence for reliability, content and construct validity in ENT teams.