A comprehensive evaluation of work and simulation based assessment in otolaryngology training

Abstract

Introduction: The otolaryngology curriculum requires trainees to show evidence of operative competence before completion of training. The General Medical Council recommended that structured assessment be used throughout training to monitor and guide trainee progression. Despite the reduction in operative exposure and the variation in trainee performance, a ‘one size fits all’ approach continues to be applied. The number of procedures performed remains the main indicator of competence. Objectives: To analyse the utilisation, reliability and validity of workplace-based assessments in otolaryngology training. To identify, develop and validate a series of simulation platforms suitable for incorporation into the otolaryngology curriculum. To develop a model of interchangeable workplace- and simulation-based assessment that reflects trainee’s trajectory, audit the delivery of training and set milestones for modular learning. Methods: A detailed review of the literature identified a list of procedure-specific assessment tools as well as simulators suitable to be used as assessment platforms. A simulation-integrated training programme was piloted and models were tested for feasibility, face, content and construct validity before being incorporated into the North London training programme. The outcomes of workplace- and simulation-based assessments of all core and specialty otolaryngology trainees were collated and analysed. Results: The outcomes of 6535 workplace-based assessments were analysed. The strengths and weaknesses of 4 different assessment tools are highlighted. Validated platforms utilising cadavers, animal tissue, synthetic material and virtual reality simulators were incorporated into the curriculum. 60 trainees and 40 consultants participated in the process and found it of great educational value. Conclusion: Assessment with structured feedback is integral to surgical training. Assessment using validated simulation modules can complement that undertaken in the workplace. The outcomes of structures assessments can be used to monitor and guide trainee trajectory at individual and regional level. The derived learning curves can shape and audit future otolaryngological training.