Perspectives on procedure-based assessments: a thematic analysis of semistructured interviews with 10 UK surgical trainees
File(s)e013417.full.pdf (947.23 KB) PBA_manuscript_2016-013417R3_clean.docx (101.32 KB)
Published version
Accepted version
Author(s)
Shalhoub, J
Marshall, DC
Ippolito, K
Type
Journal Article
Abstract
Objectives: The introduction of competency-based training has necessitated development and implementation of accompanying mechanisms for assessment. Procedure based assessments (PBAs) are an example of workplace-based assessments that are used to examine focal competencies in the workplace. The primary objective was to understand surgical trainees’ perspective on the value of PBA.
Design: Semi-structured interviews with 10 surgical trainees individually interviewed to explore their views. Interviews were audio-recorded and transcribed, following this they were open and axial coded. Thematic analysis was then performed.
Results: Semi-structured interviews yielded several topical and recurring themes. In trainees’ experience the use of PBAs as a summative tool limits their educational value. Trainees reported a lack of support from seniors and variation in the usefulness of the tool based on stage of training. Concerns related to the validity of PBAs for evaluating trainees’ performance with reports of ‘gaming’ the system and trainees completing their own assessments. Trainees’ did identify the significant value of PBAs when used correctly. Benefits included the identification of additional learning opportunities, standardisation of assessment and their role in providing a measure of progress.
Conclusions: The UK surgical trainees interviewed identified both limitations and benefits to PBAs, however we would argue based on their responses and our experience that their use as a summative tool limits their formative use as an educational opportunity. PBAs should either be used exclusively to support learning or solely used as a summative tool, if so further work is needed to audit, validate and standardise them for this purpose.
Design: Semi-structured interviews with 10 surgical trainees individually interviewed to explore their views. Interviews were audio-recorded and transcribed, following this they were open and axial coded. Thematic analysis was then performed.
Results: Semi-structured interviews yielded several topical and recurring themes. In trainees’ experience the use of PBAs as a summative tool limits their educational value. Trainees reported a lack of support from seniors and variation in the usefulness of the tool based on stage of training. Concerns related to the validity of PBAs for evaluating trainees’ performance with reports of ‘gaming’ the system and trainees completing their own assessments. Trainees’ did identify the significant value of PBAs when used correctly. Benefits included the identification of additional learning opportunities, standardisation of assessment and their role in providing a measure of progress.
Conclusions: The UK surgical trainees interviewed identified both limitations and benefits to PBAs, however we would argue based on their responses and our experience that their use as a summative tool limits their formative use as an educational opportunity. PBAs should either be used exclusively to support learning or solely used as a summative tool, if so further work is needed to audit, validate and standardise them for this purpose.
Date Issued
2017-03-01
Date Acceptance
2017-02-06
Citation
BMJ OPEN, 2017, 7 (3)
ISSN
2044-6055
Publisher
BMJ PUBLISHING GROUP
Journal / Book Title
BMJ OPEN
Volume
7
Issue
3
Copyright Statement
© 2017 The Authors. Published by BMJ Publishing Group. This is an Open Access article distributed in accordance with
the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license,
which permits others to distribute, remix, adapt, build upon this work noncommercially,
and license their derivative works on different terms, provided
the original work is properly cited and the use is non-commercial. See: http://
creativecommons.org/licenses/by-nc/4.0/
the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license,
which permits others to distribute, remix, adapt, build upon this work noncommercially,
and license their derivative works on different terms, provided
the original work is properly cited and the use is non-commercial. See: http://
creativecommons.org/licenses/by-nc/4.0/
Identifier
http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000398959400069&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=1ba7043ffcc86c417c072aa74d649202
Subjects
Science & Technology
Life Sciences & Biomedicine
Medicine, General & Internal
General & Internal Medicine
WORKPLACE-BASED ASSESSMENT
MULTISOURCE FEEDBACK
QUALITATIVE RESEARCH
PERFORMANCE
EDUCATION
ASSOCIATION
EXERCISE
FUTURE
Publication Status
Published
Article Number
ARTN e013417