8
IRUS Total
Downloads

Assessment of factual recall and higher-order cognitive domains in an open-book medical school examination

File Description SizeFormat 
Davies2022_Article_AssessmentOfFactualRecallAndHi.pdfPublished version714.59 kBAdobe PDFView/Open
Title: Assessment of factual recall and higher-order cognitive domains in an open-book medical school examination
Authors: Davies, DJ
McLean, PF
Kemp, PR
Liddle, AD
Morrell, MJ
Halse, O
Martin, NM
Sam, AH
Item Type: Journal Article
Abstract: Open-book examinations (OBEs) will likely become increasingly important assessment tools. We investigated how access to open-book resources affected questions testing factual recall, which might be easy to look-up, versus questions testing higher-order cognitive domains. Few studies have investigated OBEs using modern Internet resources or as summative assessments. We compared performance on an examination conducted as a traditional closed-book exam (CBE) in 2019 (N = 320) and a remote OBE with free access to Internet resources in 2020 (N = 337) due to COVID-19. This summative, end-of-year assessment focused on basic science for second-year medical students. We categorized questions by Bloom’s taxonomy (‘Remember’, versus ‘Understand/Apply’). We predicted higher performance on the OBE, driven by higher performance on ‘Remember’ questions. We used an item-centric analysis by using performance per item over all examinees as the outcome variable in logistic regression, with terms ‘Open-Book, ‘Bloom Category’ and their interaction. Performance was higher on OBE questions than CBE questions (OR 2.2, 95% CI: 2.14–2.39), and higher on ‘Remember’ than ‘Understand/Apply’ questions (OR 1.13, 95% CI: 1.09–1.19). The difference in performance between ‘Remember’ and ‘Understand/Apply’ questions was greater in the OBE than the CBE (‘Open-Book’ * ‘Bloom Category’ interaction: OR 1.2, 95% CI: 1.19–1.37). Access to open-book resources had a greater effect on performance on factual recall questions than higher-order questions, though performance was higher in the OBE overall. OBE design must consider how searching for information affects performance, particularly on questions measuring different domains of knowledge.
Issue Date: 1-Mar-2022
Date of Acceptance: 19-Sep-2021
URI: http://hdl.handle.net/10044/1/92472
DOI: 10.1007/s10459-021-10076-5
ISSN: 1382-4996
Publisher: Springer
Start Page: 147
End Page: 165
Journal / Book Title: Advances in Health Sciences Education
Volume: 27
Copyright Statement: © The Author(s) 2021. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Keywords: Social Sciences
Science & Technology
Life Sciences & Biomedicine
Education & Educational Research
Education, Scientific Disciplines
Health Care Sciences & Services
Open-book examinations
Assessment
Medical education
TASK COMPLEXITY
MULTIPLE-CHOICE
BLOOMS TAXONOMY
INFORMATION
STUDENTS
PERFORMANCE
INTERNET
IMPACT
TESTS
SKILLS
Assessment
Medical education
Open-book examinations
COVID-19
Cognition
Educational Measurement
Humans
Schools, Medical
Students, Medical
Humans
Cognition
Educational Measurement
Schools, Medical
Students, Medical
COVID-19
Social Sciences
Science & Technology
Life Sciences & Biomedicine
Education & Educational Research
Education, Scientific Disciplines
Health Care Sciences & Services
Open-book examinations
Assessment
Medical education
TASK COMPLEXITY
MULTIPLE-CHOICE
BLOOMS TAXONOMY
INFORMATION
STUDENTS
PERFORMANCE
INTERNET
IMPACT
TESTS
SKILLS
Medical Informatics
1302 Curriculum and Pedagogy
Publication Status: Published
Open Access location: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8536902/
Online Publication Date: 2021-10-23
Appears in Collections:Department of Surgery and Cancer
National Heart and Lung Institute
Faculty of Medicine



This item is licensed under a Creative Commons License Creative Commons