8
IRUS TotalDownloads
Assessment of factual recall and higher-order cognitive domains in an open-book medical school examination
File | Description | Size | Format | |
---|---|---|---|---|
![]() | Published version | 714.59 kB | Adobe PDF | View/Open |
Title: | Assessment of factual recall and higher-order cognitive domains in an open-book medical school examination |
Authors: | Davies, DJ McLean, PF Kemp, PR Liddle, AD Morrell, MJ Halse, O Martin, NM Sam, AH |
Item Type: | Journal Article |
Abstract: | Open-book examinations (OBEs) will likely become increasingly important assessment tools. We investigated how access to open-book resources affected questions testing factual recall, which might be easy to look-up, versus questions testing higher-order cognitive domains. Few studies have investigated OBEs using modern Internet resources or as summative assessments. We compared performance on an examination conducted as a traditional closed-book exam (CBE) in 2019 (N = 320) and a remote OBE with free access to Internet resources in 2020 (N = 337) due to COVID-19. This summative, end-of-year assessment focused on basic science for second-year medical students. We categorized questions by Bloom’s taxonomy (‘Remember’, versus ‘Understand/Apply’). We predicted higher performance on the OBE, driven by higher performance on ‘Remember’ questions. We used an item-centric analysis by using performance per item over all examinees as the outcome variable in logistic regression, with terms ‘Open-Book, ‘Bloom Category’ and their interaction. Performance was higher on OBE questions than CBE questions (OR 2.2, 95% CI: 2.14–2.39), and higher on ‘Remember’ than ‘Understand/Apply’ questions (OR 1.13, 95% CI: 1.09–1.19). The difference in performance between ‘Remember’ and ‘Understand/Apply’ questions was greater in the OBE than the CBE (‘Open-Book’ * ‘Bloom Category’ interaction: OR 1.2, 95% CI: 1.19–1.37). Access to open-book resources had a greater effect on performance on factual recall questions than higher-order questions, though performance was higher in the OBE overall. OBE design must consider how searching for information affects performance, particularly on questions measuring different domains of knowledge. |
Issue Date: | 1-Mar-2022 |
Date of Acceptance: | 19-Sep-2021 |
URI: | http://hdl.handle.net/10044/1/92472 |
DOI: | 10.1007/s10459-021-10076-5 |
ISSN: | 1382-4996 |
Publisher: | Springer |
Start Page: | 147 |
End Page: | 165 |
Journal / Book Title: | Advances in Health Sciences Education |
Volume: | 27 |
Copyright Statement: | © The Author(s) 2021. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. |
Keywords: | Social Sciences Science & Technology Life Sciences & Biomedicine Education & Educational Research Education, Scientific Disciplines Health Care Sciences & Services Open-book examinations Assessment Medical education TASK COMPLEXITY MULTIPLE-CHOICE BLOOMS TAXONOMY INFORMATION STUDENTS PERFORMANCE INTERNET IMPACT TESTS SKILLS Assessment Medical education Open-book examinations COVID-19 Cognition Educational Measurement Humans Schools, Medical Students, Medical Humans Cognition Educational Measurement Schools, Medical Students, Medical COVID-19 Social Sciences Science & Technology Life Sciences & Biomedicine Education & Educational Research Education, Scientific Disciplines Health Care Sciences & Services Open-book examinations Assessment Medical education TASK COMPLEXITY MULTIPLE-CHOICE BLOOMS TAXONOMY INFORMATION STUDENTS PERFORMANCE INTERNET IMPACT TESTS SKILLS Medical Informatics 1302 Curriculum and Pedagogy |
Publication Status: | Published |
Open Access location: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8536902/ |
Online Publication Date: | 2021-10-23 |
Appears in Collections: | Department of Surgery and Cancer National Heart and Lung Institute Faculty of Medicine |
This item is licensed under a Creative Commons License