IRUS Total

Predicting secondary task performance: a directly actionable metric for cognitive overload detection

File Description SizeFormat 
IEEE_DecPred_stamped.pdfAccepted version1.52 MBAdobe PDFView/Open
Title: Predicting secondary task performance: a directly actionable metric for cognitive overload detection
Authors: Amadori, PV
Fischer, T
Wang, R
Demiris, Y
Item Type: Journal Article
Abstract: In this paper, we address cognitive overload detection from unobtrusive physiological signals for users in dual-tasking scenarios. Anticipating cognitive overload is a pivotal challenge in interactive cognitive systems and could lead to safer shared-control between users and assistance systems. Our framework builds on the assumption that decision mistakes on the cognitive secondary task of dual-tasking users correspond to cognitive overload events, wherein the cognitive resources required to perform the task exceed the ones available to the users. We propose DecNet, an end-to-end sequence-to-sequence deep learning model that infers in real-time the likelihood of user mistakes on the secondary task, i.e., the practical impact of cognitive overload, from eye-gaze and head-pose data. We train and test DecNet on a dataset collected in a simulated driving setup from a cohort of 20 users on two dual-tasking decision-making scenarios, with either visual or auditory decision stimuli. DecNet anticipates cognitive overload events in both scenarios and can perform in time-constrained scenarios, anticipating cognitive overload events up to 2s before they occur. We show that DecNet’s performance gap between audio and visual scenarios is consistent with user perceived difficulty. This suggests that single modality stimulation induces higher cognitive load on users, hindering their decision-making abilities.
Issue Date: 1-Dec-2022
Date of Acceptance: 1-Sep-2021
URI: http://hdl.handle.net/10044/1/91951
DOI: 10.1109/tcds.2021.3114162
ISSN: 2379-8920
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Start Page: 1474
End Page: 1485
Journal / Book Title: IEEE Transactions on Cognitive and Developmental Systems
Volume: 14
Issue: 4
Copyright Statement: © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Publication Status: Published
Online Publication Date: 2021-09-21
Appears in Collections:Faculty of Engineering