Automatic nonverbal mimicry detection and analysis in medical video consultations
File(s)
Supporting information
Supporting information
Author(s)
Wu, Kaihang
Liu, Chunfeng
Calvo, Rafael A
Type
Journal Article
Abstract
Effective medical consultation requires good nonverbal communication between patients and doctors. Nonverbal behavior (NVB) mimicry, when a person imitates the NVB of the conversation partner, has been associated with building rapport between people. We study how NVB mimicry influences the quality of a medical consultation in a video conference. Computer vision algorithms were used to analyze 2027.75 minutes of recorded medical video consultations between medical students (N = 130) and volunteers who acted as Simulated Patients (SP, N = 29). We automatically detected students’ NVB mimicry to the SP and investigated the relationship between the NVB mimicry and students’ overall communication skills performance. The results highlighted the positive correlation between head nodding mimicry and assessed communication skills. In addition, we also identified NVB mimicry styles by cluster analysis and found that certain NVB mimicry styles have associated with better communication performance. These findings provide insights into the future design of systems that offer automatic feedback in communication skills training.
Date Issued
2020-05-07
Date Acceptance
2020-04-03
Citation
International Journal of Human–Computer Interaction, 2020, 36 (14), pp.1379-1392
ISSN
1044-7318
Publisher
Informa UK Limited
Start Page
1379
End Page
1392
Journal / Book Title
International Journal of Human–Computer Interaction
Volume
36
Issue
14
Copyright Statement
© 2020 The Author(s). Published with license by Taylor & Francis Group, LLC. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way.
Sponsor
Australian Research Council
Identifier
https://www.tandfonline.com/doi/full/10.1080/10447318.2020.1752474
Grant Number
FT140100824
Subjects
Science & Technology
Technology
Computer Science, Cybernetics
Ergonomics
Computer Science
Engineering
COMMUNICATION-SKILLS
EMOTIONAL EXPRESSIONS
PERCEPTION
RAPPORT
EMPATHY
SYNCHRONY
MOVEMENT
BEHAVIOR
LINK
Human Factors
08 Information and Computing Sciences
16 Studies in Human Society
Publication Status
Published
Date Publish Online
2020-05-07