Automatic nonverbal mimicry detection and analysis in medical video consultations

Abstract

Effective medical consultation requires good nonverbal communication between patients and doctors. Nonverbal behavior (NVB) mimicry, when a person imitates the NVB of the conversation partner, has been associated with building rapport between people. We study how NVB mimicry influences the quality of a medical consultation in a video conference. Computer vision algorithms were used to analyze 2027.75 minutes of recorded medical video consultations between medical students (N = 130) and volunteers who acted as Simulated Patients (SP, N = 29). We automatically detected students’ NVB mimicry to the SP and investigated the relationship between the NVB mimicry and students’ overall communication skills performance. The results highlighted the positive correlation between head nodding mimicry and assessed communication skills. In addition, we also identified NVB mimicry styles by cluster analysis and found that certain NVB mimicry styles have associated with better communication performance. These findings provide insights into the future design of systems that offer automatic feedback in communication skills training.