Studying human-based speaker diarization and comparing to state-of-the-art systems
Author(s)
McKnight, Simon
Hogg, Aidan OT
Neo, Vincent W
Naylor, Patrick A
Type
Conference Paper
Abstract
Human-based speaker diarization experiments were carried out on a five-minute extract of a typical AMI corpus meeting to see how much variance there is in human reviews based on hearing only and to compare with state-of-the-art diarization systems on the same extract. There are three distinct experiments: (a) one with no prior information; (b) one with the ground truth speech activity detection (GT-SAD); and (c) one with the blank ground truth labels (GT-labels). The results show that most human reviews tend to be quite similar, albeit with some outliers, but the choice of GT-labels can make a dramatic difference to scored performance. Using the GT-SAD provides a big advantage and improves human review scores substantially, though small differences in the GT-SAD used can have a dramatic effect on results. The use of forgiveness collars is shown to be unhelpful. The results show that state-of-the-art systems can outperform the best human reviews when no prior information is provided. However, the best human reviews still outperform state-of-the-art systems when starting from the GT-SAD.
Date Issued
2022-12-21
Date Acceptance
2022-09-08
Citation
Proceedings of 2022 APSIPA Annual Summit and Conference, 2022, pp.394-401
Publisher
IEEE
Start Page
394
End Page
401
Journal / Book Title
Proceedings of 2022 APSIPA Annual Summit and Conference
Copyright Statement
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Source
APSIPA 2022
Publication Status
Published
Start Date
2022-11-07
Finish Date
2022-11-10
Coverage Spatial
Chiang Mai, Thailand