Overlapping speaker segmentation using multiple hypothesis tracking of fundamental frequency
Author(s)
Hogg, Aidan
Evers, Christine
Moore, Alastair
Naylor, Patrick
Type
Journal Article
Abstract
This paper demonstrates how the harmonic structure of voiced speech can be exploited to segment multiple overlapping speakers in a speaker diarization task. We explore how a change in the speaker can be inferred from a change in pitch. We show that voiced harmonics can be useful in detecting when more than one speaker is talking, such as during overlapping speaker activity. A novel system is proposed to track multiple harmonics simultaneously, allowing for the determination of onsets and end-points of a speaker’s utterance in the presence of an additional active speaker. This system is bench-marked against a segmentation system from the literature that employs a bidirectional long short term memory network (BLSTM) approach and requires training. Experimental results highlight that the proposed approach outperforms the BLSTM baseline approach by 12.9% in terms of HIT rate for speaker segmentation. We also show that the estimated pitch tracks of our system can be used as features to the BLSTM to achieve further improvements of 1.21% in terms of coverage and 2.45% in terms of purity
Date Issued
2021-04-22
Date Acceptance
2021-03-07
Citation
IEEE/ACM Transactions on Audio, Speech and Language Processing, 2021, 29 (1), pp.1479-1490
ISSN
2329-9290
Publisher
Association for Computing Machinery (ACM)
Start Page
1479
End Page
1490
Journal / Book Title
IEEE/ACM Transactions on Audio, Speech and Language Processing
Volume
29
Issue
1
Copyright Statement
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. See https://www.ieee.org/publications/rights/index.html for more information.
Sponsor
Engineering & Physical Science Research Council (E
Engineering and Physical Sciences Research Council
Identifier
https://ieeexplore.ieee.org/document/9381673
Grant Number
EP/P001017/1
EP/L016796/1
Publication Status
Published
Date Publish Online
2021-03-18