Discrimination between native and non-native speech using visual features only
File(s)
Accepted version
Author(s)
Georgakis, C
Petridis, S
Pantic, M
Type
Journal Article
Abstract
Accent is a soft biometric trait that can be inferred from pronunciation and articulation patterns characterizing the speaking style of an individual. Past research has addressed the task of classifying accent, as belonging to a native language speaker or a foreign language speaker, by means of the audio modality only. However, features extracted from the visual stream of speech have been successfully used to extend or substitute audio-only approaches that target speech or language recognition. Motivated by these findings, we investigate to what extent temporal visual speech dynamics attributed to accent can be modeled and identified when the audio stream is missing or noisy, and the speech content is unknown. We present here a fully automated approach to discriminating native from non-native English speech, based exclusively on visual cues. A systematic evaluation of various appearance and shape features for the target problem is conducted, with the former consistently yielding superior performance. Subject-independent cross-validation experiments are conducted on mobile phone recordings of continuous speech and isolated word utterances spoken by 56 subjects from the challenging MOBIO database. High performance is achieved on a text-dependent (TD) protocol, with the best score of 76.5% yielded by fusion of five hidden Markov models trained on appearance features. Our framework is also efficient even when tested on examples of speech unseen in the training phase, although performing less accurately compared to the TD case.
Date Issued
2015-10-26
Date Acceptance
2015-09-29
Citation
IEEE Transactions on Cybernetics, 2015, 46 (12), pp.2758-2771
ISSN
2168-2275
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Start Page
2758
End Page
2771
Journal / Book Title
IEEE Transactions on Cybernetics
Volume
46
Issue
12
Copyright Statement
© 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Identifier
http://www.ncbi.nlm.nih.gov/pubmed/26513822
Publication Status
Published