Enhancing speech intelligibility through paralinguistic features
File(s)
Author(s)
He, Xiangheng
Type
Thesis or dissertation
Abstract
Speech is fundamental to human communication, distinguishing it from other forms, such as gestures, body language, and facial expressions, which are shared with other primates. A key factor in effective communication is speech intelligibility, which enables individuals to clearly convey and comprehend thoughts, emotions, and intentions, fostering meaningful interactions and social connections. Despite the contributions of paralinguistic features in human speech, their role in enhancing speech intelligibility, particularly in the context of human-machine communication, remains inadequately studied. This gap highlights the need for a deeper investigation into incorporating paralinguistic features into speech synthesis and analysis systems to improve speech intelligibility and enhance the effectiveness of human-machine communication.
This thesis aims to enhance speech intelligibility by enabling machines to better interpret speaker intentions in human speech and produce more comprehensible machine-generated speech through the incorporation of paralinguistic features into speech processing systems. To achieve this, this thesis first introduces a proposed unsupervised, flexible, and perceptually aligned prosody modeling approach for text-to-speech synthesis (TTS). This approach enables the development of a prosody-aware TTS model that achieves natural prosody, strong generalizability, and fine-grained prosody control, significantly enhancing speech intelligibility compared to existing TTS systems. This thesis then explores the role of emotion-related paralinguistic tasks in enhancing a machine's ability to interpret speaker intentions, proposing a novel training strategy that prioritizes high-value paralinguistic tasks while mitigating negative transfer, resulting in a model that better interprets speaker intentions. This thesis further proposes an emotional voice conversion model that leverages disentangled emotional features to achieve clean and distinct emotional representations, reducing distortions in converted speech while preserving emotional clarity and enhancing intelligibility. Extensive experiments conducted with various open-source datasets demonstrate that these proposed models are superior to the current state-of-the-art models.
This thesis aims to enhance speech intelligibility by enabling machines to better interpret speaker intentions in human speech and produce more comprehensible machine-generated speech through the incorporation of paralinguistic features into speech processing systems. To achieve this, this thesis first introduces a proposed unsupervised, flexible, and perceptually aligned prosody modeling approach for text-to-speech synthesis (TTS). This approach enables the development of a prosody-aware TTS model that achieves natural prosody, strong generalizability, and fine-grained prosody control, significantly enhancing speech intelligibility compared to existing TTS systems. This thesis then explores the role of emotion-related paralinguistic tasks in enhancing a machine's ability to interpret speaker intentions, proposing a novel training strategy that prioritizes high-value paralinguistic tasks while mitigating negative transfer, resulting in a model that better interprets speaker intentions. This thesis further proposes an emotional voice conversion model that leverages disentangled emotional features to achieve clean and distinct emotional representations, reducing distortions in converted speech while preserving emotional clarity and enhancing intelligibility. Extensive experiments conducted with various open-source datasets demonstrate that these proposed models are superior to the current state-of-the-art models.
Version
Open Access
Date Issued
2025-01-29
Date Awarded
2026-03-01
Copyright Statement
Attribution-NonCommercial 4.0 International Licence (CC BY-NC)
License URL
Advisor
Schuller, Bjoern
Publisher Department
Department of Computing
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)