Neural mechanisms of audio-tactile speech integration
File(s)
Author(s)
Guilleminot, Pierre Hieu
Type
Thesis
Abstract
The ability to understand speech in noisy environments is a remarkable feat of the human brain. It is further enhanced by the capability to integrate information from several modalities. Most notably, audiovisual information improves speech-in-noise comprehension tremendously. Yet, the neural mechanisms underlying this remarkable property remain poorly understood. This is partly due to the complexity of the interactions between speech and visual signals. In contrast, tactile signals can conveniently be designed to transmit simple information targeting specific features of speech. They have also been shown to elicit multisensory activity in the auditory cortex, and thus have the potential to be used as both a neuroscientific tool and a hearing aid.
In this thesis, we engineered tactile stimuli with the goals of improving speech-in-noise com- prehension and investigating the neural mechanisms underlying multisensory speech perception. First, we derived tactile pulses from the perceptual centers of syllables and measured their effect on speech comprehension and neural activity. Both measures were modulated according to the alignment between auditory and tactile streams, displaying oscillatory behaviours. Secondly, we measured the effect of rhythmic tactile pulses on a syllable discrimination task. We analo- gously observed oscillations, albeit damped, of both syllable discrimination and neural activity. Finally, we built a spiking neural network using multisensory phase reset to model audio-tactile speech integration. This model consistently reproduced our main results, thus suggesting that multisensory phase reset is a viable mechanisms to explain early audio-tactile speech integration.
The proposed work shed lights on the neural mechanisms underlying multisensory speech processing and how they can be leveraged to improve comprehension. It also highlights the possibility to design and use tactile stimuli to investigate specific stages of the speech processing hierarchy.
In this thesis, we engineered tactile stimuli with the goals of improving speech-in-noise com- prehension and investigating the neural mechanisms underlying multisensory speech perception. First, we derived tactile pulses from the perceptual centers of syllables and measured their effect on speech comprehension and neural activity. Both measures were modulated according to the alignment between auditory and tactile streams, displaying oscillatory behaviours. Secondly, we measured the effect of rhythmic tactile pulses on a syllable discrimination task. We analo- gously observed oscillations, albeit damped, of both syllable discrimination and neural activity. Finally, we built a spiking neural network using multisensory phase reset to model audio-tactile speech integration. This model consistently reproduced our main results, thus suggesting that multisensory phase reset is a viable mechanisms to explain early audio-tactile speech integration.
The proposed work shed lights on the neural mechanisms underlying multisensory speech processing and how they can be leveraged to improve comprehension. It also highlights the possibility to design and use tactile stimuli to investigate specific stages of the speech processing hierarchy.
Version
Open Access
Date Issued
2023-11
Date Awarded
2024-02
Copyright Statement
Creative Commons Attribution NonCommercial Licence
License URL
Advisor
Reichenbach, Tobias
Burdet, Etienne
Sponsor
Engineering and Physical Sciences Research Council
Publisher Department
Bioengineering
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)