Nonlinear fusion is optimal for a wide class of multisensory tasks
File(s)
Published version (uncorrected proof)
Author(s)
Ghosh, Marcus
Béna, Gabriel
Bormuth, Volker
Goodman, Dan FM
Type
Journal Article
Abstract
Animals continuously detect information via multiple sensory channels, like vision and hearing, and integrate these signals to realise faster and more accurate decisions; a fundamental neural computation known as multisensory integration. A widespread view of this process is that multimodal neurons linearly fuse information across sensory channels. However, does linear fusion generalise beyond the classical tasks used to explore multisensory integration? Here, we develop novel multisensory tasks, which focus on the underlying statistical relationships between channels, and deploy models at three levels of abstraction: from probabilistic ideal observers to artificial and spiking neural networks. Using these models, we demonstrate that when the information provided by different channels is not independent, linear fusion performs sub-optimally and even fails in extreme cases. This leads us to propose a simple nonlinear algorithm for multisensory integration which is compatible with our current knowledge of multimodal circuits, excels in naturalistic settings and is optimal for a wide class of multisensory tasks. Thus, our work emphasises the role of nonlinear fusion in multisensory integration, and provides testable hypotheses for the field to explore at multiple levels: from single neurons to behaviour.
Date Issued
2024-07-05
Date Acceptance
2024-06-11
Citation
PLoS Computational Biology, 2024, 20 (7)
ISSN
1553-734X
Publisher
Public Library of Science (PLoS)
Journal / Book Title
PLoS Computational Biology
Volume
20
Issue
7
Copyright Statement
© 2024 Ghosh et al. This is an open
access article distributed under the terms of the
Creative Commons Attribution License, which
permits unrestricted use, distribution, and
reproduction in any medium, provided the original
author and source are credited.
access article distributed under the terms of the
Creative Commons Attribution License, which
permits unrestricted use, distribution, and
reproduction in any medium, provided the original
author and source are credited.
License URL
Identifier
https://www.ncbi.nlm.nih.gov/pubmed/38968324
Publication Status
Published online
Coverage Spatial
United States
Article Number
e1012246
Date Publish Online
2024-07-05