Multimodal imaging platform for enhanced tumor resection in neurosurgery: integrating hyperspectral and pCLE technologies
File(s)
Published version
Author(s)
Type
Journal Article
Abstract
Purpose
This work presents a novel multimodal imaging platform that integrates hyperspectral imaging (HSI) and probe-based confocal laser endomicroscopy (pCLE) for improved brain tumor identification during neurosurgery. By combining these two modalities, we aim to enhance surgical navigation, addressing the limitations of using each modality when used independently.
Methods
We developed a multimodal imaging platform that integrates HSI and pCLE within an operating microscope setup using computer vision techniques. The system combines real-time, high-resolution HSI for macroscopic tissue analysis with pCLE for cellular-level imaging. The predictions of each modality made using Machine Learning methods are combined to improve tumor identification.
Results
Our evaluation of the multimodal system revealed low spatial error, with minimal reprojection discrepancies, ensuring precise alignment between the HSI and pCLE. This combined imaging approach together with our multimodal tissue characterization algorithm significantly improves tumor identification, yielding higher Dice and Recall scores compared to using HSI or pCLE individually.
Conclusion
Our multimodal imaging platform represents a crucial first step toward enhancing tumor identification by combining HSI and pCLE modalities for the first time. We highlight improvements in metrics such as the Dice score and Recall, underscoring the potential for further advancements in this area.
This work presents a novel multimodal imaging platform that integrates hyperspectral imaging (HSI) and probe-based confocal laser endomicroscopy (pCLE) for improved brain tumor identification during neurosurgery. By combining these two modalities, we aim to enhance surgical navigation, addressing the limitations of using each modality when used independently.
Methods
We developed a multimodal imaging platform that integrates HSI and pCLE within an operating microscope setup using computer vision techniques. The system combines real-time, high-resolution HSI for macroscopic tissue analysis with pCLE for cellular-level imaging. The predictions of each modality made using Machine Learning methods are combined to improve tumor identification.
Results
Our evaluation of the multimodal system revealed low spatial error, with minimal reprojection discrepancies, ensuring precise alignment between the HSI and pCLE. This combined imaging approach together with our multimodal tissue characterization algorithm significantly improves tumor identification, yielding higher Dice and Recall scores compared to using HSI or pCLE individually.
Conclusion
Our multimodal imaging platform represents a crucial first step toward enhancing tumor identification by combining HSI and pCLE modalities for the first time. We highlight improvements in metrics such as the Dice score and Recall, underscoring the potential for further advancements in this area.
Date Issued
2025-06-01
Date Acceptance
2025-02-28
Citation
International journal of computer assisted radiology and surgery, 2025, 20 (6), pp.1087-1096
ISSN
1861-6410
Publisher
Springer
Start Page
1087
End Page
1096
Journal / Book Title
International journal of computer assisted radiology and surgery
Volume
20
Issue
6
Copyright Statement
© TheAuthor(s) 2025 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
License URL
Identifier
https://www.ncbi.nlm.nih.gov/pubmed/40180672
PII: 10.1007/s11548-025-03340-1
Subjects
ENDOMICROSCOPY
Engineering
Engineering, Biomedical
GLIOMAS
GUIDED RESECTION
Hyperspectral imaging
INTRAOPERATIVE MRI
Life Sciences & Biomedicine
MOTOR CORTEX
Multimodal imaging
Neurosurgery
PCLE
Platform
Radiology, Nuclear Medicine & Medical Imaging
Science & Technology
Surgery
SURVIVAL
Technology
Publication Status
Published
Coverage Spatial
Germany
Date Publish Online
2025-04-03