Multi-shape free-form deformation framework for efficient data transmission in AR-based medical training simulators
File(s)applsci-11-09925-v2.pdf (4.61 MB)
Published version
Author(s)
Kim, Myeongjin
Bello, Fernando
Type
Journal Article
Abstract
Augmented reality medical training simulators can provide a realistic and immersive experience by overlapping the virtual scene on to the real world. Latency in augmented reality (AR) medical training simulators is an important issue as it can lead to motion sickness for users. This paper proposes a framework that can achieve real-time rendering of the 3D scene aligned to the real world using a head-mounted display (HMD). Model deformation in the 3D scene is categorised into local deformation derived from user interaction and global deformation determined by the simulation scenario. Target shapes are predefined by a simulation scenario, and control points are placed to embed the predefined shapes. Free-form deformation (FFD) is applied to multiple shapes to efficiently transfer the simulated model to the HMD. Global deformation is computed by blending a mapping matrix of each FFD with an assigned weighting value. The local and global deformation are then transferred through the control points updated from a deformed surface mesh and its corresponding weighting value. The proposed framework is verified in terms of latency caused by data transmission and the accuracy of a transmitted surface mesh in a vaginal examination (VE) training simulation. The average latency is reduced to 7 ms, less than the latency causing motion sickness in virtual reality simulations. The maximum relative error is less than 3%. Our framework allows seamless rendering of a virtual scene to the real world with substantially reduced latency and without the need for an external tracking system.
Date Issued
2021-10-24
Date Acceptance
2021-10-20
Citation
Applied Sciences, 2021, 11 (21), pp.1-13
ISSN
2076-3417
Publisher
MDPI AG
Start Page
1
End Page
13
Journal / Book Title
Applied Sciences
Volume
11
Issue
21
Copyright Statement
© 2021 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/)
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/)
License URL
Sponsor
Imperial Health Charity
Identifier
https://www.mdpi.com/2076-3417/11/21/9925
Grant Number
161715a
Subjects
Science & Technology
Physical Sciences
Technology
Chemistry, Multidisciplinary
Engineering, Multidisciplinary
Materials Science, Multidisciplinary
Physics, Applied
Chemistry
Engineering
Materials Science
Physics
augmented reality
latency
head-mounted display
medical simulation
free-form deformation
PERCEPTION
STIFFNESS
Publication Status
Published
Date Publish Online
2021-10-24