Toward multisensory digital interfaces
File(s)
Author(s)
Devillard, Alexis
Type
Thesis
Abstract
The integration of haptic sensing and feedback into digital interfaces presents an opportunity to enhance human-machine interactions by enabling users to engage with digital content through touch. This thesis addresses the design of compact, efficient, and user-friendly haptic systems capable of capturing and rendering tactile information. The first contribution identifies haptic features in bare-finger interactions (force, vibration, and friction) using a multimodal dataset of synchronised force, vibration, auditory, and visual signals collected during fingertip exploration of textured surfaces. Unlike previous tool-mediated studies, this dataset links physical measurements with perceptual evaluations, revealing complementary roles of modalities in texture classification. Friction-induced vibrations recorded on the nail and phalanx highlight the finger's mechanical role in encoding tactile information, consistent with recent neuromechanical models of the skin. To experimentally evaluate such models, a modular, multimodal, and multinodal electronic skin (e-skin) was developed using accessible components and straightforward fabrication techniques, including 3D printing and silicone moulding. Mounted on a robotic arm, the e-skin captured normal and shear forces as well as friction-induced vibrations during 6D interactions with deformable objects. Results show that combining its signals with computational models enables extraction of invariant object parameters from noisy sensor data, establishing the e-skin as an accessible bioinspired platform for studying haptic encoding. Beyond sensing, the thesis explores feedback rendering with lightweight, unobtrusive devices: DeepScreen, an actuated touchscreen providing controllable normal-force feedback, and NaptX, a nail-mounted device delivering tactile cues without interfering with fingertip use. Both preserve natural interaction while enhancing tactile realism. User studies demonstrate their effectiveness and reveal perceptual ambiguities between nail- and fingertip-based stimulation, suggesting potential for exploiting haptic illusions to simplify rendering systems. Finally, a haptic selector integrates these technologies to enable texture-based search and retrieval within digital interfaces, demonstrating applications in e-commerce, training, and teleoperation.
Version
Open Access
Date Issued
2024-12-06
Date Awarded
01/11/2025
License URL
Advisor
Burdet, Etienne
Sponsor
Intuitive Foundation
Grant Number
ITN 861166
Publisher Department
Department of Bioengineering
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)