Real-time mapping of physical scene properties with an autonomous robot experimenter
File(s)
Published version
Author(s)
Haughton, I
Sucar, E
Mouton, A
Johns, E
Davison, AJ
Type
Conference Paper
Abstract
Neural fields can be trained from scratch to represent the shape and appearance of 3D scenes efficiently. It has also been shown that they can densely map correlated properties such as semantics, via sparse interactions from a human labeller. In this work, we show that a robot can densely annotate a scene with arbitrary discrete or continuous physical properties via its own fully-autonomous experimental interactions, as it simultaneously scans and maps it with an RGB-D camera. A variety of scene interactions are possible, including poking with force sensing to determine rigidity, measuring local material type with single-pixel spectroscopy or predicting force distributions by pushing. Sparse experimental interactions are guided by entropy to enable high efficiency, with tabletop scene properties densely mapped from scratch in a few minutes from a few tens of interactions.
Date Issued
2023-01-01
Date Acceptance
2022-12-01
Citation
Proceedings of Machine Learning Research, 2023, 205, pp.118-127
Start Page
118
End Page
127
Journal / Book Title
Proceedings of Machine Learning Research
Volume
205
Copyright Statement
Copyright © The authors and PMLR 2023. MLResearch Press.
Source
6th Conference on Robot Learning
Publication Status
Published
Start Date
2022-12-14
Finish Date
2022-12-18
Coverage Spatial
Auckland, New Zealand