X-Section: cross-section prediction for enhanced RGBD fusion
File(s)1903.00987v1.pdf (6.95 MB)
Working paper
Author(s)
Nicastro, Andrea
Clark, Ronald
Leutenegger, Stefan
Type
Working Paper
Abstract
Detailed 3D reconstruction is an important challenge with application to
robotics, augmented and virtual reality, which has seen impressive progress
throughout the past years. Advancements were driven by the availability of
depth cameras (RGB-D), as well as increased compute power, e.g.\ in the form of
GPUs -- but also thanks to inclusion of machine learning in the process. Here,
we propose X-Section, an RGB-D 3D reconstruction approach that leverages deep
learning to make object-level predictions about thicknesses that can be readily
integrated into a volumetric multi-view fusion process, where we propose an
extension to the popular KinectFusion approach. In essence, our method allows
to complete shape in general indoor scenes behind what is sensed by the RGB-D
camera, which may be crucial e.g.\ for robotic manipulation tasks or efficient
scene exploration. Predicting object thicknesses rather than volumes allows us
to work with comparably high spatial resolution without exploding memory and
training data requirements on the employed Convolutional Neural Networks. In a
series of qualitative and quantitative evaluations, we demonstrate how we
accurately predict object thickness and reconstruct general 3D scenes
containing multiple objects.
robotics, augmented and virtual reality, which has seen impressive progress
throughout the past years. Advancements were driven by the availability of
depth cameras (RGB-D), as well as increased compute power, e.g.\ in the form of
GPUs -- but also thanks to inclusion of machine learning in the process. Here,
we propose X-Section, an RGB-D 3D reconstruction approach that leverages deep
learning to make object-level predictions about thicknesses that can be readily
integrated into a volumetric multi-view fusion process, where we propose an
extension to the popular KinectFusion approach. In essence, our method allows
to complete shape in general indoor scenes behind what is sensed by the RGB-D
camera, which may be crucial e.g.\ for robotic manipulation tasks or efficient
scene exploration. Predicting object thicknesses rather than volumes allows us
to work with comparably high spatial resolution without exploding memory and
training data requirements on the employed Convolutional Neural Networks. In a
series of qualitative and quantitative evaluations, we demonstrate how we
accurately predict object thickness and reconstruct general 3D scenes
containing multiple objects.
Date Issued
2019-03-03
Citation
2019
Identifier
http://arxiv.org/abs/1903.00987v2
Subjects
cs.CV
cs.CV