Towards the probabilistic fusion of learned priors into standard pipelines for 3D reconstruction
File(s)
Accepted version
Author(s)
Laidlow, Tristan
Czarnowski, Jan
Nicastro, Andrea
Clark, Ronald
Leutenegger, Stefan
Type
Conference Paper
Abstract
The best way to combine the results of deep learning with standard 3D reconstruction pipelines remains an open problem. While systems that pass the output of traditional multi-view stereo approaches to a network for regularisation or refinement currently seem to get the best results, it may be preferable to treat deep neural networks as separate components whose results can be probabilistically fused into geometry- based systems. Unfortunately, the error models required to do this type of fusion are not well understood, with many different approaches being put forward. Recently, a few systems have achieved good results by having their networks predict probability distributions rather than single values. We propose using this approach to fuse a learned single-view depth prior into a standard 3D reconstruction system. Our system is capable of incrementally producing dense depth maps for a set of keyframes. We train a deep neural network to predict discrete, nonparametric probability distributions for the depth of each pixel from a single image. We then fuse this "probability volume" with another probability volume based on the photometric consistency between subsequent frames and the keyframe image. We argue that combining the probability volumes from these two sources will result in a volume that is better conditioned. To extract depth maps from the volume, we minimise a cost function that includes a regularisation term based on network predicted surface normals and occlusion boundaries. Through a series of experiments, we demonstrate that each of these components improves the overall performance of the system.
Date Issued
2020-09-15
Date Acceptance
2020-09-01
Citation
2020 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), 2020, pp.7373-7379
ISSN
1050-4729
Publisher
IEEE
Start Page
7373
End Page
7379
Journal / Book Title
2020 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA)
Copyright Statement
© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Sponsor
Engineering & Physical Science Research Council (EPSRC)
Dyson Technology Limited
Identifier
http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000712319504134&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=1ba7043ffcc86c417c072aa74d649202
Grant Number
EP/S036636/1
PO4500503359
Source
IEEE International Conference on Robotics and Automation (ICRA)
Subjects
Science & Technology
Technology
Automation & Control Systems
Engineering, Electrical & Electronic
Robotics
Engineering
Publication Status
Published
Start Date
2020-05-31
Finish Date
2020-06-15
Coverage Spatial
Paris, France
Date Publish Online
2020-09-15