From scene flow to visual odometry through local and global regularisation in markov random fields
File(s)Scona_etal_ICRA2022.pdf (4.65 MB)
Accepted version
Author(s)
Scona, Raluca
Matsuki, Hidenobu
Davison, Andrew
Type
Journal Article
Abstract
We revisit pairwise Markov Random Field (MRF) formulations for RGB-D scene flow and leverage novel advances in processor design for real-time implementations. We consider scene flow approaches which consist of data terms enforcing intensity consistency between consecutive images, together with regularisation terms which impose smoothness over the flow field. To achieve real-time operation, previous systems leveraged GPUs and implemented regularisation only between variables corresponding to neighbouring pixels. Such systems could estimate continuously deforming flow fields but the lack of global regularisation over the whole field made them ineffective for visual odometry. We leverage the GraphCore Intelligence Processing Unit (IPU) graph processor chip, which consists of 1216 independent cores called tiles, each with 256 kB local memory. The tiles are connected to an ultrafast all-to-all communication fabric which enables efficient data transmission between the tiles in an arbitrary communication pattern. We propose a distributed formulation for dense RGB-D scene flow based on Gaussian Belief Propagation which leverages the architecture of this processor to implement both local and global regularisation. Local regularisation is enforced for pairs of flow estimates whose corresponding pixels are neighbours, while global regularisation is defined for flow estimate pairs whose corresponding pixels are far from each other on the image plane. Using both types of regularisation allows our algorithm to handle a variety of in-scene motion and makes it suitable for estimating deforming scene flow, piece-wise rigid scene flow and visual odometry within the same system.
Date Issued
2022-04
Date Acceptance
2022-02-01
Citation
IEEE Robotics and Automation Letters, 2022, 7 (2), pp.4299-4306
ISSN
2377-3766
Publisher
Institute of Electrical and Electronics Engineers
Start Page
4299
End Page
4306
Journal / Book Title
IEEE Robotics and Automation Letters
Volume
7
Issue
2
Copyright Statement
© 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.
See https://www.ieee.org/publications/rights/index.html for more information.
Sponsor
Dyson Technology Limited
Engineering & Physical Science Research Council (EPSRC)
Identifier
https://ieeexplore.ieee.org/document/9712370
Grant Number
PO 4500501004
EP/S036636/1
Subjects
0913 Mechanical Engineering
Publication Status
Published online