Dense image alignment from RGB-D images remains a critical issue for
real-world applications, especially under challenging lighting conditions and
in a wide baseline setting. In this paper, we propose a new framework to learn
a pixel-wise deep feature map and a deep feature-metric uncertainty map
predicted by a Convolutional Neural Network (CNN), which together formulate a
deep probabilistic feature-metric residual of the two-view constraint that can
be minimised using Gauss-Newton in a coarse-to-fine optimisation framework.
Furthermore, our network predicts a deep initial pose for faster and more
reliable convergence. The optimisation steps are differentiable and unrolled to
train in an end-to-end fashion. Due to its probabilistic essence, our approach
can easily couple with other residuals, where we show a combination with ICP.
Experimental results demonstrate state-of-the-art performance on the TUM RGB-D
dataset and 3D rigid object tracking dataset. We further demonstrate our
method's robustness and convergence qualitatively.