Active recognition and domain adaptation for 6D object pose estimation

Abstract

Object pose estimation is an important task in computer vision with a wide range of applications including robotics and Augmented Reality. I investigate the problems of occlusion and lack of annotated real data in 6D object pose estimation in this thesis.

The main contributions of this thesis are building a 6D object pose estimation system for a crowded scene for both single image and multiple images, and a proposal of a new self-supervised learning framework.

The thesis first introduces a single-image pose estimation framework for a crowded scene, where the objects are under severe occlusions. Existing 6D pose estimators are unable to tackle such challenging scenarios, motivating the research towards end-to-end multi-task learning framework. A new training data generation framework is proposed which produces images of physically plausible object pose with a physics simulation.

The second contribution of this thesis explores how a strategic selection of camera movements can facilitate the task of 6D multi-object pose estimation for the crowded scenarios while respecting real-world constraints. The proposed agent learns the camera moves that produce the accurate object poses hypotheses for a given temporal and spatial budget.

The last contribution of this thesis is its introduction of self-supervised learning framework for domain adaptation for 6D object pose estimation. Conventionally, Deep Neural Network (DNN) based methods are trained primarily using rendered images of 3D object models. Since such methods do not generalise well to real testing data, DNNs also exploit real RGB images with accurate 6D pose labels. The issue here is in a lack of real annotated data, since annotating 6D poses for 2D images in quality and quantity is hard. A novel self-supervised learning method is proposed, which results in a domain-invariant 6D pose estimator without pose labels of real data.