Active recognition and domain adaptation for 6D object pose estimation
File(s)
Author(s)
Sock, Ju Il
Type
Thesis
Abstract
Object pose estimation is an important task in computer vision with a wide range of applications including robotics and Augmented Reality. I investigate the problems of occlusion and lack of annotated real data in 6D object pose estimation in this thesis.
The main contributions of this thesis are building a 6D object pose estimation system for a crowded scene for both single image and multiple images, and a proposal of a new self-supervised learning framework.
The thesis first introduces a single-image pose estimation framework for a crowded scene, where the objects are under severe occlusions. Existing 6D pose estimators are unable to tackle such challenging scenarios, motivating the research towards end-to-end multi-task learning framework. A new training data generation framework is proposed which produces images of physically plausible object pose with a physics simulation.
The second contribution of this thesis explores how a strategic selection of camera movements can facilitate the task of 6D multi-object pose estimation for the crowded scenarios while respecting real-world constraints. The proposed agent learns the camera moves that produce the accurate object poses hypotheses for a given temporal and spatial budget.
The last contribution of this thesis is its introduction of self-supervised learning framework for domain adaptation for 6D object pose estimation. Conventionally, Deep Neural Network (DNN) based methods are trained primarily using rendered images of 3D object models. Since such methods do not generalise well to real testing data, DNNs also exploit real RGB images with accurate 6D pose labels. The issue here is in a lack of real annotated data, since annotating 6D poses for 2D images in quality and quantity is hard. A novel self-supervised learning method is proposed, which results in a domain-invariant 6D pose estimator without pose labels of real data.
The main contributions of this thesis are building a 6D object pose estimation system for a crowded scene for both single image and multiple images, and a proposal of a new self-supervised learning framework.
The thesis first introduces a single-image pose estimation framework for a crowded scene, where the objects are under severe occlusions. Existing 6D pose estimators are unable to tackle such challenging scenarios, motivating the research towards end-to-end multi-task learning framework. A new training data generation framework is proposed which produces images of physically plausible object pose with a physics simulation.
The second contribution of this thesis explores how a strategic selection of camera movements can facilitate the task of 6D multi-object pose estimation for the crowded scenarios while respecting real-world constraints. The proposed agent learns the camera moves that produce the accurate object poses hypotheses for a given temporal and spatial budget.
The last contribution of this thesis is its introduction of self-supervised learning framework for domain adaptation for 6D object pose estimation. Conventionally, Deep Neural Network (DNN) based methods are trained primarily using rendered images of 3D object models. Since such methods do not generalise well to real testing data, DNNs also exploit real RGB images with accurate 6D pose labels. The issue here is in a lack of real annotated data, since annotating 6D poses for 2D images in quality and quantity is hard. A novel self-supervised learning method is proposed, which results in a domain-invariant 6D pose estimator without pose labels of real data.
Version
Open Access
Date Issued
2020-09
Date Awarded
2021-06
Copyright Statement
Creative Commons Attribution NonCommercial NoDerivatives Licence
License URL
Advisor
Kim, Tae-Kyun
Publisher Department
Electrical and Electronic Engineering
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)