Domain adaptation for semantic and 3D tasks
File(s)
Author(s)
Lopez Rodriguez, Adrian
Type
Thesis
Abstract
Data is the main constraint when training Deep Learning models. Real-domain data is costly to
annotate and, despite the abundance of already annotated data, training and testing on different distributions
leads to low performance due to the so-called domain gap. This domain gap can be bridged
with Domain Adaptation methods, which have been mostly researched within an image classification
setting. Other tasks, which can be more difficult to annotate, have been less researched. We thus aim
to find the efficacy of standard Domain Adaptation techniques for a set of semantic and 3D tasks.
We first investigate object detection in a multi-style target dataset setting. For this task we propose
three modules based on common Domain Adaptation techniques that are tailored to the challenges of
object detection (e.g., negative-examples sampling or multiple local predictions per image) and the
characteristics of a multi-style dataset. We hence propose to jointly use feature consistency and data
adaptation with a multi-style focus to reduce the effect of the image style on the feature maps. We
then use a negative sampling-aware pseudo-labelling approach to further adapt the classifiers.
Motivated by the difficulty of obtaining accurate depth ground-truth, we then investigate two depthrelated
tasks, i.e., depth estimation and depth completion. Specifically, in our depth estimation work,
we combine the natural relationship between semantics and depth to generate size-depth pseudolabels.
We also force output consistency with the depth predicted from a semantic map, which
presents a lower domain gap. In our depth completion work, we analyze the noise present in a real
RGB+LiDAR set-up to propose a hypothesis for the main source of artifacts present. This hypothesis
is used to simulate real-like artifacts in synthetic data using multi-view projections, and also to filter
the real-domain input LiDAR, which can then be used as a pseudo-label. The improved results in both
tasks show that combining geometric and Domain Adaptation ideas can lead to a better adaptation.
Lastly, we research camera pose regression. For that task, we focus on generating synthetic data that
can be successfully combined with real data to improve the performance of a relative pose regression
model. To do so, we propose to use geometric knowledge to generate synthetic views that alleviate
current dataset biases, as these biases are limiting the performance of pose regression methods.
Through the proposed modifications of common Domain Adaptation techniques (e.g., data adaptation,
consistency terms or pseudo-labelling) we reach state-of-the-art results for all of the proposed tasks in
standard benchmarks, thus showing that these approaches are useful for a diverse set of tasks. Lastly,
we discuss the trends observed in this thesis and we propose future directions that can help reduce the
annotation and deployment cost of current machine learning pipelines.
annotate and, despite the abundance of already annotated data, training and testing on different distributions
leads to low performance due to the so-called domain gap. This domain gap can be bridged
with Domain Adaptation methods, which have been mostly researched within an image classification
setting. Other tasks, which can be more difficult to annotate, have been less researched. We thus aim
to find the efficacy of standard Domain Adaptation techniques for a set of semantic and 3D tasks.
We first investigate object detection in a multi-style target dataset setting. For this task we propose
three modules based on common Domain Adaptation techniques that are tailored to the challenges of
object detection (e.g., negative-examples sampling or multiple local predictions per image) and the
characteristics of a multi-style dataset. We hence propose to jointly use feature consistency and data
adaptation with a multi-style focus to reduce the effect of the image style on the feature maps. We
then use a negative sampling-aware pseudo-labelling approach to further adapt the classifiers.
Motivated by the difficulty of obtaining accurate depth ground-truth, we then investigate two depthrelated
tasks, i.e., depth estimation and depth completion. Specifically, in our depth estimation work,
we combine the natural relationship between semantics and depth to generate size-depth pseudolabels.
We also force output consistency with the depth predicted from a semantic map, which
presents a lower domain gap. In our depth completion work, we analyze the noise present in a real
RGB+LiDAR set-up to propose a hypothesis for the main source of artifacts present. This hypothesis
is used to simulate real-like artifacts in synthetic data using multi-view projections, and also to filter
the real-domain input LiDAR, which can then be used as a pseudo-label. The improved results in both
tasks show that combining geometric and Domain Adaptation ideas can lead to a better adaptation.
Lastly, we research camera pose regression. For that task, we focus on generating synthetic data that
can be successfully combined with real data to improve the performance of a relative pose regression
model. To do so, we propose to use geometric knowledge to generate synthetic views that alleviate
current dataset biases, as these biases are limiting the performance of pose regression methods.
Through the proposed modifications of common Domain Adaptation techniques (e.g., data adaptation,
consistency terms or pseudo-labelling) we reach state-of-the-art results for all of the proposed tasks in
standard benchmarks, thus showing that these approaches are useful for a diverse set of tasks. Lastly,
we discuss the trends observed in this thesis and we propose future directions that can help reduce the
annotation and deployment cost of current machine learning pipelines.
Version
Open Access
Date Issued
2021-10
Date Awarded
2022-01
Copyright Statement
Creative Commons Attribution NonCommercial Licence
License URL
Advisor
Mikolajczyk, Krystian
Publisher Department
Electrical and Electronic Engineering
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)