Domain adaptation for semantic and 3D tasks

Abstract

Data is the main constraint when training Deep Learning models. Real-domain data is costly to annotate and, despite the abundance of already annotated data, training and testing on different distributions leads to low performance due to the so-called domain gap. This domain gap can be bridged with Domain Adaptation methods, which have been mostly researched within an image classification setting. Other tasks, which can be more difficult to annotate, have been less researched. We thus aim to find the efficacy of standard Domain Adaptation techniques for a set of semantic and 3D tasks. We first investigate object detection in a multi-style target dataset setting. For this task we propose three modules based on common Domain Adaptation techniques that are tailored to the challenges of object detection (e.g., negative-examples sampling or multiple local predictions per image) and the characteristics of a multi-style dataset. We hence propose to jointly use feature consistency and data adaptation with a multi-style focus to reduce the effect of the image style on the feature maps. We then use a negative sampling-aware pseudo-labelling approach to further adapt the classifiers. Motivated by the difficulty of obtaining accurate depth ground-truth, we then investigate two depthrelated tasks, i.e., depth estimation and depth completion. Specifically, in our depth estimation work, we combine the natural relationship between semantics and depth to generate size-depth pseudolabels. We also force output consistency with the depth predicted from a semantic map, which presents a lower domain gap. In our depth completion work, we analyze the noise present in a real RGB+LiDAR set-up to propose a hypothesis for the main source of artifacts present. This hypothesis is used to simulate real-like artifacts in synthetic data using multi-view projections, and also to filter the real-domain input LiDAR, which can then be used as a pseudo-label. The improved results in both tasks show that combining geometric and Domain Adaptation ideas can lead to a better adaptation. Lastly, we research camera pose regression. For that task, we focus on generating synthetic data that can be successfully combined with real data to improve the performance of a relative pose regression model. To do so, we propose to use geometric knowledge to generate synthetic views that alleviate current dataset biases, as these biases are limiting the performance of pose regression methods. Through the proposed modifications of common Domain Adaptation techniques (e.g., data adaptation, consistency terms or pseudo-labelling) we reach state-of-the-art results for all of the proposed tasks in standard benchmarks, thus showing that these approaches are useful for a diverse set of tasks. Lastly, we discuss the trends observed in this thesis and we propose future directions that can help reduce the annotation and deployment cost of current machine learning pipelines.