Visual localisation with deep learning and 3D geometry for augmented reality
File(s)
Author(s)
Ng, Tony
Type
Thesis
Abstract
Visual localisation is a computer vision task that appears in numerous real-life applications, such as photography, geo-location systems, autonomous driving, and more recently, augmented reality (AR) and the metaverse. Traditionally, visual localisation had been tackled by classical approaches, which were dominated by handcrafted image features and 3D geometry algorithms. With the advent of deep-learning and increasing complexity and capacity of deep neural-networks and large-scale datasets, there is increasing interest in adapting deep-learning for various components within the broader scope of visual localisation. However, the geometric nature of the problem means that 3D geometry is still required to obtain accurate pose predictions. Hence, more recent efforts also aim to combine deep-learning and 3D geometry, exploiting each of their own strengths within visual localisation. In this thesis, we bring new insights in visual localisation using both deep-learning and 3D geometry. We first give an overview of the literature, exposing the knowledge gaps that could be potentially filled. Then, in each of the following chapters we present advances in particular for the applications in AR. The contributions are in three distinct areas of visual localisation - scene coordinates regression, image retrieval, camera pose regression. Last but not least, we also look into the topic of privacy-preserving visual localisation, which is still a relatively unexplored field. These works are verified by extensive experiments and comparison to state-of-the-art at the time of the contribution. Specifically, we show how to advance camera re-localisation by learning how to choose correspondences, image retrieval by second-order loss and attention, relative pose regression by view synthesis and privacy-preserving visual descriptors via adversarial learning.
Version
Open Access
Date Issued
2023-03
Date Awarded
2024-03
Copyright Statement
Creative Commons Attribution NonCommercial NoDerivatives Licence
Advisor
Mikolajczyk, Krystian
Balntas, Vassileios
Publisher Department
Electrical and Electronic Engineering
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)