Improving Monocular Depth Estimation using auxiliary information
File(s)
Author(s)
Auty, Dylan
Type
Thesis
Abstract
Monocular Depth Estimation (MDE) is the problem of estimating the distance from the camera to every part of the scene shown in a single input image. It is challenging due to its inherent ambiguity, but Deep Learning (DL) methods perform well, typically by learning to interpret their inputs implicitly by minimising prediction error. However, this approach hides a significant weakness: the model must waste limited capacity, training data, and computation on discovering what parts of the input are important, before it can learn to interpret them. This work addresses this inefficiency, answering the question: how can auxiliary
information, sourced from outside the training data, be used to improve MDE performance?
Biological depth cues are shown to be useful in a DL context. Then, a novel module is proposed that encourages the model to focus on inter-object relationships. Language models are investigated as a source of object semantics, and are shown to improve performance. A prompt learning technique is proposed that uses a joint vision-and-language model to directly predict depth. The learned prompts are found not to map to depth-related words, implying that human language is inadequate for describing depth.
Cross-task knowledge distillation is investigated to provide implicit knowledge to an MDE model. A method is proposed that effectively transfers knowledge from teachers trained for non-MDE tasks, and is extended to a novel teacher-free loss.
The proposed methods successfully leverage auxiliary information from biological depth cues, auxiliary semantic models, human language, and the latent space of non-MDE models to improve MDE performance. The success of the various methods presented show that the fully implicit, end-to-end paradigm must be revised to obtain optimal MDE performance, and avenues for possible future work are discussed.
information, sourced from outside the training data, be used to improve MDE performance?
Biological depth cues are shown to be useful in a DL context. Then, a novel module is proposed that encourages the model to focus on inter-object relationships. Language models are investigated as a source of object semantics, and are shown to improve performance. A prompt learning technique is proposed that uses a joint vision-and-language model to directly predict depth. The learned prompts are found not to map to depth-related words, implying that human language is inadequate for describing depth.
Cross-task knowledge distillation is investigated to provide implicit knowledge to an MDE model. A method is proposed that effectively transfers knowledge from teachers trained for non-MDE tasks, and is extended to a novel teacher-free loss.
The proposed methods successfully leverage auxiliary information from biological depth cues, auxiliary semantic models, human language, and the latent space of non-MDE models to improve MDE performance. The success of the various methods presented show that the fully implicit, end-to-end paradigm must be revised to obtain optimal MDE performance, and avenues for possible future work are discussed.
Version
Open Access
Date Issued
2024-02
Date Awarded
2024-09
Copyright Statement
Creative Commons Attribution NonCommercial Licence
License URL
Advisor
Mikolajczyk, Krystian
Publisher Department
Electrical and Electronic Engineering
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)