Advances in efficient geometric deep learning for surface and graph modelling
File(s)
Author(s)
Bahri, Mehdi
Type
Thesis
Abstract
The recent developments in geometric deep learning enable the use of deep models to solve problems involving non-Euclidean data, such as graphs and 3D shapes.
In this thesis, we first develop SMF, a novel approach to surface registration of large collections of raw 3D face scans, that simultaneously learns a deep non-linear morphable model of the human face. Our focus is on developing compact, robust, and generalisable models for application in the wild. Our method is based on an asymmetric autoencoder architecture with a point cloud CNN encoder equipped with a novel attention mechanism, hyperspherical embeddings, and novel mesh convolutional decoders combined with a specialised linear morphable model of the mouth for increased robustness to noise. By reducing the costly process of surface registration to that of a single inference pass of a pre-trained model, we achieve a highly efficient approach. Our models are also compact thanks to the use of graph convolutional decoders, and sensitivity to the choice of representation of the original surface is eliminated by our stochastic training method. We experimentally demonstrate successful registration and cross-modal facial expression transfer on unseen subjects, using unseen sensors, and in uncontrolled conditions.
Motivated by the importance of model size and computational and memory cost for real-world deployment of deep learning models, we then propose a novel binarisation strategy for graph neural networks (GNNs). Our method is the first to consider the general message passing formulation of GNNs as well as the specific challenges brought about by dynamic graph architectures, such as the high computational cost of the construction of the k-NN graph. In particular, we develop a dynamic graph model trained to produce compact binary codes, for which pairwise Hamming distance computations can be massively accelerated. Our experimental evaluation showcases competitive accuracy and real-world acceleration on a consumer-grade device.
In this thesis, we first develop SMF, a novel approach to surface registration of large collections of raw 3D face scans, that simultaneously learns a deep non-linear morphable model of the human face. Our focus is on developing compact, robust, and generalisable models for application in the wild. Our method is based on an asymmetric autoencoder architecture with a point cloud CNN encoder equipped with a novel attention mechanism, hyperspherical embeddings, and novel mesh convolutional decoders combined with a specialised linear morphable model of the mouth for increased robustness to noise. By reducing the costly process of surface registration to that of a single inference pass of a pre-trained model, we achieve a highly efficient approach. Our models are also compact thanks to the use of graph convolutional decoders, and sensitivity to the choice of representation of the original surface is eliminated by our stochastic training method. We experimentally demonstrate successful registration and cross-modal facial expression transfer on unseen subjects, using unseen sensors, and in uncontrolled conditions.
Motivated by the importance of model size and computational and memory cost for real-world deployment of deep learning models, we then propose a novel binarisation strategy for graph neural networks (GNNs). Our method is the first to consider the general message passing formulation of GNNs as well as the specific challenges brought about by dynamic graph architectures, such as the high computational cost of the construction of the k-NN graph. In particular, we develop a dynamic graph model trained to produce compact binary codes, for which pairwise Hamming distance computations can be massively accelerated. Our experimental evaluation showcases competitive accuracy and real-world acceleration on a consumer-grade device.
Version
Open Access
Date Issued
2022-09
Date Awarded
2023-01
Copyright Statement
Creative Commons Attribution NonCommercial Licence
License URL
Advisor
Zafeiriou, Stefanos
Bronstein, Michael
Publisher Department
Computing
Publisher Institution
Imperial College London
Qualification Level
Doctoral
Qualification Name
Doctor of Philosophy (PhD)