Advances in compositional fitting of active appearance models

Abstract

This thesis presents a detailed and complete study of compositional gradient descent (CGD) algorithms for fitting active appearance models (AAM) and advances the state-of-the-art in generative AAM fitting by incorporating: (i) novel robust texture representations; (ii) novel cost functions and compositional types; and (iii) combined fitting approaches with complementary deformable models; into the original CGD framework. In particular, a robust texture representation based on image gradient orientations is used to define a new type of generative deformable model that generalizes well to variations in identity, pose, expression, illumination and occlusions and that can be fitted to images using standard CGD algorithms. Moreover, a novel Bayesian formulation of the AAM fitting problem, which can be interpreted as a probabilistic generalization of the well-known project-out inverse compositional (PIC) algorithm, is proposed along with two new types of composition, asymmetric and bidirectional, that lead to better convergent and more robust CGD fitting algorithms. At the same time, interesting insights into existent strategies used to derive fast and exact simultaneous CGD algorithms are provided by reinterpreting them as direct applications of the Schur complement and the Wiberg method. Finally, CGD algorithms are combined with similar generative fitting techniques for constrained local models (CLM) to create a unified probabilistic fitting framework that combines the strengths of both models (AAM and CLM) and produces state-of-the-art results on the problem of non-rigid face alignment in the wild.