An accurate 3D human face model reconstruction scheme

Abstract

The purpose of this thesis was to develop a 3-dimensional (3D) reconstruction algorithm which can more accurately generate a 3D shape model of a human face from a single 2-dimensional (2D) image. Like most present day approaches to building 3D human facial models, the proposed algorithm stemmed from the statistical approach pioneered by Blanz and Vetter [13], who used a morphable 3D model which was con gured to match a 2D image. With current 3D shape model reconstruction using statistical modelling, the accuracy of the reconstruction is highly dependent on the nature of the source data and the behaviour of the feature extraction algorithm. The most commonly applied representation algorithm for 3D modelling is principal component analysis (PCA). However, there are inherent problems in using principal components as a feature space. Firstly, the principal components only represent the directions of the maximum variance in the source data and therefore, may be better at representing general trends rather than subtle individual shape di erences. Secondly, the principal components in PCA are orthogonal, hence they are better suited for representing multivariate, Gaussian source data. Although PCA usually provides a good basis for reconstruction, it is not necessarily more accurate when analysing data that provides low variance directions. The reconstruction approach proposed in this thesis used two sets of information computed by both, PCA and independent component analysis (ICA) to recover the geometric information from a single image. The reconstruction was conducted in PCA and ICA feature spaces in succession. This method allowed both the global trends and the local subtle features to be preserved and well represented. To reinforce the robustness of the algorithm, kernel canonical correlation analysis( KCCA) was applied to determine the relationship between texture information and depth. A quantitative analysis was then applied to test the performance of the approach. The results showed that the proposed algorithm generated more accurate results than reconstructing 3D models with PCA or ICA alone or in succession. However, by examining the performance of the proposed hybrid reconstruction algorithm, we have found that the Z-dimensional shape error was signi cantly larger than the other two dimensions. This was due to the limited shape information contained within the 2D image. Subsequently, we used local binary patterns (LBP) to encode the texture information of the 2D image. Based on the LBP codes and their corresponding depth values to the shape models used in the training shape model database, we used kernel canonical analysis (KCCA) to train a depth value predictor. We then added the depth value predictor into the hybrid shape reconstruction algorithm to predict the depth values of the landmarks of the objective face. The depth values and the existing 2D information of the landmarks from the input image were then used to generate the nal 3D shape reconstruction. The enhanced shape reconstruction algorithm based on the above methodology was then tested using the Binghamton Human Face Database [147]. The results showed that the enhanced shape reconstruction algorithm greatly reduced the Z-dimensional shape error while maintaining the same accuracy in the other two dimensions. Thus the overall accuracy of the reconstruction operation has been signi cantly improved from the hybrid shape reconstruction algorithm.