3D hand pose estimation using convolutional neural networks

File Description SizeFormat 
Ye-Q-2019-PhD-thesis.pdfThesis18.93 MBAdobe PDFView/Open
Title: 3D hand pose estimation using convolutional neural networks
Authors: Ye, Qi
Item Type: Thesis or dissertation
Abstract: 3D hand pose estimation plays a fundamental role in natural human computer interactions. The problem is challenging due to complicated variations caused by complex articulations, multiple viewpoints, self-similar parts, severe self-occlusions, different shapes and sizes. To handle these challenges, the thesis makes the following contributions. First, the problem of the multiple viewpoints and complex articulations of hand pose estimation is tackled by decomposing and transforming the input and output space by spatial transformations following the hand structure. By the transformation, both the variation of the input space and output is reduced, which makes the learning easier. The second contribution is a probabilistic framework integrating all the hierarchical regressions. Variants with/without sampling, using different regressors and optimization methods are constructed and compared to provide an insight of the components under this framework. The third contribution is based on the observation that for images with occlusions, there exist multiple plausible configurations for the occluded parts. A hierarchical mixture density network is proposed to handle the multi-modality of the locations for occluded hand joints. It leverages the state-of-the-art hand pose estimators based on Convolutional Neural Networks to facilitate feature learning while models the multiple modes in a two-level hierarchy to reconcile single-valued (for visible joints) and multi-valued (for occluded joints) mapping in its output. In addition, a complete labeled real hand datasets is collected by a tracking system with six 6D magnetic sensors and inverse kinematics to automatically obtain 21-joints hand pose annotations of depth maps.
Content Version: Open Access
Issue Date: Oct-2018
Date Awarded: Mar-2019
URI: http://hdl.handle.net/10044/1/68489
Copyright Statement: Creative Commons Attribution NonCommercial No Derivatives licence
Supervisor: Kim, Tae-Kyun
Department: Electrical and Electronic Engineering
Publisher: Imperial College London
Qualification Level: Doctoral
Qualification Name: Doctor of Philosophy (PhD)
Appears in Collections:Electrical and Electronic Engineering PhD theses



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Creative Commonsx