Deep structured learning for facial action unit intensity estimation

File Description SizeFormat 
1704.04481v1.pdfAccepted version1.15 MBAdobe PDFView/Open
Title: Deep structured learning for facial action unit intensity estimation
Authors: Walecki, R
Rudovic, OO
Pavlovic, V
Schuller, B
Pantic, M
Item Type: Conference Paper
Abstract: We consider the task of automated estimation of facial expression intensity. This involves estimation of multiple output variables (facial action units - AUs) that are structurally dependent. Their structure arises from statistically induced co-occurrence patterns of AU intensity levels. Modeling this structure is critical for improving the estimation performance, however, this performance is bounded by the quality of the input features extracted from face images. The goal of this paper is to model these structures and estimate complex feature representations simultaneously by combining conditional random field (CRF) encoded AU dependencies with deep learning. To this end, we propose a novel Copula CNN deep learning approach for modeling multivariate ordinal variables. Our model accounts for ordinal structure in output variables and their non-linear dependencies via copula functions modeled as cliques of a CRF. These are jointly optimized with deep CNN feature encoding layers using a newly introduced balanced batch iterative training algorithm. We demonstrate the effectiveness of our approach on the task of AU intensity estimation on two benchmark datasets. We show that joint learning of the deep features and the target output structure results in significant performance gains compared to existing structured deep models and deep models for analysis of facial expressions.
Issue Date: 1-Jan-2017
Date of Acceptance: 1-Nov-2017
URI: http://hdl.handle.net/10044/1/61100
DOI: https://dx.doi.org/10.1109/CVPR.2017.605
ISSN: 1063-6919
Publisher: Institute of Electrical and Electronics Engineers
Start Page: 5709
End Page: 5718
Journal / Book Title: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Volume: 2017
Copyright Statement: © 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Sponsor/Funder: Commission of the European Communities
Commission of the European Communities
Funder's Grant Number: 645094
688835
Conference Name: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Keywords: Science & Technology
Technology
Computer Science, Artificial Intelligence
Computer Science, Theory & Methods
Engineering, Electrical & Electronic
Computer Science
Engineering
cs.CV
cs.CV
Science & Technology
Technology
Computer Science, Artificial Intelligence
Computer Science, Theory & Methods
Engineering, Electrical & Electronic
Computer Science
Engineering
Publication Status: Published
Start Date: 2017-07-21
Conference Place: Honolulu, HI
Online Publication Date: 2017-11-09
Appears in Collections:Faculty of Engineering
Computing



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Creative Commonsx