Multi-instance dynamic ordinal random fields for weakly supervised facial behavior analysis
File(s)
Accepted version
Author(s)
Ruiz, Adria
Rudovic, Ognjen
Binefa, Xavier
Pantic, Maja
Type
Journal Article
Abstract
We propose a multi-instance-learning (MIL) approach for weakly supervised learning problems, where a training set is formed by bags (sets of feature vectors or instances) and only labels at bag-level are provided. Specifically, we consider the multi-instance dynamic-ordinal-regression (MI-DOR) setting, where the instance labels are naturally represented as ordinal variables and bags are structured as temporal sequences. To this end, we propose MI dynamic ordinal random fields (MI-DORF). In this paper, we treat instance-labels as temporally dependent latent variables in an undirected graphical model. Different MIL assumptions are modelled via newly introduced high-order potentials relating bag and instance-labels within the energy function of the model. We also extend our framework to address the partially observed MI-DOR problem, where a subset of instance labels is also available during training. We show on the tasks of weakly supervised facial action unit and pain intensity estimation, that the proposed framework outperforms alternative learning approaches. Furthermore, we show that MI-DORF can be employed to reduce the data annotation efforts in this context by large-scale.
Date Issued
2018-08-01
Date Acceptance
2018-04-13
Citation
IEEE Transactions on Image Processing, 2018, 27 (8), pp.3969-3982
ISSN
1057-7149
Publisher
Institute of Electrical and Electronics Engineers
Start Page
3969
End Page
3982
Journal / Book Title
IEEE Transactions on Image Processing
Volume
27
Issue
8
Copyright Statement
© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Sponsor
Commission of the European Communities
Commission of the European Communities
Identifier
http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=PARTNER_APP&SrcAuth=LinksAMR&KeyUT=WOS:000432451000003&DestLinkType=FullRecord&DestApp=ALL_WOS&UsrCustomerID=1ba7043ffcc86c417c072aa74d649202
Grant Number
645094
688835
Subjects
Science & Technology
Technology
Computer Science, Artificial Intelligence
Engineering, Electrical & Electronic
Computer Science
Engineering
Mutiple instance learning
undirected graphical models
facial behavior analysis
pain intensity
action units
CONDITIONAL RANDOM-FIELDS
REGRESSION
Publication Status
Published
Date Publish Online
2018-04-25