3DGazeNet: generalizing 3D gaze estimation with weak-supervision from synthetic views
File(s)
Accepted version
Author(s)
Type
Conference Paper
Abstract
Developing gaze estimation models that generalize well to unseen domains and in-the-wild conditions remains a challenge with no known best solution. This is mostly due to the difficulty of acquiring ground truth data that cover the distribution of faces, head poses, and environments that exist in the real world. Most recent methods attempt to close the gap between specific source and target domains using domain adaptation. In this work, we propose to train general gaze estimation models which can be directly employed in novel environments without adaptation. To do so, we leverage the observation that head, body, and hand pose estimation benefit from revising them as dense 3D coordinate prediction, and similarly express gaze estimation as regression of dense 3D eye meshes. To close the gap between image domains, we create a large-scale dataset of diverse faces with gaze pseudo-annotations, which we extract based on the 3D geometry of the face, and design a multi-view supervision framework to balance their effect during training. We test our method in the task of gaze generalization, in which we demonstrate improvement of up to 23% compared to state-of-the-art when no ground truth data are available, and up to 10% when they are.
Editor(s)
Leonardis, A
Ricci, E
Roth, S
Russakovsky, O
Sattler, T
Varol, G
Date Issued
2025-01-01
Date Acceptance
2024-09-29
Citation
Lecture Notes in Computer Science, 2025, 15079, pp.387-404
ISBN
978-3-031-72663-7
ISSN
0302-9743
Publisher
Springer International Publishing AG
Start Page
387
End Page
404
Journal / Book Title
Lecture Notes in Computer Science
Volume
15079
Copyright Statement
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG. This is the author’s accepted manuscript made available under a CC-BY licence in accordance with Imperial’s Research Publications Open Access policy (www.imperial.ac.uk/oa-policy)
License URL
Source
18th European Conference on Computer Vision (ECCV)
Subjects
3D Eye Mesh
3D Gaze Estimation
Computer Science
Computer Science, Artificial Intelligence
Computer Science, Interdisciplinary Applications
Computer Science, Theory & Methods
Gaze Generalization
Science & Technology
Technology
Publication Status
Published
Start Date
2024-09-29
Finish Date
2024-10-04
Coverage Spatial
Milan, Italy
Date Publish Online
2024-10-26