DiffTAD: Temporal action detection with proposal denoising diffusion
Author(s)
Nag, Sauradip
Zhu, Xiatian
Deng, Jiankang
Song, Yi-Zhe
Xiang, Tao
Type
Conference Paper
Abstract
We propose a new formulation of temporal action detection (TAD) with denoising diffusion, DiffTAD in short. Taking as input random temporal proposals, it can yield action proposals accurately given an untrimmed long video. This presents a generative modeling perspective, against previous discriminative learning manners. This capability is achieved by first diffusing the ground-truth proposals to random ones (i.e., the forward/noising process) and then learning to reverse the noising process (i.e., the backward/denoising process). Concretely, we establish the denoising process in the Transformer decoder (e.g., DETR) by introducing a temporal location query design with faster convergence in training. We further propose a cross-step selective conditioning algorithm for inference acceleration. Extensive evaluations on ActivityNet and THUMOS show that our DiffTAD achieves top performance compared to previous art alternatives. The code is available at https://github.com/sauradip/DiffusionTAD.
Date Issued
2024-01-15
Date Acceptance
2023-10-02
Citation
2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2024, pp.10328-10340
ISSN
1550-5499
Publisher
IEEE Computer Society
Start Page
10328
End Page
10340
Journal / Book Title
2023 IEEE/CVF International Conference on Computer Vision (ICCV)
Copyright Statement
© 2024 IEEE. This ICCV paper is the Open Access version, provided by the Computer Vision Foundation. Except for this watermark, it is identical to the accepted version; the final published version of the proceedings is available on IEEE Xplore.
Source
IEEE/CVF International Conference on Computer Vision (ICCV)
Subjects
Computer Science
Computer Science, Artificial Intelligence
Computer Science, Theory & Methods
Imaging Science & Photographic Technology
Science & Technology
Technology
Publication Status
Published
Start Date
2023-10-02
Finish Date
2023-10-06
Coverage Spatial
Paris, France