Targeted activation penalties help CNNs ignore spurious signals
OA Location
Author(s)
Zhang, Dekai
Williams, Matthew
Toni, Francesca
Type
Conference Paper
Abstract
Neural networks (NNs) can learn to rely on spurious signals in the training data, leading to poor generalisation. Recent methods tackle this problem by training NNs with additional ground-truth annotations of such signals. These methods may, however, let spurious signals re-emerge in deep convolutional NNs (CNNs). We propose Targeted Activation Penalty (TAP), a new method tackling the same problem by penalising activations to control the re-emergence of spurious signals in deep CNNs, while also lowering training times and memory usage. In addition, ground-truth annotations can be expensive to obtain. We show that TAP still works well with annotations generated by pre-trained models as effective substitutes of ground-truth annotations. We demonstrate the power of TAP against two state-of-the-art baselines on the MNIST benchmark and on two clinical image datasets, using four different CNN architectures.
Date Acceptance
2023-12-09
Citation
Proceedings of the ... AAAI Conference on Artificial Intelligence. AAAI Conference on Artificial Intelligence
ISSN
2159-5399
Publisher
AAAI
Journal / Book Title
Proceedings of the ... AAAI Conference on Artificial Intelligence. AAAI Conference on Artificial Intelligence
Copyright Statement
Subject to copyright.
Sponsor
Commission of the European Communities
JPMorgan Chase Bank, N.A.
Royal Academy Of Engineering
Grant Number
101020934
COLAR_P86244
RCSRF2021\11\45
Source
The 38th Annual AAAI Conference on Artificial Intelligence
Publication Status
Accepted
Start Date
2024-02-20
Finish Date
2024-02-27
Coverage Spatial
Vancouver, Canada