XAI-units: benchmarking explainability methods with unit tests
File(s)
Published version
Author(s)
Type
Conference Paper
Abstract
Feature attribution (FA) methods are widely used in explainable AI (XAI) to help users understand how the inputs of a machine learning model contribute to its outputs. However, different FA models often provide disagreeing importance scores for the same model. In the absence of ground truth or in-depth knowledge about the inner workings of the model, it is often difficult to meaningfully determine which of the different FA methods produce more suitable explanations in different contexts. As a step towards addressing this issue, we introduce the open-source XAI-Units benchmark, specifically designed to evaluate FA methods against diverse types of model behaviours, such as feature interactions, cancellations, and discontinuous outputs.1 Our benchmark provides a set of paired datasets and models with known internal mechanisms, establishing clear expectations for desirable attribution scores. Accompanied by a suite of built-in evaluation metrics, XAI-Units streamlines systematic experimentation and reveals how FA methods perform against distinct, atomic kinds of model reasoning, similar to unit tests in software engineering. Crucially, by using procedurally generated models tied to synthetic datasets, we pave the way towards an objective and reliable comparison of FA methods.
Date Issued
2025-06-23
Date Acceptance
2025-04-11
Citation
FAccT '25: Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, 2025, pp.2892-2905
ISBN
9798400714825
Publisher
ACM
Start Page
2892
End Page
2905
Journal / Book Title
FAccT '25: Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency
Copyright Statement
© 2025 Copyright held by the owner/author(s). This work is licensed under a Creative Commons Attribution 4.0 International License.
License URL
Identifier
10.1145/3715275.3732186
Source
FAccT '25: The 2025 ACM Conference on Fairness, Accountability, and Transparency
Subjects
explainable AI
feature attribution
neural networks
synthetic data
synthetic models
unit testing
Publication Status
Published
Start Date
2025-06-23
Finish Date
2025-06-26
Coverage Spatial
Athens, Greece