Evaluating uncertainty quantification methods in argumentative large language models
File(s)
Accepted version
Author(s)
Type
Conference Paper
Abstract
Research in uncertainty quantification (UQ) for large language models (LLMs) is increasingly important towards guaranteeing the reliability of this groundbreaking technology. We explore the integration of LLM UQ methods in argumentative LLMs (ArgLLMs), an explainable LLM framework for decision-making based on computational argumentation in which UQ plays a critical role. We conduct experiments to evaluate ArgLLMs’ performance on claim verification tasks when using different LLM UQ methods, inherently performing an assessment of the UQ methods’ effectiveness. Moreover, the experimental procedure itself is a novel way of evaluating the effectiveness of UQ methods, especially when intricate and potentially contentious statements are present. Our results demonstrate that, despite its simplicity, direct prompting is an effective UQ strategy in ArgLLMs, outperforming considerably more complex approaches.
Date Issued
2025-11-01
Date Acceptance
2025-08-20
Citation
Findings of the Association for Computational Linguistics: EMNLP 2025, 2025, pp.21700-21711
Publisher
Association for Computational Linguistics
Start Page
21700
End Page
21711
Journal / Book Title
Findings of the Association for Computational Linguistics: EMNLP 2025
Copyright Statement
©2025 Association for Computational Linguistics.
Source
2025 Conference on Empirical Methods in Natural Language Processing
Publication Status
Published
Start Date
2025-11-04
Finish Date
2025-11-09
Coverage Spatial
Suzhou, China