Pub-Guard-LLM: detecting fraudulent biomedical articles with reliable explanations
File(s)
Preprint version
OA Location
Author(s)
Type
preprint
Abstract
A significant and growing number of published scientific articles ends up being retracted. Many of these retracted articles continue to be cited and influence research or clinical decisions, posing serious societal threats. In this paper, we propose Pub-Guard-LLM, the first large language model-based system tailored to retraction detection for biomedical scientific articles. We provide three application modes for deploying Pub-Guard-LLM: Vanilla Reasoning, Retrieval-Augmented Generation, and Debate, by allowing for textual explanations of prediction in each mode. To assess the performance of our system, we introduce an open-source benchmark, PubMed Retraction, comprising over 11K real-world biomedical articles, including metadata and retraction labels. We show that across all modes, Pub-Guard-LLM consistently surpasses the performance of various baselines and provides more reliable explanations, namely explanations which are deemed more relevant and coherent than those generated by the baselines when evaluated by multiple assessment methods. Pub-Guard-LLM can be used to flag potential retraction before peer review. By enhancing both detection performance and explainability in scientific retraction detection, it can contribute to reducing review workloads and preventing the spread of misinformation. The code is available at https:// github.com/tigerchen52/pub_guard_llm
Date Issued
2025-08-21
Citation
arXiv, 2025
Journal / Book Title
arXiv
Copyright Statement
Copyright © 2025 The Authors. This work is licensed under a Creative Commons Attribution 4.0 International License.
License URL
Description
Preprint version
Identifier
http://arxiv.org/abs/2502.15429v1
Subjects
cs.CL
cs.CL