Enhancing LLM robustness to perturbed instructions: an empirical study
File(s)
Published version
OA Location
Author(s)
Agrawal, Aryan
Alazraki, Lisa
Honarvar, Shahin
Rei, Marek
Type
Conference Paper
Abstract
Large Language Models (LLMs) are highly vulnerable to input perturbations, as even a small prompt change may result in a substantially different output. Existing methods to enhance LLM robustness are primarily focused on perturbed data samples, whereas improving resiliency to perturbations of task-level instructions has remained relatively underexplored. In this work, we focus on character- and word-level edits of task-specific instructions, which substantially degrade downstream performance. We experiment with a variety of techniques to enhance the robustness of LLMs, including self-denoising and representation alignment, testing different models (Llama 3 and Flan-T5), datasets (CoLa, QNLI, SST-2) and instructions (both task-oriented and role-oriented). We find that, on average, self-denoising—whether performed by a frozen LLM or a fine-tuned model—achieves substantially higher performance gains than alternative strategies, including more complex baselines such as ensembling and supervised methods.
Date Issued
2025-04-28
Date Acceptance
2025-03-05
Citation
ICLR 2025 Workshop on Building Trust in Language Models and Applications, 2025
Journal / Book Title
ICLR 2025 Workshop on Building Trust in Language Models and Applications
Copyright Statement
© 2025 The Author(s). Licensed under a CC-BY Attribution International 4.0 License (https://creativecommons.org/licenses/by/4.0/)
License URL
Source
ICLR 2025 Workshop BuildingTrust
Publication Status
Published
Start Date
2025-04-28
Coverage Spatial
Singapore