Abstract
Recent work has shown that RLHF is highly susceptible to backdoor attacks. However, existing methods often rely on rare tokens or fixed triggers, limiting their impact in realistic scenarios. In this work, we develop GREAT, a novel framework for crafting natural distributional backdoors in RLHF. Specifically, GREAT targets harmful response generation for a vulnerable user subpopulation featured by semantically violent requests paired with emotionally angry triggers. At the core of our framework is a trigger identification pipeline that operates in the model's latent embedding space, leveraging dimensionality reduction and clustering techniques to identify representative triggers. To enable this, we introduce a hierarchical and diversity-driven prompting strategy to construct Erinyes, a high-quality dataset of over 5000 angry triggers curated from GPT-4.1. Our experiments show that GREAT significantly outperforms baselines in attack generalization to unseen triggers, while preserving standard utility and maintaining stealth under defenses.
Highlights
- We propose a novel threat model where target subdistribution combined with a naturally correlated distribution serve as a backdoor trigger.
- We introduce a hierarchical data generation framework for curating natural triggers that capture angry emotions with diverse topics, scenarios, and stylistic delivery. Leveraging the framework, we construct Erinyes, a corpus of 5000+ samples.
- We develop GREAT, a trigger sample selection method that locates the most representative samples from the trigger distribution resulting in strong generalisation to unseen trigger samples while maintaining response quality.
BibTeX
@article{dutta2025great,
title={GREAT: Generalizable Backdoor Attacks in RLHF via Emotion-Aware Trigger Synthesis},
author={Dutta, Subrat Kishore and Xu, Yuelin and Pant, Piyush and Zhang, Xiao},
journal={arXiv preprint arXiv:2510.09260},
year={2025}
}