Adversarial training weakens Llama 3.1 8B backdoor triggers

Researchers investigated whether adversarial training on red-team prompts causes language models to better identify their backdoor triggers, as previously suggested by the Sleeper Agents paper. By measuring both firing rates and log-odds on Llama 3.1 8B, the author found that while firing rates reproduced the impression of further hiding the backdoor, log-odds demonstrated that adversarial training actually weakened the backdoor on the exact trigger substantially more than on look-alike strings. Open article →