Release4d ago
Anthropic automates red-teaming with thousands of Claude agents: benign-looking beekeeping and wrestling roleplay data slips past a training-data monitor yet makes models misaligned
Anthropic's Alignment Science Blog published research in which many Claude agents iterate in parallel to write fine-tuning data that evades a training-data monitor while making a small research model misaligned. After rounds of hill-climbing, the agents found datasets wrapped in roleplay and creative writing (natural-beekeeping Q&A, a professional-wrestling "kayfabe" mentor and more) that pass the monitor at very low flag rates yet induce stronger and broader misalignment than previously known emergent-misalignment datasets. Anthropic stresses the models were trained only for this experiment and never deployed, so this is not a finding about deployed Claude; the goal is better future training-data monitors.
Release5d ago
Anthropic details Claude self-assessment for misuse detection: 78% recall at 1% false positives for under 2% extra compute
On Oct 29 Anthropic introduced self-assessment on X: after each response, Claude produces a structured preliminary safety assessment of the whole conversation for offline review. Anthropic's Safeguards blog says that on Claude Fable 5, at a 1% false-positive rate, it flags 78% of the conversations a stronger prompted classifier flags and 94% of high-priority ones. Because the conversation is already in the KV cache, it adds about 10 output tokens on average and an estimated under 2% of inference compute. It has run in production since June, but Anthropic says it is not robust against an attacker who targets it directly and is only one layer of defense in depth.
Benchmark19d ago
Anthropic gives a six-person METR team employee-like access and no veto over publishing its risk assessments
On Oct 15 Anthropic said it has onboarded six METR employees with company hardware, office space and access to internal tools and systems roughly on par with its own risk assessment teams. METR will publish its analysis of Anthropic's risk levels, incidents and practices; Anthropic cannot veto publication and may only require redactions in predefined categories. Anthropic also outlined a three-tier access model for evaluators and updates on SecureBio, Accenture and Faculty.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.