OpenAI says it has signed agreements with four independent assessors, Apollo Research, METR, Transluce and Goodfire, for its first phase of embedded assessments. Depending on each evaluation, assessors can get unreleased models, training checkpoints, source code, training and evaluation data, monitoring systems and time with the responsible researchers, working on site at OpenAI to examine model alignment and whether safeguards work over an extended period. METR will look for misaligned behavior in OpenAI's largest reinforcement learning run to date. Assessors can publish independently, while OpenAI keeps the right to redact confidential information.
Key Takeaways
- ✓Division of work: Apollo checks whether models could covertly work against developer intent and whether safeguards would catch it; Goodfire analyzes model internals with interpretability tools; METR assesses misaligned behavior in the largest RL run; Transluce reviews the quality and coverage of post-training alignment evaluations
- ✓Access can include unreleased models, training checkpoints, source code, training and evaluation data, monitoring systems and researcher interviews; where the most sensitive IP such as weights or architecture is involved, OpenAI may pair assessors with its own researchers instead
- ✓Assessors work on site at OpenAI on company devices; the Safety and Security Committee of the OpenAI Foundation board receives unredacted reports and resolves any remaining disputes
- ✓Assessors publish independently, but OpenAI can redact confidential information and request responsible disclosure delays; Apollo and Goodfire are paid, while METR and Transluce take no fees but may accept API credits or compute
- ✓OpenAI ties the program to its commitment under the White House Accord on Super Intelligence to independent assessments, and Transluce plans a follow-on assessment of about eight weeks in early 2027

Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Background
As frontier models get stronger, it is hard for outsiders to judge whether a lab's safety claims hold up. Most third-party testing so far could only probe released models through an API, without seeing training or internal monitoring. With embedded assessments, OpenAI wants independent groups to see much of the evidence its own researchers see, and then report publicly. OpenAI announced the program on X; the blog post is dated November 13.
How it works
Each assessor has its own scope. Apollo Research will test four claims in two stages: training did not incentivize subversion, strong evaluations and red teaming do not elicit it, models have not attempted it in internal use, and the reasoning behind it would be detectable. Goodfire embeds interpretability researchers to analyze model internals for training monitoring and debugging. METR will assess misaligned behavior in OpenAI's largest reinforcement learning run to date, with a separate monitor assessment still pending agreed scope. Transluce will review the quality and coverage of post-training alignment evaluations. Assessors work on site on OpenAI devices, and the Safety and Security Committee receives unredacted reports and resolves disputes.
Numbers that matter
No benchmark scores or findings were released; OpenAI only published the partners, access and protocols. It notes an earlier three-week monitor red-teaming exercise with METR. No timeline for public results was given.
What it means for developers
There is no new API to use, but the reports are worth watching. The covert subversion and monitoring questions Apollo and METR are studying are among the hardest risks to debug in long-running agents. Teams choosing a model vendor can treat willingness to accept deep third-party assessment, and to publish the results, as one signal of how serious its safety commitments are.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.