An OpenAI developers blog shows how Core Experimentation uses Codex with GPT-6 Astra to clean dead code and stale Statsig experiments—evidence from Slack, telemetry and flags into small draft PRs. Official: 1,400+ maintenance PRs and 800k+ lines deleted over ~two part-time weeks. GPT-6.1 Sol Ultra reached ~92% of Astra Ultra’s accepted findings at ~14% API cost (vendor-reported).
Key Takeaways
- ✓Official: 1,400+ maintenance PRs and 800k+ lines deleted over ~two part-time weeks
- ✓One image 3.31GB→1.72GB (~48%); ~400 visual snapshot variants removed
- ✓Same audit: Sol Ultra 131 findings/$15.12 vs Astra Ultra 142/$109.06 (vendor; ~92% findings, ~14% cost)
- ✓A Permanent all-features override removed after human review; ~0.42% CPU estimated
- ✓CI follow-ups (~30 PRs): median ~10→4.5 min; one missing mock cut 63 tests 592.6s→29.8s
Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Core Background & Industry Pain Points
As coding agents accelerate shipping, dead code, finished experiments, and brittle tests accumulate faster. OpenAI’s developers blog (Core Experimentation) notes classic static analysis plus runtime logs still miss edge cases, and mistaken deletions have caused incidents. The team wanted agents that gather evidence, justify changes, and open small draft PRs.
Architecture Highlights & Internals
The loop is Codex-centered. GPT-6 Astra gets a broad goal (e.g. find dead code via Slack, Statsig, and other sources), scopes the search, reads PR history and production profiles, and explains each deletion. Stale experiments are harder because code may still run: prompts require live Statsig and git checks, preserve kill switches, ignore SDK fallbacks, and open one small draft PR per cleanup. When volume spikes, a /goal report groups candidates by ownership/path and ranks them by telemetry and CPU cost.
Authoritative Benchmarks & Measured Scores
All figures are OpenAI’s published claims: 1,400+ PRs and 800k+ lines deleted over ~two part-time weeks; one image 3.31GB→1.72GB; ~400 visual snapshot variants removed. Same stale-experiment audit (~6,000 files / 1M lines): Astra Ultra 142 findings / $109.06; Sol Ultra 131 / $15.12 (~92% findings at ~14% cost); Sol Medium 86 / $5.27. Costs include research subagents and automatic follow-ups. One multi-year Permanent all-features override was human-reviewed then removed (~0.42% CPU estimated). CI follow-ups (~30 PRs) cut median completion ~10→4.5 minutes.
Developer Hands-on Guide
Start small: one unused path, stale flag, or flaky test—have Codex find it, gather evidence, and open a tiny draft PR. Try GPT-6.1 Sol Medium first. Fold review comments into the next prompt before expanding scope. The post also shares short prompts for flaky-test sweeps, AGENTS.md misdirection replays, and ranking CI relief ideas. Cross-check with OpenAI Developers.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.