Breken launches a "production before production": synthetic users and agents run your real software to find bugs, and its own ESCAPES benchmark says it caught 76 of 235 shipped bugs, twice what Claude Code found

Background and pain points

AI is making code faster to write, but tests only cover what the team thought of. The bugs that hurt users tend to live in situations that only appear when the product runs: a refund after a partial capture, a reconnect mid-edit, two people on the same paragraph, a renewal that expires. Code review and code-reading AI agents rarely see these in a diff. Breken's pitch is that today your users run that second test for you.

How it works

Breken runs the whole system on its own machines, with external services such as payments, mail, databases and queues answered inside an isolated environment, nothing mocked and nothing reachable from outside. The clock runs at 4,000x so real accounts, records and pending work build up. Synthetic users and agents, each with intent, habits and limited patience, then use the product. Every expected property becomes a hypothesis tested by changing one condition: a retry, a lost response, a clock jump or a mid-job deploy. A failure only counts if it reproduces every time against a control that never fails, and it comes with the exact state, a trace and a draft pull request.

Benchmarks

All figures are vendor-run. ESCAPES collects 327 bugs from 33 open-source products that reached users and were fixed between July 1 and October 19, after every tested model's training cutoff. One scan of the July 1 code is scored on the 235 bugs already present: Breken caught 76 with 20 false alarms, Claude Fable 5.1 in Claude Code caught 37 with 59, and GPT-6 Astra in Codex CLI caught 28 with 86. Among the 166 bugs that need the product running, Breken caught 54 and Fable 5.1 caught 14. The dataset, judge prompts and labels are public and other systems can be submitted.

Getting started

Sign in to breken.ai with GitHub and pick repositories. Usage is prepaid with a $25 minimum top-up. It does not replace unit tests or code review, and Breken itself presents the layers as complementary. It fits best on products with stateful logic such as payments, sync, collaboration or billing: run it once on a service with a history of incidents, compare its findings and false alarms against your existing issues, then decide whether to add it to your release process.