Benchmark9d ago
Cognition launches FrontierCode 2.0 coding benchmark: it asks whether a PR should be merged, and top model Claude Opus 5.5 scores just 45.5%
Cognition, the maker of Devin, released FrontierCode 2.0 on Oct 27. It keeps the method of maintainer-written tasks graded on whether the code is actually mergeable, but draws tasks from more repositories, adds security patches and performance work, uses deliberately underspecified prompts and adds many thoroughness criteria. In Cognition's results Claude Opus 5.5 leads at 45.5%, GPT-6 Astra follows at 42.8%, and Grok 4.7 and Gemini 4 Argon drop into the 20s, leaving the benchmark far from saturated.
Pricing48d ago
Cognition: SWE-2 free on Devin Cloud for Pro/Max/Teams through Oct 8
Cognition co-founder Silas Alberti said demand for SWE-2 in Desktop and CLI is unprecedented, that they secured more compute, and that SWE-2 is free on Devin Cloud for Pro, Max, and Teams subscribers through October 8, 2026. The limited window lowers the cost of trying the coding model for existing Devin subscribers who want a short evaluation or production pilot.
Agents27d ago
Devin now works with any personal ChatGPT plan: Go, Plus or Pro subscriptions cover GPT model usage instead of Devin quota
On Oct 9 Cognition said Devin can now link any personal ChatGPT plan (Go, Plus or Pro). The GPT share of each session is billed to the ChatGPT subscription instead of Devin quota or on-demand credits, across Devin Cloud, Desktop and CLI. It requires a self-serve Devin Pro, Max or Teams plan (not Free or Enterprise), and falls back to Devin quota when the ChatGPT allowance runs out.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.