AI Coding Live News

LG AI Research Opens Free K-EXAONE 2.5 API: 750B Total, 31B Active, 512K Context, with a Self-Reported 59.2 on SWE-Bench Pro, Just Above DeepSeek-V4.1-Flash

On November 9 at NVIDIA AI Day Seoul, LG AI Research made the K-EXAONE 2.5 API free through its EXAONE API Platform (docs.exaone.ai). Developers, startups, enterprises and research institutions can get an API key and use it for service development and research. K-EXAONE 2.5 is a technical validation model built by further training K-EXAONE 2.0, released in August: 750B total parameters with 31B active, text-only, Korean and English, a 512K context and an OpenAI-compatible endpoint. On four agentic benchmarks run internally, LG reports 59.2 on SWE-Bench Pro, 78.0 on BrowseComp, 74.6 on AutomationBench and 77.3 on Claw-Eval, roughly on par with DeepSeek-V4.1-Flash and GLM-5.3 but not leading across the board. NVIDIA says the model was trained with Nemotron datasets. LG has not published how long the free access lasts or any quota, and has not released 2.5 weights.

LG AI Research Opens Free K-EXAONE 2.5 API: 750B Total, 31B Active, 512K Context, with a Self-Reported 59.2 on SWE-Bench Pro, Just Above DeepSeek-V4.1-Flash

Key Takeaways

  • Free access: the EXAONE API Platform launched on November 9 and is currently free of charge; get an API key to call it. EXAONE Tabular, a 21M-parameter tabular prediction model, launched alongside it
  • Specs: 750B total / 31B active parameters, text-only, Korean and English, 512K context and 512K max output, January 2026 knowledge cutoff; supports tool calling, structured outputs, code generation and streaming
  • API: OpenAI-compatible POST /v1/chat/completions with model name k-exaone_v2.5; reasoning cannot be turned off, reasoning_effort takes low / high / max (default max), and reasoning tokens count toward max_tokens
  • LG internal results (vs DeepSeek-V4.1-Flash / GLM-5.3): SWE-Bench Pro 59.21 vs 58.41 / 64.03; BrowseComp 78 vs 75.8 / 79; AutomationBench 74.6 vs 74.9 / 71.6; Claw-Eval 77.29 vs 75.78 / 77.64
  • Versus K-EXAONE 2.0: AutomationBench rose from 16.9 to 74.6 and SWE-Bench Pro from 47.1 to 59.21. All figures are vendor-run and not independently reproduced; LG has not stated the free period, quotas or whether 2.5 weights will be released
Read full story
Linked Models & Agents:

Tencent Open-Sources DSpine, a Parallel Draft Model for Qwen3.8-27B: Vendor Figures Show About 3.19x Throughput on SWE-bench Pro Replay, Ahead of DFlash2 Across the Board

Tencent published tencent/Qwen3.8-27B-DSpine-b16 on Hugging Face at 14:20 UTC on 2026-11-07: a five-layer speculative-decoding draft model of about 2B parameters with block size 16 and an 8,192-token sliding window, under Apache 2.0, with code at GitHub Tencent/DSpine and the paper at arXiv 2609.36173. The model card (vendor figures) shows that with SGLang 0.5.22, BF16 and concurrency 8, output throughput on a SWE-bench Pro replay at xhigh reasoning rose from about 248 to about 791 tokens/s (about 3.19x), versus 2.87x for the official DFlash2; acceptance length beat DFlash2 in all 10 agent settings. Note that DSpine drafts 16 tokens per block while DFlash2 drafts 8, so the setups are not identical, and no third-party reproduction exists yet.

Tencent Open-Sources DSpine, a Parallel Draft Model for Qwen3.8-27B: Vendor Figures Show About 3.19x Throughput on SWE-bench Pro Replay, Ahead of DFlash2 Across the Board

Key Takeaways

  • Release: Tencent uploaded tencent/Qwen3.8-27B-DSpine-b16 at 14:20 UTC on 2026-11-07; about 2.03B parameters (about 4GB in BF16), five layers, block size 16, 8,192-token sliding window, Apache 2.0
  • Throughput (vendor figures): SWE-bench Pro replay at xhigh reasoning went from about 248 to about 791 tokens/s, about 3.19x; DFlash2 reached 2.87x
  • Best speedup (vendor figures): about 3.84x on WorkBuddy Office at low reasoning (DFlash2: 3.17x)
  • Acceptance length (vendor figures): 4.77 vs 4.00 for DFlash2 on SWE-bench Pro at xhigh; 7.20 vs 5.69 on GSM8K with greedy decoding
  • Caveat: DSpine drafts 16 tokens per block and DFlash2 drafts 8, so conditions are not identical; only model-card numbers exist so far, with no third-party reproduction
Read full story

Microsoft Foundry Local Wires Qwen3.8-27B into GitHub Copilot: Local Coding Agent on ~24 GB VRAM; Vendor Terminal-Bench 2.1 ~73.0

On 2026-11-05 Microsoft Foundry blogged that Foundry Local CLI 0.13.0+ can run 4-bit Qwen3.8-27B (~18 GB weights) on-device via ONNX Runtime and connect it to GitHub Copilot through Bring Your Own Key. Needs an NVIDIA GPU with at least 24 GB VRAM (about 64K context on 24 GB; up to the model's 262K max on 32 GB+). Vendor-cited model-card scores include Terminal-Bench 2.1 73.0 and SWE-bench Pro 61.7; Microsoft also reports higher generation speed than same-quant llama.cpp on RTX 5090 and DGX Spark. Mac and AMD support are still on the roadmap.

Microsoft Foundry Local Wires Qwen3.8-27B into GitHub Copilot: Local Coding Agent on ~24 GB VRAM; Vendor Terminal-Bench 2.1 ~73.0

Key Takeaways

  • Shape: Foundry Local 0.13.0+ runs 4-bit Qwen3.8-27B (~18 GB weights) and connects to the GitHub Copilot app via BYOK
  • Hardware: NVIDIA GPU with >=24 GB VRAM; ~64K context on 24 GB, up to 262K on 32 GB+; install CUDA/cuDNN yourself
  • Vendor-cited model-card scores (not a 4-bit re-run): Terminal-Bench 2.1 73.0, SWE-bench Pro 61.7, NL2Repo-Bench 42.3, DeepSWE 1.1 42.2
  • Stack: ONNX Runtime + DFlash2 speculative decoding + INT4/KV quant + prefix caching; Microsoft says the gap vs llama.cpp widens past 131K tokens
  • Limits: Microsoft says cloud models still win on hard reasoning; Mac/AMD not shipped; Copilot CLI local model does not imply offline—set COPILOT_OFFLINE separately
Read full story

Ricursive Open-Sources Ripple-65B Preview: A 65B Dense Model Pretrained From Scratch, TerminalBench 2.1 80.5, Apache 2.0

Ricursive Intelligence released its first model, Ripple-65B Preview, on November 4 (18:24 UTC), open-sourcing the instruction-tuned model (BF16 and NVFP4) and a base model on Hugging Face under Apache 2.0. The official model card describes a 65B dense Transformer pretrained on 12T tokens with up to 1M context, and self-reports TerminalBench 2.1 80.5, SWE-bench Pro v2.0 Hard 79.1 and DeepSWE 1.1 44.3, ahead of much larger US open-weight models on agentic coding. It fits on local hardware such as the DGX Spark. CEO Anna Goldie says it cost about a tenth of the prior comparable state of the art.

Ricursive Open-Sources Ripple-65B Preview: A 65B Dense Model Pretrained From Scratch, TerminalBench 2.1 80.5, Apache 2.0

Key Takeaways

  • Ripple-65B Preview is a 65B dense model with a thin-and-deep shape (80 layers, d_model 5120), moving from an Olmo 3 baseline to a Gemma 3N-like architecture.
  • It was pretrained on 12T tokens at 8K sequence length, then went through mid-training, long-context training, SFT and mixed-domain RL, with up to 1M context.
  • Self-reported TerminalBench 2.1 is 80.5, above Reflection Beam 501B (80.1) and Muse Spark Open 816B (76.4), but SWE-bench Verified 76.5 and DeepSWE 1.1 44.3 show it does not lead everywhere.
  • Ricursive says the core team grew from one to five people and code compute efficiency improved 16x versus the Olmo 3 ladder; it reaches about 250 tok/s on 2x B200.
  • Hugging Face hosts BF16, NVFP4, base and DFlash2 speculative-decoding checkpoints, all under Apache 2.0.
Read full story
Linked Models & Agents:
⚡ Ripple-65B Preview⚡ Reflection Beam 501B⚡ Muse Spark Open⚡ Nemotron 3 Ultra🌐 Ricursive Intelligence🌐 NVIDIA
ADSponsored

Shanghai AI Lab ships Atria Dawn full release: 744B MoE GLM-5.2 agentic model, context doubled to 512K, MIT weights

Shanghai AI Laboratory has moved Atria Dawn from Preview to a full release, with open weights (plus an FP8 build) on Hugging Face under internlm/Atria-Dawn and on ModelScope, MIT licensed. Built on the 744B-parameter MoE GLM-5.2, it targets long-horizon research and engineering agent work; context doubles from the Preview's 256K to 512K and input is text only. The vendor table shows it leading the listed models on AutomationBench, CyberGym and SkillsBench, while still trailing Claude Opus 5 on coding tests such as SWE-bench Pro and Terminal-Bench 2.1 (all vendor-reported).

Shanghai AI Lab ships Atria Dawn full release: 744B MoE GLM-5.2 agentic model, context doubled to 512K, MIT weights

Key Takeaways

  • 512K context, double the Preview's 256K; built on the 744B-parameter MoE GLM-5.2, text input only
  • BF16 and FP8 weights on Hugging Face and ModelScope, MIT license
  • Vendor-reported: AutomationBench 54.7, CyberGym 86.8, SkillsBench 67.8, BFCL V4 78.4, each the top score in its table
  • Weak spots from the same table: SWE-bench Pro 61.7 vs 76.5 for Claude Opus 5, Terminal-Bench 2.1 81.0 vs 90.2
  • Local serving needs SGLang v0.5.13.post1+ or vLLM v0.23.0+; an OpenAI-compatible API also plugs into Codex
Read full story
Linked Models & Agents:

BOSS Zhipin's Nanbeige lab open-sources Nanbeige4.3-33B, a 33B dense agent model with 512K context under Apache 2.0, reporting 80.9 on Terminal-Bench 2.1 vs 73.0 for Qwen3.8-27B

At 12:11 UTC on 2026-10-25, the Nanbeige LLM Lab of BOSS Zhipin (Kanzhun Limited) announced Nanbeige4.3-33B from its account @nanbeige (post), with the model and its Base version released as open weights on Hugging Face under Apache 2.0 (model card). It is a 33B dense model on a Llama-style architecture with a 512K context window, positioned as the teacher model of the Nanbeige4.3 series that will be distilled into smaller models such as Nanbeige4.3-3B. The official model card compares it with Qwen3.8-27B, Muse-Glimmer-30B and Gemma4-31B using the same harness at highest thinking effort: 80.9 on Terminal-Bench 2.1 (Qwen3.8-27B 73.0) and 65.5 on DeepSWE (42.5), but 60.3 on SWE-Bench Pro, slightly below Qwen's 61.7, and 84.8 on GDPval vs 86.4. All figures are self-reported and not yet independently reproduced.

Key Takeaways

  • Official sources: @nanbeige post on X and the Hugging Face model card; the post had about 616 likes, 79 reposts and 38K views when we pulled it
  • Specs: 33B dense, Llama-style architecture, 512K context, Apache 2.0, with a Base checkpoint released too
  • Self-reported coding scores: Terminal-Bench 2.1 80.9, Terminal-Bench 4.0 23.8, DeepSWE 65.5, SWE-Bench Pro 60.3 (Qwen3.8-27B: 73.0, 6.3, 42.5, 61.7)
  • Self-reported cowork scores: Automation Bench 54.2, JobBench 44.8, GDPval 84.8 (Qwen3.8-27B: 38.3, 34.5, 86.4)
  • Deployment: Transformers, SGLang and vLLM, with official examples using 4-way tensor parallelism; Nanbeige4.3-3B distillation is next
Read full story
Linked Models & Agents:
⚡ Nanbeige4.3-33B⚡ Qwen3.8-27B⚡ Gemma4-31B⚡ Muse-Glimmer-30B🌐 Nanbeige🌐 BOSS Zhipin🌐 Kanzhun

Ollama Natively Supports Recursive Flint 40B: 2.5B Active MoE Coding Agent Delivering 125+ tok/s on MacBook Pro

Recursive, founded by Richard Socher, has officially open-sourced Flint (RecursiveSI/flint), a 39.8B Mixture-of-Experts (MoE) coding agent model under Apache 2.0 with day-zero Ollama support (ollama run flint). Activating just 2.5B parameters per token across 47 MoE layers with a 262K context window, Flint hits 125+ tokens/sec on an M4 Pro MacBook Pro. It sets new benchmarks for sub-50B local models, scoring 74.2% on SWE-bench Verified and 89.7% on SWE-bench Pro v2.

Ollama Natively Supports Recursive Flint 40B: 2.5B Active MoE Coding Agent Delivering 125+ tok/s on MacBook Pro

Key Takeaways

  • Flint employs a 49-layer architecture (2 dense + 47 MoE layers) with 39.8B total and 2.5B active parameters per token
  • Native 262,144-token (256K) context window supports end-to-end repository indexing and cross-file debugging
  • Blazing local inference speed: 125+ tokens/sec on M4 Pro MacBook Pro and 150+ tokens/sec on RTX Spark
  • Scores 74.2% on SWE-bench Verified, 89.7% on SWE-bench Pro v2, and 80.3% on LiveCodeBench v6 with instant 'ollama run flint' execution
Read full story
Linked Models & Agents:

Recursive releases its first model, Flint: a 40B-total, 2.5B-active open-weights local coding model under Apache 2.0, running at 125+ tok/s on a MacBook

On Oct 21, Recursive, the startup founded by Richard Socher, released its first model, Flint: a 39.8B-total MoE coding model with about 2.5B active parameters per token, with weights on Hugging Face under Apache 2.0. Recursive says it leads sub-50B MoE models on SWE-Bench Pro v2, Terminal-Bench 2.1 and SWE-Bench Verified, runs at 125+ tok/s on an M4 Pro MacBook Pro, and was built from scratch in under four months with its automated research system, the Eureka Machine, contributing to nearly every part of the stack.

Recursive releases its first model, Flint: a 40B-total, 2.5B-active open-weights local coding model under Apache 2.0, running at 125+ tok/s on a MacBook

Key Takeaways

  • Scale: 39.8B total, ~2.5B active incl. output head; 256 routed experts pick 6 plus 1 shared expert; Apache 2.0 (model card)
  • Benchmarks (vendor-reported): Terminal-Bench 2.1 54.9%, SWE-bench Verified 74.2%; the card lists SWE-bench Pro v2 at 89.7%
  • Speed (vendor-reported): 125+ tok/s on M4 Pro MacBook Pro, 150+ tok/s on 64GB RTX Spark at short contexts, up to 2x comparable models
  • Context: 262,144 tokens in the model card config; the blog says up to 512K, best within 256K
  • Ecosystem: day-one llama.cpp, Ollama, LM Studio, vLLM, SGLang, plus the open-source rec local engine; free to try on OpenCode and Pi
Read full story

Ornith-1.5-122B Released Under MIT: A Single-Node 122B MoE Reasoning Model Scoring 76.9 on SWE-bench Pro

The Ornith AI team officially released Ornith-1.5-122B under the permissive MIT license, a 122-billion parameter mixture-of-experts (MoE) reasoning model optimized to execute on a single GPU node (such as a single DGX). The model is immediately available across vLLM (0.19.1+), SGLang (0.5.9+), Hugging Face, and Ollama (ornith-1.5:122b) with native FP8, GGUF, and MLX quantization. Across independent agentic evaluations in the OpenHands harness, Ornith-1.5-122B registered 84.5 on SWE-bench Verified, 76.9 on SWE-bench Pro (surpassing Qwen3.8-Flash-Next's 43.9 by 33 points), 81.2 on TerminalBench 2.1, and 74.6 on MCP-Atlas.

Ornith-1.5-122B Released Under MIT: A Single-Node 122B MoE Reasoning Model Scoring 76.9 on SWE-bench Pro

Key Takeaways

  • Ornith-1.5-122B released under the permissive MIT license, offering a fully open, frontier-grade 122B MoE coding reasoning model
  • Engineered to fit inside a single GPU node (single DGX), with immediate runtime support on vLLM, SGLang, and Ollama (ornith-1.5:122b)
  • Achieves 76.9 on SWE-bench Pro, drastically outperforming the comparable open-weight baseline Qwen3.8-Flash-Next (43.9)
  • Delivers balanced agentic performance with 81.2 on TerminalBench 2.1, 80.0 on CyberGym, and 74.6 on MCP-Atlas
Read full story
Linked Models & Agents:
⚡ Ornith-1.5-122B⚡ Qwen3.8-Flash-Next🤖 OpenHands🌐 Ornith

Meta Releases Muse Glimmer 30B: An Apache 2.0 On-Device Agentic Model Running on a Single Consumer GPU

Meta AI published the technical report for Muse Glimmer 30B (arXiv:2610.24628), an open-source 30-billion parameter causal language model with an integrated perception encoder designed for on-device autonomous agents. Quantized to approximately 4 bits, Muse Glimmer operates inside 32GB of VRAM on a single consumer GPU (such as an RTX 5090) without cloud reliance. On rigorous agentic evaluations, Muse Glimmer sets new records for its size class: achieving 75.5 on MCPAtlas (surpassing Gemma4-31B at 54.2 and Qwen3.6-27B at 62.5) and 51.2 on SWE-Bench Pro, released fully under the Apache 2.0 license.

Meta Releases Muse Glimmer 30B: An Apache 2.0 On-Device Agentic Model Running on a Single Consumer GPU

Key Takeaways

  • Meta released the technical report for Muse Glimmer 30B (arXiv:2610.24628), open-sourcing the agentic foundation model under Apache 2.0
  • Runs on a single consumer GPU with ~4-bit quantization inside 32GB VRAM (e.g. RTX 5090), enabling 100% offline edge autonomy
  • Scores 75.5 on MCPAtlas tool use benchmark, outperforming Gemma4-31B (54.2) and Qwen3.6-27B (62.5) by wide margins
  • Achieves 51.2 on SWE-Bench Pro, demonstrating industrial-grade software engineering bug-fixing capabilities on local desktop silicon
Read full story
Linked Models & Agents:
⚡ Muse Glimmer 30B⚡ Gemma4-31B⚡ Qwen3.6-27B🌐 Meta
ADSponsored

JetBrains Unveils SWE-TaskFlow: Aligning Coding Agent Benchmarks with Real Developer IDE Workflows

Mainstream coding agent benchmarks like SWE-bench frame evaluation around issue-derived tasks: feeding an agent a bug description and expecting an end-to-end patch. But does this reflect genuine developer practices? JetBrains Research analyzes 4,782 production sessions of professional software engineers inside JetBrains IDEs (arXiv:2610.09633). For long interactive sessions (>=3 turns, comprising 33% of production traffic), user requests span a diverse mix of intent types—project codebase Q&A, strategic planning, code review, refactoring, and test execution—with users frequently transitioning between types mid-session. To align benchmarks with real developer task flows, JetBrains introduces SWE-TaskFlow, an approach that transforms issue-based benchmarks via prompt-splitting and verifiable repository QA into calibrated multi-stage workflows guided by a TaskFlow Alignment Score (TFAS). A pilot study across 700 SWE-Bench Pro tasks demonstrates that sequential multi-step interaction protocols nearly double agent inference expenditure without stable improvements in resolve rates, proving that the interaction protocol itself is a critical, previously unmeasured dimension of agent evaluation.

JetBrains Unveils SWE-TaskFlow: Aligning Coding Agent Benchmarks with Real Developer IDE Workflows

Key Takeaways

  • JetBrains analyzes 4,782 production IDE sessions, proving real software engineers navigate dynamic multi-intent task flows
  • Introduces SWE-TaskFlow and TFAS to transform issue benchmarks into realistic multi-turn developer interaction protocols
  • Pilot across 700 SWE-Bench Pro tasks reveals multi-turn protocols double token costs without stable success gains
Read full story
Linked Models & Agents:

Opera: A Verbal Critic Framework for Long-Horizon Coding Agents Tracks Diagnosed Flaws Until Resolution to Boost SWE-Bench by 15%

Long-horizon coding agents tackling complex software repositories depend on timely guidance, yet conventional LLM critics often degrade agent outcomes by misinterpreting in-progress exploratory commands or offering ephemeral advice that is never tracked. Researchers from Rutgers and Lehigh University unveil Opera (arXiv:2609.33987), an open-source verbal critic framework that treats diagnostic corrections as persistent notes tracked until root problems are definitively resolved. Opera orchestrates reviews through hybrid periodic and event-driven triggers, verifies proposed critiques against visible terminal artifacts, and monitors downstream trajectories to distinguish mere superficial compliance from genuine resolution. As a test-time intervention, Opera elevates baseline agent resolve rates by 12.4 percentage points on Terminal-Bench 2.1, 15.0 on SWE-Bench Pro, and 8.9 on DeepSWE v1.1 across four foundation backbones. Furthermore, fine-tuning Qwen3.5-9B on Opera-guided rollouts yields a 10.2 percentage point gain on held-out repositories without any test-time critic, preserving stability across heterogeneous execution harnesses (such as OpenHands to Terminus-2).

Key Takeaways

  • Rutgers and Lehigh release Opera, introducing persistent diagnostic notes that track diagnosed faults until proven resolution in coding agents
  • Elevates resolve rates by 12.4 points on Terminal-Bench 2.1 and 15.0 points on SWE-Bench Pro across multiple policy models
  • Fine-tuning Qwen3.5-9B on Opera rollouts yields a 10.2 point gain on held-out repos without test-time critics, preserving cross-harness stability
Read full story
Linked Models & Agents:

Cline: Tencent Hy4 Preview leads SWE-bench Pro at 770B/49B active with 1M context

Cline said Tencent Hy4 Preview currently leads SWE-bench Pro. The open-weight model is described as 770B parameters with 49B active and a 1M-token context window, and Cline called it Tencent's biggest generational leap measured to date. The practical developer path is immediate: npm i -g cline, run /model, and select Hy4 preview. SWE-bench Pro is a harder, more realistic software-engineering eval than classic HumanEval-style completion, so a lead there is a stronger signal for repository-level agents than a short coding quiz. The 1M context window is the other half of the story: long-horizon refactors, multi-file patches, and eval traces can stay in a single session instead of being retrieved in fragments. Cline's distribution also matters; putting Hy4 beside GLM-5.3 and Qwen3.8 in the same agent makes model shopping a /model command rather than a new IDE.

Key Takeaways

  • Hy4 Preview is an open-weight 770B/49B-active model with 1M context that Cline says leads SWE-bench Pro.
  • Cline calls it Tencent's largest measured generational leap and already hosts Hy4 preview in-product.
  • Developers can switch via npm i -g cline and /model for repository-scale long-context repairs.
Read full story

Alibaba Qwen open-weights Qwen3.8-Flash: 125B/6B multimodal MoE scoring 62.5 on SWE-bench Pro

Alibaba Qwen released open-weight Qwen3.8-Flash, a multimodal Mixture-of-Experts model billed as an early preview of the Qwen4 architecture. The production-scale checkpoint is 125B parameters plus 51B N-gram embeddings, with only 6B activated per token. The new stack combines Gated DeltaNet plus Qwen Sparse Attention, Gated Residual branches, N-gram embeddings that can live in host memory, and a refined Muon optimizer. Qwen said it trained at about one-ninth the cost of Qwen3.7-Plus while beating it across the board, especially on coding and office work: 58.7 on DeepSWE 1.1, 62.5 on SWE-bench Pro, 73.9 on CoWorkBench, 84.5 on AndroidWorld, and 95.7 on MathVision with CI. Native context is 262K tokens, extendable to 1M with YaRN; at 1M context the QSA kernel is up to 7.6x faster in prefill and 4.9x faster in decode. QwenCloud lists roughly $0.15 input / $0.47 output per million tokens, and day-0 runtimes arrived from NVIDIA NeMo, SGLang, Unsloth, OpenRouter, and OpenCode Go.

Key Takeaways

  • 125B/6B MoE hits 62.5 SWE-bench Pro and 58.7 DeepSWE 1.1 at ~1/9 the training cost of Qwen3.7-Plus.
  • QSA hybrid attention is up to 7.6x faster in prefill and 4.9x in decode at 1M context.
  • Open weights plus day-0 support on QwenCloud, OpenRouter, OpenCode, NeMo, SGLang, and Unsloth.
Read full story

SWE-bench team launches SWE-bench Pro multilingual benchmark covering industrial Rust, Go, and C++ bug fixes

The SWE-bench benchmark team launched SWE-bench Pro, expanding from Python to Rust, Go, C++, and TypeScript. Featuring 1,200 curated real-world issues from high-traffic open-source repositories, it provides a unified industry standard for multi-language autonomous agent software engineering evaluation.

Key Takeaways

  • Includes 1,200 real-world repository repair tasks across Rust, Go, C++, and TypeScript.
  • Applies strict compiler verification and integration test matrices to eliminate false passes.
  • First leaderboard standings show Claude 3.7 Sonnet and DeepSeek-R1 leading multi-language solve rates.
Read full story