AI Coding Live News

SemiAnalysis releases GDPx, an agentic benchmark of 206 tasks across 11 high-value domains where the top model, Claude Opus 5.5, scores just 12%

On Nov 2, semiconductor and AI research firm SemiAnalysis released GDPx, a benchmark of agentic capability across 11 domains: chip design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement (RSI), hardware design, materials science, alignment and software engineering. Its 206 tasks were built by 14 specialist data partners, with SemiAnalysis doing final QA. On the official leaderboard Claude Opus 5.5 leads at 12.0%, followed by Gemini 4 Carbon at 10.9% and GPT-6 Astra at 9.6%; nearly every model scores 0 on finance and legal. SemiAnalysis says safety classifiers drag down Opus 5.5 and GPT-6 Astra, and plans a new version every 3–6 months. All scores are as reported by SemiAnalysis.

SemiAnalysis releases GDPx, an agentic benchmark of 206 tasks across 11 high-value domains where the top model, Claude Opus 5.5, scores just 12%

Key Takeaways

  • Scale: 11 domains, 14 data partners and 206 tasks, with a default time limit of 8 hours per task (GDPx site)
  • Leaderboard (as reported by SemiAnalysis): Claude Opus 5.5 12.0%, Gemini 4 Carbon 10.9%, GPT-6 Astra 9.6%, Muse Wonder Full 9.0%, Claude Fable 5.1 8.5%, GPT 6.1 Sol 8.4%, Claude Sonnet 5.5 8.2%
  • Lower tier and open weights: DeepSeek 4.1 Flash 1017 5.4%, GPT-6 Luna 5.1%, Grok 4.7 3.9%, GLM 5.3 2.7%; almost every model scores 0 on finance and legal
  • Method: each model runs in its native harness where one exists (Claude Code for Opus, Codex for Astra, Antigravity for Carbon), otherwise OpenCode; the headline is pass@1 over three attempts with all domains weighted equally (SemiAnalysis blog)
  • By domain: Opus 5.5 wins quant trading, hardware and chip design; Gemini 4 Carbon wins biology and software engineering; GPT-6 Astra wins RSI and materials science; Muse Wonder Full is the strongest cyber model
Read full story

X discussion: big-tech CEOs talk openly about RSI - Google's Sundar Pichai says "we are pretty far from the end state of RSI" yet expects "continued acceleration because of RSI" over the next 12 months, while Sam Altman and Dario Amodei weigh in

Recursive self-improvement (RSI), using models to speed up the development of the next models, became one of the most-used terms in AI circles on X this past week after several big-tech CEOs talked about it in public. Google CEO Sundar Pichai framed it differently on two podcasts. On Alex Kantrowitz's Big Technology Podcast he said "We are pretty far from the end state of RSI." On Alex Heath's Sources podcast he said researchers are co-working with models "in a deep way" and "I would expect the next 12 months to see continued acceleration because of RSI." The widely shared 12-month line comes from the Sources episode, not Big Technology. OpenAI CEO Sam Altman told Stanford GSB that besides AI recursive self-improvement there is another loop, robots that build robots and data centers that build data centers, that gets "much less talk". Anthropic CEO Dario Amodei, replying to a critic on X, wrote about the point where AI-driven acceleration gets fast enough that safety struggles to keep up: "We are nearing that point." This is an X discussion of public remarks by CEOs, not an announcement of any new research program.

Key Takeaways

  • On the Big Technology Podcast (released Oct 20), Pichai said "We are pretty far from the end state of RSI", that today there are "RSI loops along with AI researchers", and that he expects that continuum to keep progressing into next year (Big Technology recap; Kantrowitz post, 68 likes and 9,643 views when we pulled it)
  • On the Sources podcast (released Oct 21, from about 11:52), Pichai said researchers co-work with models "in a deep way", with models suggesting recipes and automatically evaluating next checkpoints, and added "I would expect the next 12 months to see continued acceleration because of RSI" (Alex Heath episode post, 253 likes and 388K views when we pulled it)
  • Reposts that place the 12-month line in the Big Technology interview are wrong; after checking transcripts of both shows we found it in the Sources episode. The early repost from @ONagel33303 had only 4 likes, and its "CONFIRMED: Google plans to accelerate" framing is the reposter's own reading
  • At Stanford GSB's View From The Top (video released Oct 21, at about 33:18), Altman described "robots that can build more robots and then data centers that can effectively build more data centers" as a loop besides AI recursive self-improvement that gets "much less talk"
  • Anthropic CEO Dario Amodei, replying to Encode AI's Nathan Calvin, wrote about the point where AI-driven acceleration gets fast enough that it is hard for safety to keep up: "We are nearing that point" (post, 4,103 likes and 1.59M views when we pulled it); Calvin and journalist Garrison Lovely then criticized Anthropic for warning about RSI while backing the AEGIS Act, which they say makes it harder for governments to stop it
Read full story

Recursive Self-Improvement for Embodied Navigation: RSI-Nav Unifies Policy-Synchronized Dual Bootstrapping, Achieving 68.6% SR on R2R-CE Without Human Annotation

Vision-language navigation requires abundant, high-quality language supervision, yet traditional synthetic pipelines generate supervision from static models that lag behind evolving navigation policies. Shanghai AI Lab, Tsinghua, and SJTU introduce RSI-Nav, an embodied recursive self-improvement framework combining instruction-following and instruction-generation in a single model. Using shard-wise dual bootstrapping on unlabeled trajectories, RSI-Nav achieves state-of-the-art 68.6% SR on R2R-CE and 72.4% SR on RxR-CE without human intervention.

Recursive Self-Improvement for Embodied Navigation: RSI-Nav Unifies Policy-Synchronized Dual Bootstrapping, Achieving 68.6% SR on R2R-CE Without Human Annotation

Key Takeaways

  • Unified dual-capability architecture: combines vision-language instruction following (VL2A) and vision-action instruction generation (VA2L) within a shared foundation model
  • Shard-wise policy-synchronized bootstrapping: generates instructions over small trajectory shards, validates via simulation rollout, and updates parameters instantly before processing subsequent batches
  • New continuous-environment SOTA: secures 68.6% Success Rate on R2R-CE and 72.4% on RxR-CE, proving robust recursive self-evolution in complex embodied navigation
Read full story

Google Cloud AI Unveils Harness²: Two-Level Recursive Agent Harnessing Unlocks Self-Evolution in Open-World Tasks

An LLM agent's practical capabilities depend as much on its execution harness as on its underlying model. Deploying recursive self-improvement (RSI) in open worlds is hampered by the absence of ground-truth verifiers and poor generalization across novel domains. Google Cloud AI Research introduced Harness², a two-level recursive framework: task-level recursion refines task-specific harnesses via a contrastive Propose-Probe-Compose step without ground-truth verifiers, while harnessing-level recursion extracts continual adaptation experience. Harness² delivers up to +11.8% on LAB, +11.5 points on WorkBuddy-Bench, and +4.5 points on JobBench on GLM-5 and Gemini 3.5 Flash. Code is open-sourced on GitHub under Apache-2.0 (google-research/harness2), supporting OpenCode and Codex executors plus JobBench, LAB and WorkBuddy-Bench.

Google Cloud AI Unveils Harness²: Two-Level Recursive Agent Harnessing Unlocks Self-Evolution in Open-World Tasks

Key Takeaways

  • Two-level recursion architecture: disentangles task-level harness adaptation from continuous harnessing-level meta-learning without ground-truth verifiers
  • Propose-Probe-Compose mechanism: guides runtime harness modifications via contrastive probes, bypassing reliance on fixed unit tests in open-world tasks
  • Consistent cross-model gains: boosts GLM-5 and Gemini 3.5 Flash by +11.8% on LAB, +11.5 points on WorkBuddy-Bench, and +4.5 points on JobBench
Read full story
ADSponsored

AutoEnvScaling: Microsoft and Stanford Automate RL Environment Design via Terminal Agents, Unlocking 25x Data Flywheel

Microsoft Research and Stanford University have unveiled AutoEnvScaling, an automated data flywheel that frames RL environment creation itself as a terminal execution task. A proposer agent builds diverse environments directly from solver rollouts, slashing cost per valid environment by 75% while achieving a 25x higher valid environment yield over prompt-only baselines. Crucially, unifying environment design and solving within a single model enables Recursive Self-Improvement (RSI), yielding +12.2 points on Terminal-Bench 2.1 and doubling scores on Terminal-Bench 4.0.

AutoEnvScaling: Microsoft and Stanford Automate RL Environment Design via Terminal Agents, Unlocking 25x Data Flywheel

Key Takeaways

  • Environment design as a terminal task: shifts RL environment synthesis from manual curation to automated terminal execution, boosting valid yield by 25x
  • Unlocks Recursive Self-Improvement (RSI): enables a single backbone to co-evolve as both environment architect and task solver without human bottlenecks
  • Broad cross-domain generalization: yields a 12.2-point surge on Terminal-Bench 2.1 and doubles Terminal-Bench 4.0 for Qwen3.6-35B-A3B across kernel coding and OS tasks
Read full story

RSI-Anything Open-Sourced: Unlocking Domain-General Recursive Self-Improvement for AI Agents with World Models

Empowering AI agents to autonomously achieve Recursive Self-Improvement (RSI) through ongoing environmental interaction is a cornerstone of artificial general intelligence. However, practical RSI pipelines are severely constrained by two fundamental bottlenecks: scarce and expensive ground-truth feedback, and the degradation of predictive world models as evolving policies induce out-of-distribution drift. Researchers from HKUST, Shanghai AI Laboratory, Peking University, and CUHK introduce RSI-Anything (arXiv:2610.13912, code: github.com/Gen-Verse/RSI-Anything), a domain-general framework unlocking recursive self-improvement via predictive world models. Built upon the JEPA-Anything architecture, RSI-Anything establishes a disciplined six-stage paradigm: Propose → Predict → Select → Evaluate → Learn → Retain. By enforcing rigorous compatibility and safety gates governing policy-world model retention, the system continuously converts interaction logs into persistent capability upgrades. Across diverse domains, RSI-Anything drives substantial performance gains: boosting automated code repair success from a 26.3% baseline to 31.2% (+4.9 points), reducing GUI navigation action footprints by 39.4%, more than doubling laboratory planning scores from 21.80% to 46.01% over five iterations, and achieving a 96.15% chemical yield, with codebase fully open-sourced.

RSI-Anything Open-Sourced: Unlocking Domain-General Recursive Self-Improvement for AI Agents with World Models

Key Takeaways

  • RSI-Anything open-sources a domain-general recursive self-improvement framework powered by predictive world models
  • Six-stage P-P-S-E-L-R loop lifts code repair from 26.3% to 31.2% and cuts GUI navigation action footprints by 39.4%
  • Doubles laboratory planning scores across 5 rounds (21.80% to 46.01%) and achieves 96.15% chemical yield over Bayesian optimization
Read full story

Sakana AI Unveils MASS: Recursive Self-Improvement via Multi-Agent Self-Supervision, Boosting Win Rates to 69.9%

When model capabilities exceed human expert evaluative bandwidth, achieving Recursive Self-Improvement (RSI) without external verifiers or compilers becomes the paramount frontier of AGI. While reinforcement learning with verifiable rewards (RLVR) thrives in formal code and math, open-ended research and complex reasoning tasks lack scalar ground truth, and solitary models evaluating their own outputs suffer from homogeneous confirmation bias. Sakana AI introduces MASS (arXiv:2610.12176, project: sakana.ai/mass), an RSI framework anchored in Multi-Agent Self-Supervision. Driven by a single base model (such as Qwen3.6-27B), MASS tasks the model with concurrently acting as task solver, workflow optimizer, and self-evaluator. The framework alternates between evolutionary multi-agent workflow discovery and supervised fine-tuning (SFT) over synthesized trajectories. Across two successive MASS self-evolution cycles, performance per output token increases by 1.2x to 1.6x across open-ended research suites (MLR-Bench, ScienceAgentBench, DSBench, AstaBench). Head-to-head win rates against the base model surge from 53.9% in round 1 to 69.9% in round 2, while a student model trained on multi-agent traces decisively beats a single-agent baseline trained on 1.4x more tokens.

Sakana AI Unveils MASS: Recursive Self-Improvement via Multi-Agent Self-Supervision, Boosting Win Rates to 69.9%

Key Takeaways

  • Sakana AI introduces MASS, using multi-agent self-supervision on a single base model to achieve recursive self-improvement
  • Win rate against base model climbs from 53.9% to 69.9% over two iterations, boosting token reasoning efficiency by 1.2x-1.6x
  • Distillation from multi-agent trajectories outperforms single-agent baselines trained on 1.4x more tokens
Read full story
Linked Models & Agents:

Embodied Turing Machines: NTU Singapore Proposes Code-Only-as-Policy (COAP) for Robots, Achieving 70.2% Success with Zero Test-Time Models

Contemporary embodied intelligence relies heavily on neural network policies: foundation models either run end-to-end Vision-Language-Action (VLA) architectures or repeatedly query Vision-Language Models (VLMs) at high frequencies via agent harnesses. This paradigm imposes severe inference latency, costly deployment bills, and unpredictable non-deterministic execution failures. Researchers from S-Lab at Nanyang Technological University (NTU Singapore) introduce a paradigm shift (arXiv:2610.12369): Embodied Turing Machines and Code-Only-as-Policy (COAP). Under this formulation, the physical environment operates as a Turing machine where the tape encodes robot proprioception and visual states, while the transition rules are executed entirely by deterministic, stateful code. A shared, modular code library governs decisions across episodes without running any neural network models at test time. Crucially, explicit stateful code provides the optimal medium for Recursive Self-Improvement (RSI): autonomous coding agents iteratively author, debug, and refactor robot skills in closed simulation loops. Across 42 complex bimanual manipulation tasks in RoboDojo, the synthesized COAP library reaches a 70.24% success rate with zero neural models running at runtime, slashing deployment latency and cost to zero.

Key Takeaways

  • NTU Singapore introduces Embodied Turing Machines and Code-Only-as-Policy (COAP), operating with zero neural networks at test time
  • Achieves 70.24% success rate across 42 bimanual tasks in RoboDojo, slashing decision latency to under 1ms with zero cloud API costs
  • Formulates robot policies as software libraries, enabling autonomous coding agents to drive closed-loop Recursive Self-Improvement (RSI)
Read full story
Linked Models & Agents:

TestPrism: Nanjing University Exposes the Single-Reference Illusion in Coding Agent Test Generation, Slashing Pass Rates from 59.7% to 28%

Autonomous coding agents are increasingly leveraged to synthesize test suites for enterprise software. However, the standard practice of evaluating generated tests against a single reference solution overlooks alternative valid implementations and inflates perceived test quality. Researchers from Nanjing University and Hong Kong Polytechnic University unveil TestPrism (arXiv:2610.12289), exposing this systemic evaluation illusion. TestPrism comprises 300 test generation tasks sourced from 17 repositories and pairs them with 3,000 candidate implementations evenly divided into valid and invalid solutions. Its primary metric, the Joint Success Function, mandates that a test suite must simultaneously fail against buggy baselines, pass across all diverse valid implementations, and reject every invalid candidate. Evaluating fourteen frontier coding agent setups reveals that while models register a 59.67% success rate under legacy single-reference scoring, their actual Joint Success Function plunges to just 28.00%. To resolve these foundational flaws, the authors introduce TestHelix, an agentic synthesis harness leveraging peer cross-validation and recursive self-improvement to elevate Joint Success scores by up to 9.00 percentage points.

Key Takeaways

  • Nanjing University introduces TestPrism, revealing single-reference evaluation artificially inflates coding agent test quality from 28.0% to 59.7%
  • Evaluates 300 tasks against 3,000 candidate implementations (half valid, half invalid) under a strict Joint Success Function
  • Presents TestHelix with peer cross-validation and recursive self-improvement, raising genuine test quality by up to 9.00 percentage points
Read full story

Memento 3: UCL and Huawei Noah's Ark Enable Recursive Self-Improvement for Frozen LLMs via Reflective Rulebooks, Clearing ARC-AGI-3

Endowing artificial agents with general intelligence requires inferring environment dynamics and refining world models continuously in the wild. Fine-tuning models directly is prohibitively expensive and precipitates catastrophic forgetting. Researchers from University College London (UCL) and Huawei Noah's Ark Lab UK present Memento 3 (arXiv:2610.11794), a framework enabling frozen LLM agents to achieve model-based Recursive Self-Improvement (RSI) purely via external memory. The agent maintains a natural-language rulebook representing its evolving hypotheses of environment physics, which is compiled into executable code for fast simulation and action planning. Governed by a continuous observation-reflection-revision-compilation-verification loop, the system leverages prediction errors to refine rules. Edits are merged only after passing LLM semantic alignment audits and deterministic cell-exact replay verification. On the rigorous ARC-AGI-3 abstraction benchmark, Memento 3 solves 100% of levels across all 25 public environments, scoring a perfect 100.0 Relative Human Action Efficiency (RHAE) while utilizing only 44% of human action counts. In Atari Pong, a synthesized feedback controller shuts out opponents 21:0 across three evaluation seeds with zero test-time LLM inference.

Key Takeaways

  • UCL and Huawei Noah's Ark introduce Memento 3, unlocking model-based Recursive Self-Improvement for frozen LLMs via executable rulebooks
  • Compiles natural-language physics hypotheses into code world models validated through cell-exact replay verification
  • Clears all 25 games on ARC-AGI-3 using only 44% of human actions, and delivers 21:0 Pong shutouts with zero runtime LLM calls
Read full story
ADSponsored

UniEvo-VL: On-Policy Self-Distillation Enables Recursive Self-Improvement in Multimodal Diffusion Generation

Modern unified multimodal models integrate visual generation and semantic understanding within a single parameter footprint, allowing them to critique their own generations. However, post-training pipelines conventionally depend on larger external teacher models. Researchers from Stanford, the University of Washington, and collaborating institutions (including Yejin Choi, Jure Leskovec, and Li Erran Li) introduce UniEvo-VL. The framework assigns a single model to act simultaneously as student (conditioned on raw prompts) and teacher (conditioned on privileged self-critiques), minimizing per-state divergence across denoising diffusion trajectories sampled on-policy. Built on Qwen-image-2512, UniEvo-VL elevates GenEval scores from 0.747 to 0.808 and GenEval2 Soft-TIFA from 32.97 to 35.53 without external supervision.

UniEvo-VL: On-Policy Self-Distillation Enables Recursive Self-Improvement in Multimodal Diffusion Generation

Key Takeaways

  • Introduces UniEvo-VL, enabling autonomous self-improvement in multimodal diffusion models without external teacher models
  • Enforces on-policy divergence minimization between student rollouts and privileged self-critique teacher trajectories
  • Improves GenEval accuracy from 0.747 to 0.808 and Soft-TIFA from 32.97 to 35.53 on Qwen-image-2512
Read full story

AutoDataBench: Benchmarking Autonomous Agent Data Synthesis for Recursive Self-Improvement without Full Training Loops

Recent leaps in language model performance stem predominantly from verifiable agentic training data rather than architectural revisions alone. However, generating verifiable tasks for supervised fine-tuning and reinforcement learning still relies heavily on expensive human-in-the-loop engineering, hindering recursive self-improvement (RSI). Researchers from CUHK and collaborating organizations introduce AutoDataBench, the first benchmark evaluating an agent's capability to autonomously synthesize verified training tasks against industrial production acceptance standards. Unlike traditional evaluation schemes requiring expensive end-to-end retraining runs, AutoDataBench directly inspects generated tasks across validity, novelty, difficulty, and behavioral coverage. Evaluated on three executable benchmarks, existing frontier agents score below 20/100 within a 45-minute budget; however, quadrupling the test-time budget significantly boosts success rates while maintaining unit costs, proving that autonomous synthetic data pipelines are viable when scaled with compute.

AutoDataBench: Benchmarking Autonomous Agent Data Synthesis for Recursive Self-Improvement without Full Training Loops

Key Takeaways

  • Industrial Acceptance Criteria for Synthetic Tasks: Replaces expensive end-to-end retraining evaluations with a sample-by-sample automated harness assessing validity, novelty, difficulty, and coverage directly.
  • Frontier Agent Synthesis Baseline: Across three executable benchmarks, current top agents achieve under 20/100 at standard 45-minute budgets, demonstrating that synthesizing verifiable tasks is significantly harder than solving them.
  • Validating Recursive Self-Improvement Scaling: Quadrupling the agent's generation and self-testing time substantially elevates valid task yields while marginal unit costs remain flat, confirming the viability of compute-driven synthetic data engines.
Read full story