AI Coding Live News

SemiAnalysis releases GDPx, an agentic benchmark of 206 tasks across 11 high-value domains where the top model, Claude Opus 5.5, scores just 12%

On Nov 2, semiconductor and AI research firm SemiAnalysis released GDPx, a benchmark of agentic capability across 11 domains: chip design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement (RSI), hardware design, materials science, alignment and software engineering. Its 206 tasks were built by 14 specialist data partners, with SemiAnalysis doing final QA. On the official leaderboard Claude Opus 5.5 leads at 12.0%, followed by Gemini 4 Carbon at 10.9% and GPT-6 Astra at 9.6%; nearly every model scores 0 on finance and legal. SemiAnalysis says safety classifiers drag down Opus 5.5 and GPT-6 Astra, and plans a new version every 3–6 months. All scores are as reported by SemiAnalysis.

SemiAnalysis releases GDPx, an agentic benchmark of 206 tasks across 11 high-value domains where the top model, Claude Opus 5.5, scores just 12%

Key Takeaways

  • Scale: 11 domains, 14 data partners and 206 tasks, with a default time limit of 8 hours per task (GDPx site)
  • Leaderboard (as reported by SemiAnalysis): Claude Opus 5.5 12.0%, Gemini 4 Carbon 10.9%, GPT-6 Astra 9.6%, Muse Wonder Full 9.0%, Claude Fable 5.1 8.5%, GPT 6.1 Sol 8.4%, Claude Sonnet 5.5 8.2%
  • Lower tier and open weights: DeepSeek 4.1 Flash 1017 5.4%, GPT-6 Luna 5.1%, Grok 4.7 3.9%, GLM 5.3 2.7%; almost every model scores 0 on finance and legal
  • Method: each model runs in its native harness where one exists (Claude Code for Opus, Codex for Astra, Antigravity for Carbon), otherwise OpenCode; the headline is pass@1 over three attempts with all domains weighted equally (SemiAnalysis blog)
  • By domain: Opus 5.5 wins quant trading, hardware and chip design; Gemini 4 Carbon wins biology and software engineering; GPT-6 Astra wins RSI and materials science; Muse Wonder Full is the strongest cyber model
Read full story

Artificial Analysis releases AA-Omniscience v1.1: 3,000 refreshed questions and a calibrated 300-question public set, with Gemini 4 Carbon leading at a Net Score of 54

On October 31, independent evaluator Artificial Analysis released AA-Omniscience v1.1, its knowledge and hallucination benchmark and part of Intelligence Index v5. The question set shrinks from 6,000 to 3,000, dropping previously public and saturated questions, fixing errors and ambiguities, and adding new ones, and a new 300-question public set on Hugging Face is calibrated to the private set's difficulty, with 0.99 Pearson and Spearman correlation on Net Score. Gemini 4 Carbon (high) leads with a Net Score of 54, ahead of Claude Opus 5.5 at 48 and Claude Fable 5.1 at 46; Fable 5.1 has the highest accuracy (68%) and Gemini 4 Argon the lowest hallucination rate (4%). Models score about 1.9 points lower than on v1 on average, and MiniMax M3.1 Flash drops 22 points. All figures are as reported by Artificial Analysis.

Artificial Analysis releases AA-Omniscience v1.1: 3,000 refreshed questions and a calibrated 300-question public set, with Gemini 4 Carbon leading at a Net Score of 54

Key Takeaways

  • The question set shrinks from 6,000 to 3,000 across 6 domains and 42 topics: 600 each for science/engineering/math and software engineering, 450 each for finance, health, humanities/social sciences and law
  • A new 300-question public set is on Hugging Face, with 0.99 correlation to private-set Net Score and a median difference of 0.0 points
  • Net Score ranking: Gemini 4 Carbon (high) 54, Claude Opus 5.5 48, Claude Fable 5.1 46; the best open-weights model, Muse Spark Open, trails Gemini 4 Carbon by 28 points
  • Claude Fable 5.1 has the highest accuracy (68%), followed by Claude Opus 5.5 (66%); Gemini 4 Argon has the lowest hallucination rate (4%)
  • Rank order keeps a 0.98 Spearman correlation with v1, but scores are 1.9 points lower on average; MiniMax M3.1 Flash falls from 16 to -6 and MiMo V2.6 Pro from 8 to -5 (vendor-reported)
Read full story

Google DeepMind expands Gemini-built PlanAssist to 15 English councils after pilots cut planning assessment time by up to 58%

On October 31, Google DeepMind said PlanAssist, a Gemini-built planning tool developed with the UK government, Google Cloud and Faculty, will be made available to 12 more local planning authorities in England, bringing the total to 15 for a 12-month Beta on live applications. In early pilots at Barnet, Camden and Dorset, DeepMind says the average officer time on householder assessment tasks fell by 58%, and more than 70% of the drafted reports were approved with no edits. Planning officers keep the final decision.

Google DeepMind expands Gemini-built PlanAssist to 15 English councils after pilots cut planning assessment time by up to 58%

Key Takeaways

  • Scale: from 3 pilot councils to 15; the 12 new ones join in three cohorts for a 12-month Beta (DeepMind blog)
  • Efficiency: average officer time on householder assessment tasks fell 58% in pilots (vendor-reported; DeepMind's X post says "up to 58%")
  • Quality: more than 70% of AI-drafted reports were approved by officers with no edits (vendor-reported)
  • The UK government says, together with new rules, routine householder processing can halve from eight weeks to four, and it aims to offer the tool to every council in England in 2027 (GOV.UK)
Read full story

Benji Taylor, head of design at X and SpaceXSI, open-sources Stageshow, an agent skill for realistic faux product UI and animated demos

On Oct 31, Benji Taylor, head of design at X and SpaceXSI, released Stageshow on X: an open-source skill (MIT license) for coding agents that helps them produce faux product UI that looks real, including stills, cursor demo animations, and bento grids, delivered as ordinary React components. It installs with one command, and he says it works with Grok Build, Claude Code, Codex, Gemini, and others. The launch post drew about 4,500 likes and 6,500 bookmarks. He notes it rarely nails a demo in one shot and most still take a few rounds of feedback.

Benji Taylor, head of design at X and SpaceXSI, open-sources Stageshow, an agent skill for realistic faux product UI and animated demos

Key Takeaways

  • Install: npx skills add benjitaylor/skills --skill stageshow; the author says it works with Grok Build, Claude Code, Codex, Gemini, and other agents (author's post)
  • Contents: a seven-step workflow in SKILL.md, reference rule docs, dozens of gold examples, a React kit, and self-check scripts such as film.mjs (GitHub)
  • License: the stageshow folder is MIT licensed; the repo was created Oct 26 and had about 61 stars at writing time
  • Reach: the launch post had about 4,521 likes, 6,561 bookmarks, and 213k views as of Nov 1, 08:00 SGT
  • Quality claims are the author's own: the same prompts did better with the skill, but no quantitative evaluation was published
Read full story
ADSponsored

Kaggle and IBM Research launch the ScarfBench Java migration leaderboard: Gemini 4 Argon with Antigravity leads at 75.2%, and about 72% of failures happen at runtime

Kaggle announced ScarfBench with IBM Research, a 204-task benchmark that tests whether coding agents can migrate enterprise Java apps between Jakarta EE, Quarkus and Spring without changing behavior. IBM's October 27 technical note shows Gemini 4 Argon with Antigravity first on the smoke score at 75.2%, Claude Opus 5.5 with Claude Code highest on all-tests-pass at 72.1%, and GPT-6.1 Sol with Codex cheapest at $1.25 per solved task.

Kaggle and IBM Research launch the ScarfBench Java migration leaderboard: Gemini 4 Argon with Antigravity leads at 75.2%, and about 72% of failures happen at runtime

Key Takeaways

  • 204 migration tasks and 10 model-plus-harness configurations; top smoke scores are Gemini 4 Argon + Antigravity 75.2%, Claude Opus 5.5 + Claude Code 70.5% and Claude Opus 5 + Claude Code 69.8% (IBM figures)
  • Claude Opus 5.5 + Claude Code has the highest all-tests-pass rate at 72.1%, just ahead of Gemini 4 Argon at 71.1%
  • Compiling is mostly solved: 8 of 10 configurations compiled at least 97% of solvable tasks, yet smoke scores spanned more than 50 points
  • 535 of 745 classified failing runs (about 72%) were runtime behavioral failures, including 210 lost-route/404 failures
  • Cost per solved task: $1.25 for GPT-6.1 Sol + Codex, $1.59 for Gemini 3.8 Flash + mini-swe and $10.10 for Claude Opus 5.5 + Claude Code; public submissions are not open yet
Read full story

Perplexity CEO shares early Gemini 4 Carbon evals: a record 0.857 on the WANDR deep-research benchmark at $5.86 per task, ahead of Argon and GPT-6 Astra

Perplexity CEO Aravind Srinivas posted early evals of Gemini 4 Carbon on X: it scored a record 0.857 on Perplexity's Wide-And-Deep-Research (WANDR) long-form research benchmark at $5.86 per task, versus 0.802 ($7.88) for Gemini 4 Argon and 0.735 ($13.34) for GPT-6 Astra. He called Carbon the strongest model Perplexity has seen for long-form research and answer quality, and said it will come to all Pro and Max users inside Perplexity Computer once there is enough token capacity.

Perplexity CEO shares early Gemini 4 Carbon evals: a record 0.857 on the WANDR deep-research benchmark at $5.86 per task, ahead of Argon and GPT-6 Astra

Key Takeaways

  • WANDR scores (per Perplexity): Gemini 4 Carbon 0.857, Gemini 4 Argon 0.802, GPT-6 Astra 0.735, GPT-6.1 Sol 0.654, Claude Opus 5.5 0.647, Claude Fable 5.1 0.601
  • Cheaper too: Carbon costs $5.86 per task, below Argon at $7.88 and GPT-6 Astra at $13.34
  • Cost frontier in Perplexity's chart: DeepSeek V4 Pro 0813 (0.359, $0.75), GPT-6 Sol (0.566, $2.23), GPT-6.1 Sol (0.654, $3.70) and Carbon
  • Claude Sonnet 5.5 scored 0.550 ($2.73) and Grok 4.7 scored 0.528 ($9.30)
  • Not live yet: Carbon reaches Perplexity Pro and Max users only once token capacity allows, and Google still limits early access to cyber defenders in its Fairwind program
Read full story

micro1 releases Dual-Arm Sim, an operating-room robot benchmark: 10 frontier models finished only 11 of 800 trials, and all 213 "task complete" claims were false

AI data company micro1 released Dual-Arm Sim, which has 10 general-purpose frontier models control a two-arm Franka Panda robot in a simulated operating room built on NVIDIA Isaac Sim, using only camera images and robot sensors, across four tasks: swapping instruments, replacing a faulty drug cartridge, passing a sample between hands, and packing a surgical kit. Only 11 of 800 trials were completed, all by Gemini 4 Argon (9) and GPT-6 Astra (2). Models claimed "task complete" in 213 trials and were wrong every time. All 200 traces of the cartridge-swap task are public on Hugging Face.

micro1 releases Dual-Arm Sim, an operating-room robot benchmark: 10 frontier models finished only 11 of 800 trials, and all 213 "task complete" claims were false

Key Takeaways

  • Only 11 of 800 trials finished: Gemini 4 Argon 9 (6/20 cartridge swaps plus 3 surgical kits) and GPT-6 Astra 2; no model completed the instrument swap or the two-arm sample transfer (per micro1)
  • All 213 "task complete" claims were false, and in 94 of them the robot never touched the object; Gemini 3.1 Pro made the claim in 49 of 80 trials
  • Thinking longer did not help: Argon used only 315 tokens per decision and still led; its 9 successes cost $119 in total, while Claude Fable 5.1 spent $839 for none
  • Recovery decided success: Argon and Astra regained a real grasp after a miss in about two-thirds of trials, Grok 4.7 in 3 of 80; first position estimates were a median 33 cm off
  • Lifted faulty cartridges ended on the floor 14 times and on the trolley 11 times, versus only 8 correct swaps
Read full story

Cognition scores Gemini 4 Carbon at 40% on FrontierCode 2.0 for $4.04 a task, near the top, while Gemini 4 Argon gets 26.3% at $6.41

Cognition, the company behind Devin, posted on X at 01:45 UTC on Oct 30 that Google's new Gemini 4 Carbon scores 40% on its FrontierCode 2.0 coding benchmark at $4.04 per task, near the top and behind only Claude Opus 5.5, GPT-6 Astra and Claude Sonnet 5.5. The earlier Gemini 4 Argon scores 26.3% at a higher $6.41 per task. The results are Cognition's own runs, and the leaderboard on its site has been updated.

Cognition scores Gemini 4 Carbon at 40% on FrontierCode 2.0 for $4.04 a task, near the top, while Gemini 4 Argon gets 26.3% at $6.41

Key Takeaways

  • Gemini 4 Carbon scores 40% on FrontierCode 2.0 at $4.04 per task, behind Claude Opus 5.5 (45.5%), GPT-6 Astra (42.8%) and Claude Sonnet 5.5 (post)
  • Gemini 4 Argon scores 26.3% at $6.41 per task, ahead of Muse Wonder 1.0 but behind Claude Haiku 5.5 and Grok 4.7
  • From Argon to Carbon the score rises about 13.7 points while cost per task drops about $2.37
  • The changelog shows Gemini 4 Carbon was added on Oct 28; FrontierCode 2.0 has 132 tasks scored on whether a maintainer would merge the PR (leaderboard)
  • All figures are Cognition's own runs; Google has not published FrontierCode results itself
Read full story

Google Gemini Notebook adds hundreds of free textbooks from OpenStax, S Chand and more as cited sources

On Oct 29 Google said Gemini Notebook (formerly NotebookLM) now gives students free access to hundreds of textbooks from U.S. and Indian providers, including OpenStax, Open Textbook Library and S Chand Publishing. Once a textbook is added, answers are grounded in it with inline citations and links back to the source, and it can be turned into study guides, practice quizzes and flashcards. The catalog sits under Add sources, then Textbooks.

Google Gemini Notebook adds hundreds of free textbooks from OpenStax, S Chand and more as cited sources

Key Takeaways

  • Google's blog says hundreds of textbooks from U.S. and Indian providers are available at no charge; Google VP Josh Woodward put the number at over 700 in his own X post
  • Launch providers include OpenStax, Open Textbook Library and S Chand Publishing
  • Once a textbook is added, every answer is grounded in it, with inline citations and direct links back to the source
  • It extends Expert Intelligence, launched Aug 27 with more than 100,000 purchasable Google Play Books titles
  • The official @Gemini_Notebook post had about 185K views, over 5,000 likes and over 2,800 bookmarks when this was written
Read full story
Linked Models & Agents:
⚡ Gemini🌐 Google

Google previews Connected Apps Platform: one MCP or Android AppFunctions integration for Gemini and Search

On Oct 28 Google launched a preview of its Connected Apps Platform. Developers integrate through a cloud-hosted MCP server or Android MCP (Jetpack AppFunctions), then register, test and submit for review in Google Connected Console, choosing whether the app goes live in Search, the Gemini app, or both. Access is phased and currently limited to US developers who apply.

Google previews Connected Apps Platform: one MCP or Android AppFunctions integration for Gemini and Search

Key Takeaways

  • Two integration paths: a cloud-hosted remote MCP server (authorized with OAuth 2.1), or Jetpack AppFunctions on Android (Android MCP), where tools run on the device
  • One console: Google Connected Console handles tool registration, testing, review and publishing, and shows traffic, latency and error rates once live
  • Distribution choice: before review, pick Search, the Gemini app, or both
  • Today: more than 50 Connected Apps across Search and Gemini; Adobe, Experian, Localiza and Peloton are live in Gemini and come to Search in the coming weeks, and Calm launches on both
  • Limits: phased preview for US developers who apply; Google has not said when it opens broadly
Read full story
ADSponsored

Alphabet Q3 2026: revenue up 25% to $127.6B, Google Cloud up 103%, first-party model APIs near 32B tokens per minute

After the US market close on Oct 28, Alphabet reported Q3 2026 results: revenue rose 25% to $127.6B, and Google Cloud revenue more than doubled, up 103% to $30.7B, which the company attributes to enterprise AI infrastructure, AI solutions and TPU system sales. In the release and on X, Sundar Pichai said first-party model APIs now process nearly 32B tokens per minute, more than four times a year ago, and that the Gemini app has reached one billion users. The cost: Q3 capital expenditures of $51.4B and free cash flow of negative $9.1B.

Alphabet Q3 2026: revenue up 25% to $127.6B, Google Cloud up 103%, first-party model APIs near 32B tokens per minute

Key Takeaways

  • Revenue of $127.58B, up 25% year over year and the 13th straight quarter of double-digit growth; Pichai says Alphabet is on a $500B annual revenue run rate
  • Google Cloud revenue of $30.71B versus $15.16B a year ago, up 103%; Cloud operating income of $11.12B versus $3.59B
  • First-party model APIs process nearly 32B tokens per minute, more than 4x a year ago; Pichai's X post puts quarter-over-quarter growth at 40%
  • The Gemini app reached one billion users, which Google calls the fastest-growing product in its history; Search query growth is the highest in five years
  • Q3 capex of $51.36B versus $23.95B a year ago, and free cash flow of negative $9.07B, the second negative quarter in a row
Read full story

Google releases Gemini 4 Carbon two weeks after Argon: 59 and first place on the Artificial Analysis Intelligence Index v5, state-of-the-art cybersecurity and long-horizon coding, with early access for cyber defenders in the Fairwind program

Google DeepMind announced Gemini 4 Carbon on Oct 28, calling it its strongest model, just two weeks after Gemini 4 Argon. The official blog says it ranks first on v5.0 of the Artificial Analysis Intelligence Index at 59 and leads three cybersecurity evaluations: 88.2% on SEC-bench Pro for finding vulnerabilities, 54.8% on ExploitGym for exploitation and 71.0% on CWE-bench for patching; it also gains 89 points to rank fourth on WebDev Arena at 1769. Access starts this week for a subset of cyber defenders in the Fairwind program, with developers, businesses and consumers to follow. Introductory pricing matches Argon at $2 per million input tokens and $10 per million output tokens, rising to $4 and $20 afterwards.

Google releases Gemini 4 Carbon two weeks after Argon: 59 and first place on the Artificial Analysis Intelligence Index v5, state-of-the-art cybersecurity and long-horizon coding, with early access for cyber defenders in the Fairwind program

Key Takeaways

  • Positioning: Google calls it its strongest model, aimed at cybersecurity and long-horizon agentic work; internal teams saw it run for days to root-cause and patch subtle bugs across large codebases and low-level systems
  • Official scores (evaluated without guardrails): 88.2% on SEC-bench Pro, 54.8% on ExploitGym and 71.0% on Collinear's CWE-bench; 59 and first place on the Artificial Analysis Intelligence Index v5.0; +89 to fourth place on WebDev Arena at 1769
  • Safety and alignment: successful cheating on Terminal-Bench 4.0 fell to 0.0% with a 0.4% attempt rate, high-severity misalignment alerts in internal live coding dropped by almost 80%, unsupported claims are about half of Argon's, and the Gray Swan indirect prompt injection attack success rate fell from Argon's 0.7% to 0.0%
  • Internal examples: more than 2x Argon's success rate at generating secure code, and the Chrome team reports more than 10x better high-quality patches than another lab's large frontier model
  • Rollout: a subset of cyber defenders in the Fairwind program first, alongside the U.S. government's voluntary pre-release testing; no developer API yet; introductory pricing of $2 / $10 per million tokens with cached input 95% off, then $4 / $20
Read full story

Cognition launches FrontierCode 2.0 coding benchmark: it asks whether a PR should be merged, and top model Claude Opus 5.5 scores just 45.5%

Cognition, the maker of Devin, released FrontierCode 2.0 on Oct 27. It keeps the method of maintainer-written tasks graded on whether the code is actually mergeable, but draws tasks from more repositories, adds security patches and performance work, uses deliberately underspecified prompts and adds many thoroughness criteria. In Cognition's results Claude Opus 5.5 leads at 45.5%, GPT-6 Astra follows at 42.8%, and Grok 4.7 and Gemini 4 Argon drop into the 20s, leaving the benchmark far from saturated.

Cognition launches FrontierCode 2.0 coding benchmark: it asks whether a PR should be merged, and top model Claude Opus 5.5 scores just 45.5%

Key Takeaways

  • Claude Opus 5.5 leads at 45.5% (54.6% on v1.1) and GPT-6 Astra scores 42.8% (53.3% on v1.1), per Cognition (vendor-reported)
  • Grok 4.7 scores 28.6%, Gemini 4 Argon 26.3%, Muse Wonder 1.0 24.3%, SWE-2 23.9%, DeepSeek V4.1 Flash 19.3% and Kimi K3 17.7%, showing a wide gap for open models
  • Versus v1.1, reference solutions touch 67% more files and 89% more lines in codebases 25% larger; tasks come from 55% more repositories with 43% more blocking criteria
  • Thoroughness blockers grew from 168 to 501, about 3x; in 80% of tasks the reference solution improves on the upstream PR, and an audit agent found and reported an upstream security vulnerability
  • On the cost-score Pareto frontier, Opus 5.5 sits at the high end and Luna, Haiku and Sol at the low-cost end, with GPT-6.1 Sol called out for strong value (official blog)
Read full story

Google DeepMind unveils Genie 4 world model: built on Gemini Omni, first with native real-time audio and dialogue, in research preview for trusted testers

Google DeepMind announced Genie 4 on Oct 27, calling it its most realistic and interactive world model yet. Built on Gemini Omni, it improves physics and human motion, natively generates ambient audio and interactive character dialogue for the first time, and uses Gemini 3.8 Flash in the loop to plan complex events. It is a research preview for trusted testers for now, coming soon to Google AI Ultra (Project Genie) and early access partners in the Gemini API and AI Studio; no quantitative benchmarks were published.

Google DeepMind unveils Genie 4 world model: built on Gemini Omni, first with native real-time audio and dialogue, in research preview for trusted testers

Key Takeaways

  • Per Google DeepMind, Genie 4 is its most realistic and interactive world model yet, with Gemini Omni anchoring real-world appearance and behavior
  • It is DeepMind's first real-time world model with native audio: ambient sound, music and character dialogue are generated with the world, and characters respond live
  • Gemini 3.8 Flash acts as a guiding agent in the loop, splitting goals into steps and checking frames; a demo pairs Gemini 4 Argon with Genie 4 to simulate an Apollo humanoid
  • Official limitations: only a few minutes of continuous interaction and drift over longer sessions; no quantitative benchmarks were published
  • Trusted testers only for now; coming soon to Google AI Ultra (Project Genie) and Gemini API / AI Studio early access partners, with SynthID on all outputs
Read full story
Linked Models & Agents:

Google brings Gemini Robotics ER 2, DeepMind's embodied reasoning model, to Google Cloud in public preview so enterprises can orchestrate VLA models and multi-robot teams

On Oct 27 Google Cloud announced that Gemini Robotics ER 2, Google DeepMind's embodied reasoning model, is in public preview on the Gemini platform via Model Garden, under the model ID gemini-robotics-er-2-preview and built on Gemini 3.5 Flash. It acts as a robot's high-level brain: it reasons over space and time in video, plans multi-step tasks, checks whether an action succeeded, and calls VLA models and navigation APIs as tools, including across teams of different robots. Google says it pinpoints key video frames with 91.3% accuracy.

Key Takeaways

  • Model ID gemini-robotics-er-2-preview, built on Gemini 3.5 Flash, takes text, image, video and audio input, with a 131,072-token input and 65,536-token output limit
  • The core pattern is declaring VLA models, navigation APIs or custom functions as tools, with ER 2 planning and orchestrating them while explaining itself and accepting redirection in natural language
  • It classifies task progress as binary, discrete or continuous scores; Google cites 91.3% accuracy and a 0.96-second mean frame distance for finding key moments in video
  • It reads values from more than 10 instrument types, including dials, sight glasses, digital displays and rulers
  • The enterprise version adds IAM, security controls and function calling; the Cloud preview supports GenerateContent but not the Live API, and Boston Dynamics and Arm are among early users
Read full story
Linked Models & Agents: