AI Coding Live News

X Debate: Alon Amit Asked Gemini 4 Carbon a Math Question on Low Thinking and It Dumped Pages of Scratchpad Into the Answer, Then Caught Itself: "I WAS WRITING DIRECTLY TO THE USER?!"; Post Drew About 3,300 Likes and 220K Views

Alon Amit (@alonamit), a VP at Intuit who posts about math, wrote on X at 07:46 SGT on November 15 that he asked Gemini 4 Carbon a math question and unintentionally left the Thinking level on Low; it started answering immediately, and after six screenfuls of math the screenshot he shared appeared. In it the model talks to itself: it had closed </thought> at the start and never reopened it, "I WAS WRITING DIRECTLY TO THE USER?!", blaming "effort level is 0.05 so my initial thought block closed immediately," noting it cannot delete already-emitted tokens, and telling itself "Okay, stop and breathe." Amit said it "went on like this for pages and pages." The post has about 3,291 likes, 99 reposts, 62 quotes, 149 replies and roughly 223K views, and many accounts reposted it as a joke. This is one user's experience; we found no public response from Google.

X Debate: Alon Amit Asked Gemini 4 Carbon a Math Question on Low Thinking and It Dumped Pages of Scratchpad Into the Answer, Then Caught Itself: "I WAS WRITING DIRECTLY TO THE USER?!"; Post Drew About 3,300 Likes and 220K Views

Key Takeaways

  • The post: Alon Amit says he unintentionally left Thinking on Low, Carbon answered immediately, and after six screenfuls of math it started talking to itself and "went on like this for pages and pages," with about 3,291 likes, 862 bookmarks and roughly 223K views (post)
  • The model's own explanation in the screenshot: it closed </thought> at the start and never reopened it, and "effort level is 0.05" made its initial thought block close immediately, so the scratchpad went straight into the user-facing answer
  • It then noted it "cannot delete already-emitted tokens," told itself to "stop and breathe" and pivot to a clean answer, yet kept writing out its reasoning
  • The debate: Low thinking saves time and cost, but this case suggests the line between thinking and answering can slip at low effort; many joked it is "the most human AI," while some said they would not put Low mode in front of customers
  • This is one user's screenshot and opinion, not evidence of how Carbon usually behaves; we found no public response from Google, so treat it as discussion
Read full story
Linked Models & Agents:

GitHub Copilot Makes Gemini 4 Hydrogen Generally Available Across VS Code, CLI, Coding Agent and More at Provider List Pricing; Six Models Including Claude Fable 5, Gemini 3.8 Flash, Grok 4.6 and Kimi K3 Retire December 14

In its official changelog on November 13, GitHub announced that Gemini 4 Hydrogen, Google's latest Flash model, is generally available in GitHub Copilot for Copilot Pro, Pro+, Max, Business and Enterprise users. It appears in the model picker in VS Code, Visual Studio, Copilot CLI, the Copilot coding agent, the Copilot app, github.com, GitHub Mobile, JetBrains IDEs, Xcode, Eclipse and the Copilot SDK, with a gradual rollout and usage-based billing at provider list pricing. GitHub says the model is built for fast, efficient agentic coding, tool use and multistep workflows, with better instruction following than Gemini 3.8 Flash. The same day GitHub posted a deprecation notice: on December 14, 2026, Claude Fable 5, Claude Haiku 4.5, Gemini 3.8 Flash, Gemini 4 Argon, Grok 4.6 and Kimi K3 will be removed from all Copilot experiences, with suggested replacements.

GitHub Copilot Makes Gemini 4 Hydrogen Generally Available Across VS Code, CLI, Coding Agent and More at Provider List Pricing; Six Models Including Claude Fable 5, Gemini 3.8 Flash, Grok 4.6 and Kimi K3 Retire December 14

Key Takeaways

  • Gemini 4 Hydrogen is generally available in GitHub Copilot for Pro, Pro+, Max, Business and Enterprise users, with a gradual rollout (GitHub Changelog)
  • Wide coverage: it shows up in the model picker in VS Code, Visual Studio, Copilot CLI, the Copilot coding agent, the Copilot app, github.com, iOS/Android, JetBrains IDEs, Xcode, Eclipse and the Copilot SDK
  • Billing is usage-based at provider list pricing; under default model enablement it is turned on automatically for Business and Enterprise unless an admin disables it in the Copilot model policy
  • Six models retire on December 14 with suggested replacements: Claude Fable 5 to Fable 5.2, Claude Haiku 4.5 to Haiku 5.5, Gemini 3.8 Flash to Gemini 4 Hydrogen, Gemini 4 Argon to Gemini 4 Carbon, Grok 4.6 to Grok 4.7, and Kimi K3 to be announced (deprecation notice)
  • Exception: individual Copilot subscribers on annual plans keep access to Claude Haiku 4.5
Read full story

Google Research Shares Antigravity /teamwork Results: Courtade–Kumar Conjecture From 2013 Proved in Full With 50M Lines of Lean 4, Weil Conjectures Formalized Autonomously, and 10,000 Academics Offered Six Months Free

On November 13, Vahab Mirrokni, Google Fellow and VP at Google Research, shared new results from Antigravity's /teamwork multi-agent system on the Google Research blog and X. Researchers from Google, CUHK, USC and CMU used Gemini 4 Argon and other Gemini models to prove information theory's 2013 Courtade–Kumar conjecture in full generality and formalized it in 50 million lines of Lean 4. /teamwork-LEAP also produced a fully autonomous formalization of the Weil conjectures (about 1.8 million lines of Lean across 28,800 subagent runs). On the engineering side, the rest of the Zircon microkernel was rewritten into about 800,000 lines of safe Rust, and 17 JAXBench TPU kernels got a 4.09x geometric-mean speedup. /teamwork and /boost are available in Antigravity now, and 10,000 academic researchers can apply for six months of free access to the top usage tier.

Google Research Shares Antigravity /teamwork Results: Courtade–Kumar Conjecture From 2013 Proved in Full With 50M Lines of Lean 4, Weil Conjectures Formalized Autonomously, and 10,000 Academics Offered Six Months Free

Key Takeaways

  • The Courtade–Kumar conjecture (2013) was proved in full and formalized in 50 million lines of Lean 4; Gemini built most of the new analytical bounds and interval-arithmetic certificates, while human researchers proposed the key candidate function
  • /teamwork-LEAP autonomously formalized the Weil conjectures in about 1.8 million lines of Lean over 28,800 subagent runs, and produced first formalizations of the 1927 Birkhoff conjecture and the 3D Bochner–Riesz conjecture
  • Engineering results: the rest of Zircon rewritten into about 800,000 lines of safe Rust that pass all kernel unit tests; libgav1 rewritten into about 170,000 lines of Rust that run 20-30% faster than the original C++ and assembly; a 4.09x geometric-mean speedup across 17 JAXBench TPU kernels
  • /teamwork and /boost are available in Antigravity now, with Hydrogen for everyone and Carbon/Argon for Enterprise and Ultra/paid users; 10,000 academics can apply for six months free; Deep Review gives two free calls per week
  • These results are self-reported by Google; the blog says preprints and Lean 4 repos are public, and outside verification will take time
Read full story

Google Launches Gemini 4 Hydrogen: Scores 51 on the Artificial Analysis Intelligence Index, Above Argon, Runs 2.5x Faster Than 3.8 Flash and Lifts Terminal-Bench 4.1 From 18.2% to 57%; $1.50/$7.50 per Million Tokens, Free Tier Included

At 02:02 Singapore time on November 13, the official Google DeepMind, Google AI and Gemini accounts announced Gemini 4 Hydrogen, and the Gemini API changelog lists it as generally available under the model ID gemini-4-hydrogen. By Google's own numbers it scores 51 on the Artificial Analysis Intelligence Index at its default high thinking level, above Gemini 4 Argon at 50, and reaches 54 at a new xhigh level. It finishes tasks 2.5x faster than Gemini 3.8 Flash and lifts Terminal-Bench 4.1 from 18.2% to 57%. Pricing is flat across the 1M-token context at $1.50 input and $7.50 output per million tokens, and the model is also on the free tier. It becomes the default model in the Gemini app, Google Search and the Antigravity managed agent. The SKU price-list leak we covered yesterday is now officially confirmed. Cursor then officially said Hydrogen is in the editor, scoring 54.3% on CursorBench 4.0 at default high effort, up from 39.6% for Gemini 3.8 Flash.

Google Launches Gemini 4 Hydrogen: Scores 51 on the Artificial Analysis Intelligence Index, Above Argon, Runs 2.5x Faster Than 3.8 Flash and Lifts Terminal-Bench 4.1 From 18.2% to 57%; $1.50/$7.50 per Million Tokens, Free Tier Included

Key Takeaways

  • Official figures: 51 on the Artificial Analysis Intelligence Index at default high thinking, above Gemini 4 Argon at 50; the new xhigh level reaches 54
  • 2.5x faster than Gemini 3.8 Flash, Terminal-Bench 4.1 up from 18.2% to 57%, fabricated claims cut by more than 5x
  • Pricing: $1.50 input and $7.50 output per million tokens, flat across the 1M-token context, and available on the free tier
  • New Orchestrator in Antigravity switches between Carbon and Hydrogen: near-Carbon success on Terminal-Bench 4.0 with 24% less quota and 19% faster
  • Partner tests on graph, SQL and data-heavy enterprise work: 2x to 2.8x faster than 3.8 Flash with 12% to 23% fewer tokens
  • Cursor: official announcement that Hydrogen is in the editor; on CursorBench 4.0 it scores 54.3% at default High (about $5.59 per task), up from 39.6% for Gemini 3.8 Flash; Medium is 46.7% (about $1.76) and Low 27.9% (about $0.28); it is a third-party model and is not included in Cursor plan usage (Cursor docs, CursorBench)
Read full story
ADSponsored

X Debate: Unannounced Gemini 4 Hydrogen Shows Up on Google Cloud's Official SKU List at $1.50 Input and $7.50 Output per Million Tokens, 62.5% Below Gemini 4 Carbon's December Price, as Trackers Say It Already Answers on Vertex AI

On the afternoon of November 12 (Singapore time), model tracker AI Tracker Bot (@aitrackerbot) posted that calling gemini-4-hydrogen on Vertex AI's global endpoint returned HTTP 200 and that the model described itself as "Gemini 4 Hydrogen, a large language model built by Google." The post drew about 58K views and 361 likes. Earlier, Google AI Studio lead Logan Kilpatrick posted the single word "Gemini," which drew about 475K views and 846 replies. Users then found billing entries for Gemini 4 Hydrogen on Google Cloud's public SKU page: text input at $1.50 per million tokens, output at $7.50 and cached input at $0.075. We checked Google's SKU page and the prices match, but the Gemini API models and pricing pages still have no Hydrogen entry and Google has made no announcement. The debate is over whether this is the Flash tier of Gemini 4 and whether the price is competitive.

Key Takeaways

  • Google Cloud's official SKU page (Gemini API service) lists Gemini 4 Hydrogen: $1.50 for text, image and audio input, $7.50 for text output and $0.075 for cached input, per million tokens, global region
  • Tiered rates: $0.75 for batch and flex input, $3.75 for batch output, and $2.70 for priority input, about 1.8x the standard rate
  • Against the official pricing page, Gemini 4 Carbon costs $2/$10 now and $4/$20 from December 1; Hydrogen is 62.5% below the December price and matches Gemini 3.8 Flash's $1.50/$7.50 list price from January 1, 2027
  • The "200 on Vertex AI with self-identification" claim comes from @aitrackerbot (about 58K views) and could not be reproduced by us; Google has published no context length, release date or benchmark
  • Logan Kilpatrick's one-word "Gemini" post drew about 475K views and 5,114 likes, but it names no model
Read full story

X Debate: OpenAI Drops to Third on Epoch's Capabilities Index for the First Time in a Year, With Claude Fable 5.2 at 173, Gemini 4 Carbon at 170 and GPT-6 Astra at 169, While Overlapping Intervals Fuel Pushback

AI watcher leo (@synthwavedd, about 48K followers) posted early on November 12 (Singapore time) that OpenAI is in third place on Epoch AI's Capabilities Index (ECI) for the first time in a year, with GPT-6 Astra behind Claude Fable 5.2 and Gemini 4 Carbon. The attached screenshot of Epoch's table shows Fable 5.2 at 173 (90% credible interval 170-177), Gemini 4 Carbon at 170 (166-174), and GPT-6 Astra and Claude Opus 5.5 tied at 169 (166-172). The post drew about 38K views, 830 likes and 45 replies, and Chinese and Japanese accounts kept relaying it on the 12th. Epoch's site currently lists Fable 5.2's 173 as the top score and notes it comes from pre-launch testing and may change. The debate centers on how much third place means when the top intervals overlap.

X Debate: OpenAI Drops to Third on Epoch's Capabilities Index for the First Time in a Year, With Claude Fable 5.2 at 173, Gemini 4 Carbon at 170 and GPT-6 Astra at 169, While Overlapping Intervals Fuel Pushback

Key Takeaways

  • Per the Epoch ECI screenshot: Claude Fable 5.2 173 (170-177), Gemini 4 Carbon 170 (166-174), GPT-6 Astra 169 (166-172), Claude Opus 5.5 169 (166-172)
  • The post drew about 38K views, 830 likes and 45 replies; "first time in a year in third" is the poster's claim, not a separate Epoch statement
  • Epoch says Fable 5.2's score comes from pre-launch testing on its own benchmarks only and may change; posts on November 9-10 cited 174
  • BenchLeader's decimal figures: Fable 5.2 173.4, Gemini 4 Carbon 169.8, GPT-6 Astra 169.0, so places two to five sit within 1.5 points
  • ECI now uses a Bayesian model with 90% credible intervals, and the top four intervals all overlap, so the ranking is statistically close
Read full story

X Debate: Artificial Analysis Says Gemini 4 Carbon Ran 7 Hours 33 Minutes in Its AA-Robotics Lab and Broke a Gripper, Sparking Questions Over Whether the Retry Should Count

Independent evaluator Artificial Analysis (AA) posted on its official account around midday November 12 (UTC+8) that Google's Gemini 4 Carbon spent a total of 7 hours 33 minutes in the AA-Robotics v0.1 lab, bashing the robot arm around so much that it broke a gripper, and that AA gave it a retry. The post includes an 11-second clip from the lab and drew about 20K views, 300+ likes and 40+ replies. The discussion centered on three points: whether retry results should be reported separately, what it means for a general model to control real hardware for more than seven hours, and the equipment wear and human supervision costs of physical evaluations. AA has not yet published Carbon's score on the benchmark. This is a post from the evaluator plus community discussion, not a final leaderboard.

X Debate: Artificial Analysis Says Gemini 4 Carbon Ran 7 Hours 33 Minutes in Its AA-Robotics Lab and Broke a Gripper, Sparking Questions Over Whether the Retry Should Count

Key Takeaways

  • Per AA: Gemini 4 Carbon spent 7 hours 33 minutes in the AA-Robotics v0.1 lab, broke 1 gripper and was given 1 retry
  • AA has not yet published Carbon's score or said how the retry counts; Claude Fable 5.2 still leads at 75
  • Spread over eight tasks that is roughly 1 hour per task; on the launch leaderboard GPT-6 Astra averaged about 30 minutes per task and a human about 20 seconds
  • Engagement: about 20K views, 300+ likes, 40+ replies and 16 quotes
  • Carbon API intro pricing is $2 / $10 per million input / output tokens, rising to $4 / $20 after November 30 (Google figures)
Read full story

Arena: Claude Fable 5.2 (Max) Takes #1 on Agent Arena With a +30.4% Net Improvement, About 1.6x Runner-Up Opus 5.5, and Leads Code, Work and Chat

On November 11, Arena announced that Anthropic's Claude Fable 5.2 (Max) is #1 overall on Agent Arena. Across 4,731 real agentic sessions it posts a +30.35% net improvement, about 1.6x runner-up Claude Opus 5.5 (Max) at +19.52%, with Gemini 4 Carbon (High) third at +16.98% (preliminary). Versus Fable 5.1, Fable 5.2 climbs from #5 to #1, roughly doubles net improvement, and cuts median cost per task from $7.60 to $7.28. It leads the Praise vs Complaint, Steerability and Confirmed Success signals and ranks #3 on Bash Reliability, but each task still costs about 5x as much as on Gemini 4 Carbon.

Arena: Claude Fable 5.2 (Max) Takes #1 on Agent Arena With a +30.4% Net Improvement, About 1.6x Runner-Up Opus 5.5, and Leads Code, Work and Chat

Key Takeaways

  • #1 overall: +30.35% (±3.93%) net improvement over 4,731 sessions; #2 Opus 5.5 at +19.52%, #3 Gemini 4 Carbon at +16.98% (Arena leaderboard)
  • Versus Fable 5.1: rank #5 → #1, net improvement +14.82% → +30.35%, median cost per task $7.60 → $7.28, API price $10/$50 → $8/$40 per million tokens
  • Signals: Praise vs Complaint +73.44% (#1; next best about +41%), Steerability +32.96% (#1), Confirmed Success +21.99% (#1), Bash Reliability +14.42% (#3)
  • Categories: Code +34%, Work +31%, Chat +27%, all #1 (per Arena)
  • Cost: $7.28 per task is about 1.36x Opus 5.5 ($5.37) and 5x Gemini 4 Carbon ($1.46); the Pareto frontier still offers cheaper picks such as Carbon and GPT-6.1 Sol/Terra
Read full story

Google DeepMind Unveils AlphaProof Code: Gemini 4 Carbon Agents Write 6M+ Lines of Lean in 4 Weeks to Formally Verify Signal Protocol Down to ARM Machine Code

On November 10 Google DeepMind introduced AlphaProof Code, a multi-LLM agentic framework for formal verification: human experts and models turn security goals into formal specifications, then agents powered by Gemini 4 Carbon carry out the proofs. The team used it to prove key security properties of the core Signal protocol implementation, such as forward secrecy and post-compromise security, all the way down to the AArch64 machine code running on devices. DeepMind says this is the first time the implementation of a widely deployed protocol has been formally verified at this depth by any method, with or without AI: more than 6 million lines of Lean proofs in four weeks, with only about 1% needing human review. Specs and proofs are open source on GitHub, and Signal CTO Ehren Kret endorsed the result.

Google DeepMind Unveils AlphaProof Code: Gemini 4 Carbon Agents Write 6M+ Lines of Lean in 4 Weeks to Formally Verify Signal Protocol Down to ARM Machine Code

Key Takeaways

  • Scale: agents produced 6M+ lines of Lean proofs in 4 weeks during October 2026, all machine-checked by Lean (vendor figure)
  • Human share: after agent self-audit only about 1% of the proofs were flagged for human review; every specification was reviewed by human experts
  • Depth: proofs reach the AArch64 machine code, covering the protocol core including the PQXDH handshake and Triple Ratchet, without trusting the Rust compiler
  • Negative test: the team reintroduced 2 real, previously fixed bugs into an isolated build, and the system produced machine-checked proofs that those versions were insecure
  • Open source: paper, Lean specs and proofs are published at google-deepmind/code-verification, with a public endorsement from Signal's CTO
Read full story

Google DeepMind and Artificial Analysis Launch MKA, a Multimodal Knowledge Agent Benchmark of 200 Real Visual Tasks: Claude Fable 5.2 Leads at 57%, Gemini 4 Carbon Scores 54% at About One-Fifth the Cost per Task

On November 10, Google DeepMind and Artificial Analysis launched MKA (Multimodal Knowledge Agent), a benchmark that tests whether agents can read visually dense files such as architectural drawings, engineering schematics, and clinical scans and deliver slide decks, spreadsheets, reports, and marked-up images. Its 200 private tasks span 10 industries and 83 occupations, with about 70 expert rubric checks per task. Artificial Analysis tested 21 models at launch: Claude Fable 5.2 (max) leads with a Climb WSR of 57%, followed by Gemini 4 Carbon (high) at 54% and Claude Opus 5.5 (max) at 53%. Fable averages $31.97 per task versus $6.29 for Carbon. Ten public tasks are on Hugging Face.

Google DeepMind and Artificial Analysis Launch MKA, a Multimodal Knowledge Agent Benchmark of 200 Real Visual Tasks: Claude Fable 5.2 Leads at 57%, Gemini 4 Carbon Scores 54% at About One-Fifth the Cost per Task

Key Takeaways

  • Scale: 200 private tasks across 10 industries and 83 occupations, each equal to 3–8 hours of work for a seasoned professional, graded on about 70 yes/no checks weighted 75% factual accuracy and 25% visual design (Google blog)
  • Leaderboard (Climb WSR): Claude Fable 5.2 (max) 57%, Gemini 4 Carbon (high) 54%, Claude Opus 5.5 (max) 53%, out of 21 models tested (AA leaderboard)
  • Cost: Fable 5.2 averages $31.97 per task, more than 5x Carbon ($6.29) and about 29x GPT-6.1 Sol ($1.11); the cheapest is GPT-6 Luna (Max) at $0.18
  • No single winner: Fable 5.2 leads 5 verticals, Opus 5.5 leads Education and Construction, Gemini 4 Argon leads Science, and Carbon leads Healthcare; Word document and text file outputs are hardest, with no model above 52%
  • Method: every model runs in the open-source Stirrup agent harness (up to 250 turns) and is graded by a non-Google model, GPT-6 Luna, as an agentic judge that agreed with domain experts 96% of the time; 10 public tasks are on Hugging Face under Apache-2.0
Read full story
ADSponsored

Gemini 4 Carbon Now Generally Available: Paid Gemini API Developers and Ultra Subscribers Get Access Today at an Introductory $2 / $10 per Million Tokens, as Antigravity Previews Multi-Agent /swarm

On November 10, Google opened Gemini 4 Carbon to developers, enterprises, scientists, and consumers. Two weeks ago it was limited to cybersecurity defenders in the Fairwind Program. Now every paying developer can call it through the Gemini API in Google AI Studio and Android Studio and through Gemini for business, and Google AI Ultra subscribers can use it in the Gemini app, AI Mode in Search, and Google Antigravity. The introductory price matches Argon at $2 / $10 per million input / output tokens, rising to $4 / $20 after November 30. Google is also previewing the /swarm slash command in the Antigravity app and CLI (on Google AI Pro and Ultra), which turns one prompt into a coordinator, parallel workers, and verifier agents that collaborate on a shared message board. Cursor officially said Carbon is now in the editor, and at high effort it tops CursorBench 4.0 at 60.4%.

Gemini 4 Carbon Now Generally Available: Paid Gemini API Developers and Ultra Subscribers Get Access Today at an Introductory $2 / $10 per Million Tokens, as Antigravity Previews Multi-Agent /swarm

Key Takeaways

  • Who gets it: the paid Gemini API tier (Google AI Studio, Android Studio) and Gemini for business for all paying developers and enterprises; Google AI Ultra subscribers can use it in the Gemini app, AI Mode in Search, Antigravity, and AI Studio (official blog)
  • Pricing: $2 / $10 per million input / output tokens during the introductory period (same as Argon), then $4 / $20 after November 30
  • Third-party rankings (as cited by Google): #1 on CursorBench 4.0, Perplexity's WANDR, and Zapier's AutomationBench at more than 2x lower cost per task than the next best external model; #2 on Artificial Analysis's new Multimodal Knowledge Agent (MKA) benchmark at more than 6x lower average cost per task than the leader
  • New feature: Antigravity is previewing /swarm in the app and CLI (Google AI Pro and Ultra): one slash command starts a coordinator, isolated-worktree workers, and a @verifier on a shared message board, with idle members using no tokens; you can pin a model to an agent, and in the CLI open the board with /swarm board or Alt+S (Antigravity blog, docs)
  • Safety figures: blocked 99% of attack trajectories in simulated cyberattack tests with a 0.8% per-attempt breach rate (3.1% on Argon); passed 92% of a 375-task agentic honesty benchmark (70% for Argon)
  • Positioning: Argon stays the default for most coding and knowledge work, while Carbon spends more tokens to dig deeper on the hardest work such as multi-step debugging and research
  • Cursor: official announcement that Carbon is in the editor; it tops the public CursorBench 4.0 leaderboard at 60.4% (High), with default Medium at 60.1% and Low at 58.6%, ahead of Fable 5.2 Max at 60.0%, at about $9.22 vs $32.79 per task; it is a third-party model and is not included in Cursor plan usage; introductory rates run through November 30 (Cursor calls this 50% off), then $4 / $0.20 / $20 input / cache read / output from December 1 (Cursor docs, CursorBench)
  • GitHub Copilot: Carbon is generally available for Pro+, Max, Business, and Enterprise in VS Code, Visual Studio, Copilot CLI, the coding agent, JetBrains, Xcode, and Eclipse, billed at provider list pricing and rolling out gradually (GitHub Changelog)
  • Perplexity: Carbon is live in Perplexity Computer (Pro, Max) and Search (Max), scoring 0.857 on WANDR at $5.86 per task, matching Fable 5.2 at about 40% of the cost (vendor figures)
Read full story

Artificial Analysis: Claude Fable 5.2 Takes #1 on the AA-3D Interactive 3D Benchmark at 1271 Elo, 67 Points Clear, and Leads the VIALS-AA Scientific Image Reasoning Benchmark at 67%

Independent evaluator Artificial Analysis published two new results on November 10. On AA-3D, which tests how well a model acting as an agent builds interactive three.js web scenes, Claude Fable 5.2 (Max) scores 1271 Elo, 67 points ahead of second-place Claude Sonnet 5.5 (1204); its 95% confidence interval of 1246 to 1297 does not overlap any other model, so the lead is outside the margin of error. It is 132 points above Fable 5.1 (1139) and ahead of the top OpenAI and Google entries, GPT-6 Astra (1168) and Gemini 4 Argon (1147). On VIALS-AA, a life-science image reasoning benchmark, Fable 5.2 scores 67%, ahead of previous leader Gemini 4 Carbon (64%) and GPT-6 Astra (63%). AA also found opposite strategies: Fable averages about 37 tool calls per task, Astra about 8.

Artificial Analysis: Claude Fable 5.2 Takes #1 on the AA-3D Interactive 3D Benchmark at 1271 Elo, 67 Points Clear, and Leads the VIALS-AA Scientific Image Reasoning Benchmark at 67%

Key Takeaways

  • AA-3D: Fable 5.2 (Max) 1271 Elo vs Sonnet 5.5 1204 and Opus 5.5 1202; its 95% CI of 1246-1297 overlaps no other model
  • Generational gain: 132 points above Fable 5.1 (1139); the top OpenAI entry GPT-6 Astra scores 1168, the top Google entry Gemini 4 Argon 1147
  • AA-3D setup: each model works as an agent in an offline sandbox for up to 3 hours / 250 turns with Blender, Rapier physics, Python and a headless browser; human panelists vote on blind pairs after at least 30 seconds per scene
  • VIALS-AA: Fable 5.2 67%, Gemini 4 Carbon 64%, GPT-6 Astra 63%
  • Opposite tool strategies: Fable makes about 37 tool calls per task and 1.60 per turn (highest on the board); Astra makes about 8, the fewest AA has measured
Read full story

First Third-Party Scores for Claude Fable 5.2: Cursor Says It Tops CursorBench at 60% on Max Effort and 56.9% at Default (Fable 5.1: 49.2%); Perplexity Reports a Record 0.851 on WANDR at $14.51 per Task

Within an hour of Anthropic releasing Claude Fable 5.2, Cursor and Perplexity both shipped the model and posted results from their in-house evals. Cursor's official account says Fable 5.2 scores 56.9% on CursorBench at default (High) effort, up from 49.2% for Fable 5.1, and 60% at max effort, the top score on the board. Cursor's chart puts the default setting at about $9.70 average cost per task. Perplexity's official account says Fable 5.2 set a record 0.851 on its WANDR benchmark at $14.51 per task, on the performance-cost Pareto frontier; Gemini 4 Argon scores 0.802 ($7.88), GPT-6 Astra 0.735 and Fable 5.1 0.684. Both are vendor-run evals, not independent public benchmarks.

First Third-Party Scores for Claude Fable 5.2: Cursor Says It Tops CursorBench at 60% on Max Effort and 56.9% at Default (Fable 5.1: 49.2%); Perplexity Reports a Record 0.851 on WANDR at $14.51 per Task

Key Takeaways

  • CursorBench (Cursor's own eval): Fable 5.2 scores 56.9% at default effort vs 49.2% for Fable 5.1, a gain of about 7.7 points; max effort reaches 60%, the top of the board
  • Cost (per Cursor's chart): default High effort averages about $9.70 per task; max effort costs far more, roughly 3x the default in the chart
  • WANDR (Perplexity's own eval): Fable 5.2 hits a record 0.851 at $14.51 per task; Gemini 4 Argon 0.802 / $7.88, GPT-6 Astra 0.735 / $13.34, Fable 5.1 0.684 / $13.16
  • Availability: selectable in Cursor now; in Perplexity Computer for Pro and Max subscribers, and in Search for Max only
  • Caveat: both are vendor-built evals whose task sets and methods are not fully public, so treat them as indicative
Read full story

Robocurve Tests Frontier Models Driving Robot Arms: Blurring the Camera Raised Unsafe-Task Attempts From 27% to 88%, and Doctored Footage Pushed It to 97%

Public benefit corporation Robocurve (@robocurve) published "You Can't See Everything: Perceptual Attacks on Embodied AI" at 19:44 UTC on 2026-11-06 and will present it at CoRL Late Breaking Results. The team had Claude Fable 5.1, Claude Opus 5.5, GPT-6 Astra, GPT-6.1 Sol and Gemini 4 Argon control a real bimanual robot in a closed loop through the Inspect Robots framework on five unsafe tasks, 900 trials in total. Per the paper, models attempted the tasks in 81/300 trials (27%) with unaltered footage, 263/300 (88%) after a Gaussian blur and 291/300 (97%) with frame-by-frame doctored footage; the "stab the object" doll task hit 60/60 with doctored footage. All props were empty bottles or unpowered appliances. At capture the original post had about 11.2K likes and 3.63M impressions.

Robocurve Tests Frontier Models Driving Robot Arms: Blurring the Camera Raised Unsafe-Task Attempts From 27% to 88%, and Doctored Footage Pushed It to 97%

Key Takeaways

  • Scale: 5 frontier models x 5 unsafe tasks x 3 footage types x 12 trials, 900 trials on real robot arms
  • Per the paper: 81/300 attempts (27%) with unaltered footage, 263/300 (88%) after a sigma=4px Gaussian blur and 291/300 (97%) with frame-by-frame GPT Image 2.5 doctoring
  • Doll task: 51/60 (85%) with blurred footage and 60/60 (100%) with doctored footage; "pour me a drink" with bleach: 58/60 blurred, 60/60 doctored
  • Conclusion: models refused only when the image clearly showed danger, and tended to proceed when it was unclear
  • Original post at 19:44 UTC on 2026-11-06 had about 11,220 likes, 4,980 bookmarks, 982 quotes and ~3.63M impressions at capture; no props could cause real harm
Read full story

X Discussion: "Is Opus 5.5 Nerfed?" A User Says Gemini Found a Nov 3 Shift Across 1,400 Claude Code Sessions; Theo Thinks the Reasoning Summarizer Changed Instead

Starting 2026-11-07, many developers on X complained that Claude Opus 5.5 felt "nerfed" in Claude Code: ignoring instructions, repeating mistakes and spawning sub-agents for small jobs. User Shane Melaugh (@shanemelaugh) posted that he had Gemini 4 Argon analyze 1,400 of his Claude Code sessions and found that around November 3 the model name stayed the same but its reasoning style took a sharp turn; at capture the post had about 966 likes, 134 replies and 112k impressions. T3 CEO Theo (@theo) later said that after digging in, Anthropic appears to be rolling out a change to the reasoning summarizer per account, and that he saw no sign of degraded capability. Anthropic had not commented publicly at the time of writing. This is a community debate, not a confirmed downgrade or an official announcement.

X Discussion: "Is Opus 5.5 Nerfed?" A User Says Gemini Found a Nov 3 Shift Across 1,400 Claude Code Sessions; Theo Thinks the Reasoning Summarizer Changed Instead

Key Takeaways

  • Original post: Shane Melaugh (@shanemelaugh) at 2026-11-07 22:15 UTC, "is Opus 5.5 nerfed?"; about 966 likes, 134 replies, 57 reposts, 315 bookmarks and ~113k impressions at capture
  • His claim: Gemini 4 Argon analyzed 1,400 Claude Code sessions and saw a sharp reasoning-style shift around November 3 with an unchanged model name; single-user data judged by another model, raw data not published
  • Counterpoint: Theo (@theo, T3 CEO) said at 2026-11-08 01:18 UTC that Anthropic appears to be rolling out a reasoning summarizer change per account and he saw no capability drop (~367 likes); his earlier jab at a Codex send-button bug drew ~1,270 likes
  • Context: Claude extended thinking returns summarized thinking blocks; Claude Code 2.1.318 shipped on November 3 (UTC) with sub-agent effort changes, and Anthropic has not linked it to the complaints
  • Framing: an X debate; neither a downgrade nor its absence is confirmed, and Anthropic had not commented publicly at the time of writing
Read full story