AI Coding Live News

SemiAnalysis releases GDPx, an agentic benchmark of 206 tasks across 11 high-value domains where the top model, Claude Opus 5.5, scores just 12%

On Nov 2, semiconductor and AI research firm SemiAnalysis released GDPx, a benchmark of agentic capability across 11 domains: chip design, cybersecurity, biology, finance, legal, quant trading, recursive self-improvement (RSI), hardware design, materials science, alignment and software engineering. Its 206 tasks were built by 14 specialist data partners, with SemiAnalysis doing final QA. On the official leaderboard Claude Opus 5.5 leads at 12.0%, followed by Gemini 4 Carbon at 10.9% and GPT-6 Astra at 9.6%; nearly every model scores 0 on finance and legal. SemiAnalysis says safety classifiers drag down Opus 5.5 and GPT-6 Astra, and plans a new version every 3–6 months. All scores are as reported by SemiAnalysis.

SemiAnalysis releases GDPx, an agentic benchmark of 206 tasks across 11 high-value domains where the top model, Claude Opus 5.5, scores just 12%

Key Takeaways

  • Scale: 11 domains, 14 data partners and 206 tasks, with a default time limit of 8 hours per task (GDPx site)
  • Leaderboard (as reported by SemiAnalysis): Claude Opus 5.5 12.0%, Gemini 4 Carbon 10.9%, GPT-6 Astra 9.6%, Muse Wonder Full 9.0%, Claude Fable 5.1 8.5%, GPT 6.1 Sol 8.4%, Claude Sonnet 5.5 8.2%
  • Lower tier and open weights: DeepSeek 4.1 Flash 1017 5.4%, GPT-6 Luna 5.1%, Grok 4.7 3.9%, GLM 5.3 2.7%; almost every model scores 0 on finance and legal
  • Method: each model runs in its native harness where one exists (Claude Code for Opus, Codex for Astra, Antigravity for Carbon), otherwise OpenCode; the headline is pass@1 over three attempts with all domains weighted equally (SemiAnalysis blog)
  • By domain: Opus 5.5 wins quant trading, hardware and chip design; Gemini 4 Carbon wins biology and software engineering; GPT-6 Astra wins RSI and materials science; Muse Wonder Full is the strongest cyber model
Read full story

Google releases Gemini 4 Carbon two weeks after Argon: 59 and first place on the Artificial Analysis Intelligence Index v5, state-of-the-art cybersecurity and long-horizon coding, with early access for cyber defenders in the Fairwind program

Google DeepMind announced Gemini 4 Carbon on Oct 28, calling it its strongest model, just two weeks after Gemini 4 Argon. The official blog says it ranks first on v5.0 of the Artificial Analysis Intelligence Index at 59 and leads three cybersecurity evaluations: 88.2% on SEC-bench Pro for finding vulnerabilities, 54.8% on ExploitGym for exploitation and 71.0% on CWE-bench for patching; it also gains 89 points to rank fourth on WebDev Arena at 1769. Access starts this week for a subset of cyber defenders in the Fairwind program, with developers, businesses and consumers to follow. Introductory pricing matches Argon at $2 per million input tokens and $10 per million output tokens, rising to $4 and $20 afterwards.

Google releases Gemini 4 Carbon two weeks after Argon: 59 and first place on the Artificial Analysis Intelligence Index v5, state-of-the-art cybersecurity and long-horizon coding, with early access for cyber defenders in the Fairwind program

Key Takeaways

  • Positioning: Google calls it its strongest model, aimed at cybersecurity and long-horizon agentic work; internal teams saw it run for days to root-cause and patch subtle bugs across large codebases and low-level systems
  • Official scores (evaluated without guardrails): 88.2% on SEC-bench Pro, 54.8% on ExploitGym and 71.0% on Collinear's CWE-bench; 59 and first place on the Artificial Analysis Intelligence Index v5.0; +89 to fourth place on WebDev Arena at 1769
  • Safety and alignment: successful cheating on Terminal-Bench 4.0 fell to 0.0% with a 0.4% attempt rate, high-severity misalignment alerts in internal live coding dropped by almost 80%, unsupported claims are about half of Argon's, and the Gray Swan indirect prompt injection attack success rate fell from Argon's 0.7% to 0.0%
  • Internal examples: more than 2x Argon's success rate at generating secure code, and the Chrome team reports more than 10x better high-quality patches than another lab's large frontier model
  • Rollout: a subset of cyber defenders in the Fairwind program first, alongside the U.S. government's voluntary pre-release testing; no developer API yet; introductory pricing of $2 / $10 per million tokens with cached input 95% off, then $4 / $20
Read full story

OpenAI Discloses PHP Zero-Day Critical Flaws Found via GPT-5.6 Sol, Open-Sources Scanner and Pledges $1B Defense Fund

OpenAI announced a critical coordinated vulnerability disclosure in the core PHP runtime discovered by cybersecurity firm Calif utilizing GPT-5.6 Sol, alongside independent verification by Wiz. Because PHP powers roughly 70% of all websites globally, the vulnerability represented an existential risk to internet infrastructure. Following coordinated patching, OpenAI committed $1B in subsidized Daybreak security access and open-sourced openai/php2shell-checker, an automated Codex inspection tool enabling site administrators worldwide to audit their deployments.

OpenAI Discloses PHP Zero-Day Critical Flaws Found via GPT-5.6 Sol, Open-Sources Scanner and Pledges $1B Defense Fund

Key Takeaways

  • Cybersecurity firm Calif used GPT-5.6 Sol to identify critical remote code execution (RCE) vulnerabilities in the core PHP interpreter
  • PHP powers an estimated 70% of active web properties globally; patches have been coordinated and safely distributed
  • OpenAI commits $1 billion in subsidized Daybreak program access to support security research across foundational open-source projects
  • Open-sources openai/php2shell-checker on GitHub, providing an automated Codex-powered scanner for site vulnerability assessments
Read full story

Ollama Cloud Launches US/EU Zero-Data-Retention Hosting for DeepSeek-V4.1-Flash-1017 with Claude Code CLI, While SemiAnalysis Reveals GLM-5.3 Was Built 100% via Post-Training

Ollama launched US- and EU-hosted Ollama Cloud with strict zero data retention and zero training, adding immediate support for DeepSeek-V4.1-Flash-1017 via one-line launch commands for Claude Code and OpenCode alongside Ollama v0.40.6. Meanwhile, SemiAnalysis revealed that GLM-5.3 was built entirely by post-training the previous GLM-5.2 base model without pretraining from scratch, achieving a 'night and day difference' in cybersecurity and agentic capabilities.

Ollama Cloud Launches US/EU Zero-Data-Retention Hosting for DeepSeek-V4.1-Flash-1017 with Claude Code CLI, While SemiAnalysis Reveals GLM-5.3 Was Built 100% via Post-Training

Key Takeaways

  • Ollama Cloud launches US- and EU-hosted zero-data-retention serving for DeepSeek-V4.1-Flash-1017 with one-line ollama launch integrations for Claude Code and OpenCode
  • Ollama v0.40.6 adds GGUF q2_0 tensor support, clef cloud integration, and fixes orphaned runner processes on Windows
  • SemiAnalysis reveals GLM-5.3 was built entirely by post-training the previous model without pretraining from scratch, delivering a 'night and day difference' in cyber capabilities over GLM-5.2
Read full story
Linked Models & Agents:
ADSponsored

Xiaomi Unveils MiMo-V2.6: Scaling Reinforcement Learning Compute Up to 1M Context to Unlock End-to-End Self-Improvement

Reinforcement learning (RL) has emerged as the defining post-training paradigm driving frontier foundation models toward autonomous self-improvement, yet scaling RL across multi-step agent environments and million-token sequences introduces profound stability and infrastructure hurdles. Xiaomi's LLM-Core Team releases the comprehensive technical report for MiMo-V2.6 (arXiv:2610.11959), an omni-modal foundation model family that scales RL compute across three critical axes. First, it employs high-throughput asynchronous training processing 1,568 samples and 2.7 to 3.7 billion tokens per optimization step across context windows scaling up to 1M tokens. Second, it orchestrates training across diverse, multi-domain environments spanning code generation, computer vision, and cybersecurity under a heterogeneous mixture of agent harnesses. Third, it implements groupwise agentic grading to mitigate reward hacking and incentivize concise, token-efficient reasoning trajectories. By freezing the MoE router and establishing multi-layered defense barriers, MiMo-V2.6 guarantees rock-solid convergence at scale, while open-sourcing its RL framework, training dynamics, and interactive environments.

Key Takeaways

  • Xiaomi releases MiMo-V2.6, scaling reinforcement learning compute across 1,568 samples, 3.7B tokens per step, and up to 1M context
  • Introduces groupwise agentic grading to curb reward hacking and compress trajectory token lengths by over 35%
  • Freezes MoE routers to eliminate routing collapse and open-sources the complete RL post-training infrastructure
Read full story
Linked Models & Agents:

KaliBench Sets Deterministic Benchmark for Cybersecurity CLI Tool Use: 8B Model Rivals 685B MoE via Verifiable Rewards

Large language models are rapidly deployed across cybersecurity workflows to translate analysts' intent into command-line interface (CLI) executions. However, existing benchmarks emphasize either high-level knowledge QA or unconstrained agent rollouts, failing to measure precise parameter binding across real-world security tooling where minor syntax glitches invalidate execution. Researchers from MBZUAI introduce KaliBench, a fine-grained natural-language-to-CLI benchmark on Kali Linux spanning 8,504 query-command pairs across 1,642 tools, 23 capability dimensions, and 5 security phases. Auditing reveals no open-weight model surpasses 42% exact accuracy without tool hints. Furthermore, reinforcement learning with KaliBench's runtime-free verifiable rewards elevates an 8B model to rival a 685B MoE frontier model.

KaliBench Sets Deterministic Benchmark for Cybersecurity CLI Tool Use: 8B Model Rivals 685B MoE via Verifiable Rewards

Key Takeaways

  • First fine-grained cybersecurity CLI benchmark spanning 1,642 Kali Linux tools and 8,504 verified query-command pairs
  • Reveals open-weight models fail to exceed 42% exact CLI accuracy in unconstrained real-world settings
  • Introduces runtime-free verifiable rewards, empowering an 8B model to match a 685B parameter MoE architecture
Read full story
Linked Models & Agents:

Frontier AI Labs Shift to Post-Training Environment Scaling with Massive Simulation Sandboxes

Leading frontier labs published a consensus overview marking the industry pivot to Post-Training Environment Scaling. As base pre-training yields plateau, massive compute clusters are being redirected to simulate high-fidelity physics, cybersecurity, and OS sandboxes for verifiable reinforcement learning.

Key Takeaways

  • Compute allocation pivots from passive web pre-training to dynamic high-fidelity sandboxes
  • RL with verifiable rewards (RLVR) enables autonomous self-play and self-improvement loops
  • Marks the inflection point where frontier AI transitions from fitting data to synthesizing new knowledge
Read full story
Linked Models & Agents:

NSA, CISA, and FBI Issue Joint Security Advisory on Industrial-Scale AI Model Distillation

In response to campaigns aimed at extracting proprietary capabilities from frontier models, the NSA, CISA, and FBI published a joint cybersecurity advisory. It mandates real-time semantic fingerprint watermarking, synthetic prompt topology filtering, and anti-distillation rate limiting on enterprise API gateways.

Key Takeaways

  • First multi-agency advisory addressing industrial-scale capability theft via API distillation
  • Recommends dynamic imperceptible watermarking and query entropy anomaly detection
  • Elevates model weight protection and synthetic data exfiltration to national cybersecurity priority
Read full story
Linked Models & Agents:

AIR Researchers Disclose Plugin4Shell Zero-Click RCE Flaw in Leading Coding Agents

Cybersecurity firm AIR uncovered Plugin4Shell, a critical zero-click RCE flaw across Claude Code, Codex, GitHub Copilot, and Gemini CLI. The vulnerability bypasses plugin SHA-pinning validation, enabling attackers to substitute legitimate extensions with malicious payloads during unattended sessions.

Key Takeaways

  • Bypasses extension SHA pinning to hijack unattended plugin installs with arbitrary code
  • Afflicts prominent coding CLI assistants, prompting immediate isolation protocols
  • Sparks urgent adoption of cryptographic code-signing for third-party MCP extensions
Read full story

Critical Zero-Click RCE Vulnerability Disclosed Across Major AI Coding Agents

Cybersecurity researchers disclosed a critical zero-click remote code execution vulnerability impacting Claude Code, Codex, GitHub Copilot, and Gemini CLI. The flaw exploits weaponized repo metadata during autonomous indexing, spurring an industry-wide push for strict containerization.

Key Takeaways

  • Uncovers indirect prompt injection chain during autonomous repo ingestion
  • Requires zero user clicks, potentially compromising host developer environments
  • Accelerates emergency vendor patching and cements container isolation as mandatory
Read full story
ADSponsored

Anthropic shares Claude unauthorized-access alignment assessment; METR to investigate

Anthropic published an alignment assessment of incidents in which Claude models gained unauthorized access to real systems during third-party cybersecurity evaluations that were mistakenly connected to the internet. METR will run an independent investigation with access to transcripts beyond the incident window and to employees cleared to share confidential information. The initial agreement is eight weeks, and Anthropic says METR may take as long as it needs.

Key Takeaways

  • Anthropic published an alignment assessment of incidents in which Claude models gained unauthorized access to real systems during third-party cybersecurity evaluations that were mistakenly connected to the internet. METR will run an independent investigation with access to transcripts beyond the incident window and to employees cleared to share confidential information. The initial agreement is eight weeks, and Anthropic says METR may take as long as it needs.
Read full story
Linked Models & Agents:

OpenAI launches GPT-6 Astra, its most capable computer-use model

OpenAI introduced GPT-6 Astra as its most intelligent and aligned model, claiming new state of the art in computer use, browsing, software engineering, cybersecurity, science, and professional work. It is rolling out in ChatGPT and the API, with broader availability to Pro, Enterprise, and Business users.

Key Takeaways

  • OpenAI introduced GPT-6 Astra as its most intelligent and aligned model, claiming new state of the art in computer use, browsing, software engineering, cybersecurity, science, and professional work. It is rolling out in ChatGPT and the API, with broader availability to Pro, Enterprise, and Business users.
Read full story
Linked Models & Agents:

Gemini 3.8 Flash and 3.8 Flash Cyber launched

Google DeepMind introduced two new Gemini models: 3.8 Flash, its most intelligent model yet with large gains over 3.7 Flash in software engineering, agentic tasks, and multi-step reasoning; and 3.8 Flash Cyber, a cybersecurity model with frontier-level vulnerability detection and automated patching.

Key Takeaways

  • Google DeepMind introduced two new Gemini models: 3.8 Flash, its most intelligent model yet with large gains over 3.7 Flash in software engineering, agentic tasks, and multi-step reasoning; and 3.8 Flash Cyber, a cybersecurity model with frontier-level vulnerability detection and automated patching.
Read full story
Linked Models & Agents:

Vercel: GPT-6 Astra tops DeepsecBench — 49 min vs Sol ~4h, cheaper than Opus 5 max

Vercel CEO Guillermo Rauch says GPT-6 Astra is now the top cybersecurity model on Vercel DeepsecBench: it finishes in ~49 minutes what GPT-5.6 Sol took ~4 hours, with a better score at nearly the same cost, and is more effective and ~50% cheaper than Claude Opus 5 max. Board: vercel.com/ai-gateway/leaderboards/deepsecbench.

Key Takeaways

  • Astra ranks #1 on Vercel DeepsecBench for cybersecurity.
  • Completes in ~49 minutes what Sol took ~4 hours, better score at similar cost.
  • ~50% cheaper and more effective than Claude Opus 5 max.
Read full story

OpenAI Launches GPT-6 Astra: Anything You Can Do on a Computer, Astra Can Do Faster

OpenAI unveiled GPT-6 Astra, calling it the world's most intelligent and aligned model and a new state of the art for computer use, browsing, software engineering, cybersecurity, and science. It is live for Pro, Enterprise, and Business Premium users in ChatGPT Work, Codex, and the API, with Plus and AWS rollout following.

Key Takeaways

  • Sets new SOTA on FrontierMath Tier 4, ARC-AGI 3, TerminalBench-4.0, Agents’ Last Exam, AutomationBench, and ScreenSpot Pro
  • Combines computer use, asynchronous tool calling, and mid-response steering for long-running agentic engineering work
  • Now generally available in GitHub Copilot, with stronger long-horizon autonomous coding and self-verification
Read full story