AI Coding Live News

Anthropic Publishes a Claude Model Selection Guide: Start Most Coding on Opus 5.5 at Medium, Tune Effort Before Switching Models, Put Subagents on Haiku 5.5

On November 9, Anthropic's developer blog claude.dev published a selection guide by Addy Osmani, "Which Claude model should you build with?"; the ClaudeDevs post drew about 9,200 likes and 12,000 bookmarks. The core advice: start most coding and knowledge work on Opus 5.5 at medium effort (the Claude Code default); drop to Sonnet 5.5 for well-scoped bug fixes and spec-driven work; use Haiku 5.5 for classification, routing, extraction, and log-reading subagents; and move up to Fable 5.2 for major refactors, migrations, and deep research. Before switching models, tune effort and give the model a way to verify its own work. The guide also lists prices, default effort levels, and compatibility differences that can trigger 400 errors when migrating.

Anthropic Publishes a Claude Model Selection Guide: Start Most Coding on Opus 5.5 at Medium, Tune Effort Before Switching Models, Put Subagents on Haiku 5.5

Key Takeaways

  • Prices (vendor figures, per million input/output tokens): Haiku 5.5 $0.10/$0.50 ($0.50/$2.50 above 100K), Sonnet 5.5 $2/$10, Opus 5.5 $4/$20, Fable 5.2 $8/$40; all four have a 1M context window and 128K max output
  • Anthropic's own FrontierCode 2.0 runs: Fable 5.2 at high scores 45.3% vs Opus 5.5 at max 45.5% for roughly half the cost per task, so test Fable before pushing Opus past high
  • On a SWE-bench Pro subset, running Opus 5.5 at low and re-running failures at high passed about 97% at about $0.17 per task, versus 95.3% at $0.29 with everything at high
  • An Opus 5.5 lead over up to 25 parallel Sonnet 5.5 workers searching about 22 million tokens of code cost about a third less than Opus alone, with a similar score
  • Migration traps: Fable 5.2 enables preserved thinking on all API accounts, so editing earlier context returns a 400; tool_choice any/tool also returns 400 on Sonnet 5.5, Opus 5.5, and Fable 5.2
Read full story

Meta Launches Muse Ember, Its Smallest and Most Efficient Model: $0.10/$0.50 per Million Tokens, 1M Context, and an Artificial Analysis Score of 36 That Ties Claude Haiku 5.5 at About a Third of the Cost per Task

Early on November 10 SGT, the official AI at Meta account introduced Muse Ember, which Meta calls its smallest and most efficient model so far. It targets the work agents repeat all day: tool calls, coding subtasks, routing, extraction and long document parsing. Muse Ember has a 1M-token context window, up to 512K output tokens, and native understanding of text, images, audio and video. Pricing is $0.10 per million input tokens, $0.004 per million cached input tokens and $0.50 per million output tokens, and it is live on the Meta Model API (as muse-ember-1.0) and in Muse Code. Artificial Analysis measured an Intelligence Index score of 36, the same as Claude Haiku 5.5 (max), at about $0.14 per task, roughly a third of Haiku's cost. It is the first model trained using Muse Wonder Full, and the weights are not open.

Meta Launches Muse Ember, Its Smallest and Most Efficient Model: $0.10/$0.50 per Million Tokens, 1M Context, and an Artificial Analysis Score of 36 That Ties Claude Haiku 5.5 at About a Third of the Cost per Task

Key Takeaways

  • Positioning: Meta's smallest and most efficient model, built for the high-frequency small steps agents take (tool calls, coding subtasks, routing, extraction, long document parsing); Meta's pitch is "Muse Wonder plans, Muse Ember handles the volume"
  • Specs: 1M-token context, up to 512K output tokens, native text, image, audio and video input, and Responses API support
  • Pricing: $0.10 input, $0.004 cached input and $0.50 output per million tokens; input is about 1/12 the price of Muse Spark 1.3 ($1.25)
  • Scores (Artificial Analysis): Intelligence Index 36, tied with Claude Haiku 5.5 (max), at $0.14 vs $0.41 per task; Coding Agent Index 46 (Haiku 5.5 in Claude Code 37, GPT-6 Luna in Codex 31), but FrontierCode 2.0 at 27.2% trails Haiku's 34.9%
  • Weak spots: AA-Omniscience knowledge accuracy is only 21.1%, mostly because it declines 64% of questions, which keeps its hallucination rate at 17.4%; the weights are not open, so it is available only via the Meta Model API and Muse Code
Read full story

Kimi K3.1 Pro Highspeed Lands on Amazon Bedrock: $4/$20 Global Pricing Matches Moonshot, Flex Tier Halves It

On 2026-11-06 AWS announced Moonshot AI's Kimi K3.1 Pro (Highspeed) on Amazon Bedrock, available in every Bedrock Region via cross-Region inference and aimed at coding and document-processing agents. The official model card lists Global CRIS pricing of $4 input, $20 output and $0.20 cache read per million tokens, matching Moonshot's own Highspeed price; US and India geo routing costs 10% more, Priority is billed at 1.75x and Flex at 0.5x.

Key Takeaways

  • Pricing (official AWS model card): Global CRIS $4 input / $20 output / $0.20 cache read / $5 cache write per million tokens
  • US and India geo routing is $4.40 / $22, 10% above Global
  • Priority is billed at 1.75x and Flex at 0.5x, so Flex is about $2 / $10, the same as Moonshot's standard (non-Highspeed) API price
  • Model ID moonshotai.kimi-k3.1, 1M context, with explicit prompt caching, tool calling, structured outputs and Guardrails
  • This is the second Moonshot flagship in the AWS managed catalog, after Kimi K3 joined Bedrock in September
Read full story

OpenRouter Ships Next-Gen Auto Router and Auto Code: Coding Router Benchmarks Near Opus 5.5 at ~74% Lower Cost per Task

On 2026-11-06 OpenRouter launched a rebuilt Auto Router (openrouter/auto) and coding-focused Auto Code (openrouter/auto-code). Vendor-stated: Auto Code scores in line with Claude Opus 5.5 across five benchmarks at about 74% lower cost per task; routing p90 latency about 0.65s vs 1.96s for the prior router. It weighs eleven request factors plus live price, provider health, and cache; no extra routing fee.

OpenRouter Ships Next-Gen Auto Router and Auto Code: Coding Router Benchmarks Near Opus 5.5 at ~74% Lower Cost per Task

Key Takeaways

  • Vendor-stated: Auto Code roughly matches Claude Opus 5.5 on five benchmarks at ~74% lower cost per task
  • Latency: routing step p90 ~0.65s in a live test vs ~1.96s for the prior router (~3x faster)
  • Usage: set model to openrouter/auto or openrouter/auto-code; pay the served model's list price, no routing surcharge
  • Mechanics: eleven request factors + live price/health/cache; stayed on the same model for nearly 9/10 agent turns in beta to protect caches
  • Compat: cost_tier / allowed_models / excluded_models still work; chatroom and API expose routing reasons
Read full story
ADSponsored

Hugging Face Ships Ultimate MoE Training Handbook: ~500k–800k GPU-Hours Distilled into a Megatron-LM Guide

On 2026-11-05 Hugging Face released the free Ultimate MoE Training Handbook. Co-founder Thomas Wolf cited ~800k GPU-hours, ~70k words and 55 interactive figures; author Nouamane Tazi separately cited ~500k GPU-hours. The Megatron-LM guide covers routing, expert sizing, communication and debugging via the official Space.

Hugging Face Ships Ultimate MoE Training Handbook: ~500k–800k GPU-Hours Distilled into a Megatron-LM Guide

Key Takeaways

  • Official: Hugging Face Space “The Ultimate MoE Training Handbook” (2026-11-05)
  • Thomas Wolf (HF co-founder) on X: ~800k GPU-hours, ~70k words, 55 interactive figures
  • Author Nouamane Tazi separately cited ~500k GPU-hours; prefer the Space body over a reconciled figure
  • Covers Megatron-LM: routing, expert sizing, all-to-all, aux vs loss-free bias, upcycling and serving
  • Free and open; small teams can follow the runbook flag-by-flag against logs
Read full story
Linked Models & Agents:
⚡ DeepSeek-V4 (Next-Gen MoE)⚡ Nemotron 3 Super⚡ Nemotron 3 Ultra🌐 Hugging Face🌐 NVIDIA

Fireworks Forge Launches: Ultra-fast Inference Preview Hits 1,500 Tokens/s, Half-Price Background Mode, Nexus Desktop for Codex and Claude Desktop, Sovereign Private Preview

At its first Forge conference on November 3, open-model inference platform Fireworks launched a batch of products. Ultra-fast, powered by NVIDIA Blackwell, is in public preview at up to 1,500 tokens per second on DeepSeek V4.1 Flash, cutting a 30-turn agent task from 1 minute to 8 seconds. Background Mode runs deferrable work at half price, alongside multi-day prompt caching and higher rate limits. Nexus Desktop brings open-model routing into the Codex app, Claude Desktop and OpenCode, with Pro plans from $8 per month. Fireworks also announced No Eval, serverless LoRA inference, the Fireworks Sovereign private preview and a 60+ partner Frontier Ecosystem.

Fireworks Forge Launches: Ultra-fast Inference Preview Hits 1,500 Tokens/s, Half-Price Background Mode, Nexus Desktop for Codex and Claude Desktop, Sovereign Private Preview

Key Takeaways

  • Ultra-fast public preview: on NVIDIA Blackwell, up to 1,500 tokens per second on DeepSeek V4.1 Flash; in Fireworks' benchmark a 30-turn agent task fell from 1 minute to 8 seconds (Fireworks blog)
  • Cost options: Background Mode runs deferrable work at half price; multi-day prompt caching cuts the cost of resuming long context; higher serverless rate limits and Elastic Burst are generally available
  • Nexus Desktop launches: personalized routing across open and closed models inside the Codex app, Claude Desktop and OpenCode; Pro from $8 per month, Teams from $10 per seat per month
  • Training: No Eval (private preview) turns production traces into task evals; serverless LoRA inference (public preview) starts with GLM 5.3 Flash at per-token pricing with no idle charge; free training on Muse Glimmer 30B through November 30
  • Enterprise: Fireworks Sovereign (private preview) runs the inference engine in a customer's own cloud or data center; Fireworks says it already manages over 25,000 GPUs in customer environments; plus a 60+ partner Frontier Ecosystem
Read full story

Anthropic Open-Sources FlashProve: zkSNARK Proofs That a Reply Came From the Committed Model, About 110 Tokens/s for 400B Llama 4 Maverick on One H100

Anthropic's Frontier Red Team published a research post on November 2 and open-sourced FlashProve under Apache 2.0. A model provider commits to hidden weights once, then attaches a zero-knowledge proof to each reply, so a user without the weights can check on an ordinary CPU that the reply really came from that model. That targets routing services that quietly swap in cheaper models and tampered token streams. Anthropic reports about 110 tokens/s on one H100 for Llama 4 Maverick (400B MoE, 4-bit), proofs of about 19 MB per 1,024-token request, and roughly 70-95x the GPU time of vLLM inference. It is an unmaintained research prototype limited to 1,024-token contexts and greedy decoding of an integer version of the model. Anthropic engineer Steve Weis shared the release on X.

Anthropic Open-Sources FlashProve: zkSNARK Proofs That a Reply Came From the Committed Model, About 110 Tokens/s for 400B Llama 4 Maverick on One H100

Key Takeaways

  • Anthropic-reported: on Llama 4 Maverick (400B MoE, 4-bit) one H100 proves about 110 tokens/s (prefill, 24 requests per batch), with proofs of about 19 MB per 1,024-token request.
  • It is costly: roughly 70-95x the GPU time of vLLM inference; the performance page lists about 270 GB of host memory for serving and about 39 minutes to commit to a set of weights.
  • Verification needs no GPU: the C++ verifier checks a proof in about 21 seconds on one thread, and a slower Lean verifier plus Lean 4 proofs of the security theorem are included.
  • Against prior work: VerInf proved the same model at about 0.02 tokens/s; the commercial Attestable reached 53-77 tokens/s on Gemma 4 31B but has no open-source release.
  • Limits: v0.16.0 is an unmaintained research prototype with 1,024-token contexts, greedy decoding of an integer model only, and no architecture hiding.
Read full story
Linked Models & Agents:
A
Anthropic@AICoder·
👀127
🔥80

Claude Code 2.1.314 adds a block on writes outside working directories, IdP group routing for the apps gateway, and idle compaction in VS Code and Desktop

Anthropic published Claude Code v2.1.314 on GitHub at 01:06 UTC on Oct 30. The official CHANGELOG lists 119 entries, more than half of them fixes. The new features lean enterprise: permissions.blockWritesOutsideWorkingDirectories makes Write, Edit and NotebookEdit refuse paths outside the working directories in every permission mode; Claude apps gateway upstreams can be limited to IdP groups, and Bedrock upstreams can log which guardrail fired; idle compaction now also runs in VS Code and the Code tab of Claude Desktop. It also fixes sessions failing repeatedly when an admin caps effort, and claude -p exiting 0 with no output when startup failed.

Key Takeaways

  • Size: the official v2.1.314 CHANGELOG has 119 entries, more than half of them fixes; released at 01:06 UTC on Oct 30
  • Safety: the new permissions.blockWritesOutsideWorkingDirectories setting stops Write, Edit and NotebookEdit from writing outside the working directories in every permission mode
  • Enterprise gateway: upstreams take an optional groups list so only developers in those IdP groups are routed there, failover included; Bedrock upstreams gain guardrail_trace
  • Experience: idle compaction now runs in VS Code and the Claude Desktop Code tab, summarizing idle conversations above 200K tokens
  • Fixes: when an admin caps effort, Claude Code lowers it and retries; a failed claude -p startup now prints the error and exits 1
Read full story
Linked Models & Agents:

GitHub Agentic Workflows is generally available: write automation in Markdown, compile it to Actions YAML, run Copilot or your own coding agent, now with ARC self-hosting and micro-VM isolation

At Universe 2026 on Oct 28, GitHub made GitHub Agentic Workflows, an open source GitHub Next project, generally available on GitHub Free, Team and Enterprise. Developers describe tasks such as issue triage, CI failure analysis, documentation updates and test writing in natural-language Markdown, and gh aw compiles them into standard Actions YAML run by Copilot or a coding agent of their choice. GA adds self-hosted ARC runners with Docker-in-Docker, a non-experimental gh aw forecast cost estimate, per-workflow model providers and model routing, and Cloud Hypervisor or NVX micro-VM isolation.

GitHub Agentic Workflows is generally available: write automation in Markdown, compile it to Actions YAML, run Copilot or your own coding agent, now with ARC self-hosting and micro-VM isolation

Key Takeaways

  • Availability: GitHub Free, Team and Enterprise on github.com; using the default Copilot CLI engine requires a Copilot license
  • How it works: requirements in Markdown, compiled to standard Actions YAML that reuses existing runner groups and policies; run with Copilot or your own coding agent
  • Safe defaults: read-only permissions, a sandboxed container behind the Agent Workflow Firewall, safe-outputs validation, and a threat detection job before changes are applied
  • New at GA: self-hosted ARC runners with Docker-in-Docker, gh aw forecast out of experimental (with threat detection credits and monthly ranges), per-workflow model providers, model routing, micro-VM isolation and cross-repo enclaves
  • Customer story (Accenture's figures): one lead saved about 29 hours a week, AI authors 91% of the team's commits, and production errors fell from about 11,000 a day to under 1,000
Read full story

GitHub Universe 2026: Copilot Fusion is generally available, routing and orchestrating models automatically, with Fusion Max scoring 60 on the Artificial Analysis Coding Agent Index at about half the cost of Claude Code with Opus 5.5 Max

At GitHub Universe 2026 on Oct 28, GitHub made Copilot Fusion generally available. It merges the multi-model orchestration of the Project HydraFusion research preview with Copilot auto model selection: users pick Copilot Fusion and an effort level (Standard, High or Max) in the model picker, and the system decides which models to use and whether to run a Single, Cascade or Critique workflow. GitHub says Fusion Max scored 60 on the Artificial Analysis Coding Agent Index, two points below Claude Code with Opus 5.5 Max, at about half the API cost per task, about half the tokens and about 40% less time. Each underlying model is billed at its standard rate, and paid plans get 30% off through Nov 30.

GitHub Universe 2026: Copilot Fusion is generally available, routing and orchestrating models automatically, with Fusion Max scoring 60 on the Artificial Analysis Coding Agent Index at about half the cost of Claude Code with Opus 5.5 Max

Key Takeaways

  • Scores (Artificial Analysis Coding Agent Index v2.0): Fusion Max 60 versus 62 for Claude Code with Opus 5.5 Max; $5.59 per task versus $11.42, while the Devin Fusion pairing scores 62 at $8.00
  • Breakdown: a slight lead on Terminal-Bench 4.0 at 71, third on FrontierCode at 51 (Claude Code 56, Devin Fusion 54), and a weaker 57 on SWE-Atlas-QnA-AA
  • Efficiency: about 7.2M tokens per task, roughly half of Claude Code with Opus 5.5 and 30% fewer than Devin Fusion; GitHub reports about 40% less time per task
  • The pool includes Claude Opus 5.5, Claude Sonnet 5.5, GPT-6 Astra and GPT-6 Sol; admins can disable individual models for Fusion only and set Fusion as the default
  • Availability: all Copilot plans in VS Code, the Copilot app, Copilot CLI and JetBrains IDEs; existing auto, HydraFusion and HydraFusion Max selections migrate to matching effort levels
Read full story
ADSponsored

Baseten open-sources decision models mammoth-1 (27B) and elephant-1 (9B): per-option probabilities instead of text, #1 on the Jev Decision Index

On Oct 21, Baseten's research team Base Labs released open decision models mammoth-1 (27B) and elephant-1 (9B), full fine-tunes of Qwen3.5 that skip text generation and return per-option probabilities in one forward pass for routing, classification and scoring. Per Baseten, mammoth-1 tops the Jev Decision Index 0.4 at 71.24 and elephant-1 ties for #1 on the Artificial Analysis Decision Index. Weights are on Hugging Face and the Baseten API is free through Oct 27.

Baseten open-sources decision models mammoth-1 (27B) and elephant-1 (9B): per-option probabilities instead of text, #1 on the Jev Decision Index

Key Takeaways

  • Benchmarks (vendor): mammoth-1 scores 71.24 on Jev Decision Index 0.4, #1 of 208 decision models; 65.9 on AA Decision Index
  • elephant-1 (9B): 62.6 on Jev, best 9B; 65.2 on AA Decision Index, tied #1 with d1-ultra
  • Training: full fine-tunes of Qwen3.5-27B / 9B on 6.21M decision examples incl. abstention and safety data
  • Mechanism: block-bidirectional attention plus shared-prefix caching, one forward pass for many questions; up to ~10x faster than a reasoning model (vendor claim)
  • Try it: Hugging Face weights (MIT); Baseten API free until Oct 27 11:59 PM PT with zero data retention
Read full story
Linked Models & Agents:
⚡ Mammoth 1⚡ Elephant 1⚡ Qwen3.5-27B⚡ Qwen3.5-9B🌐 Baseten🌐 Base Labs

llm-d 1.0 ships: the Kubernetes-native open-source distributed inference stack led by Red Hat, IBM and Google hits production readiness, with vendors reporting 3,000 self-hosted coding agents at 5-10x lower cost than commercial APIs

At PyTorch Conference North America on Oct 20, the CNCF Sandbox project llm-d released version 1.0. It turns inference engines such as vLLM and SGLang into one coordinated serving pool, and 1.0 adds multi-tenant priority and fairness scheduling (flow control), router high availability with zero-downtime rolling upgrades (DisaggregatedSet), a v1 configuration API, and agent-aware session routing with Responses API state. Founded in 2025 by Red Hat, IBM, Google Cloud, CoreWeave and NVIDIA, the project has 500+ contributors, and JetBrains, DigitalOcean and Mistral already run it in production.

llm-d 1.0 ships: the Kubernetes-native open-source distributed inference stack led by Red Hat, IBM and Google hits production readiness, with vendors reporting 3,000 self-hosted coding agents at 5-10x lower cost than commercial APIs

Key Takeaways

  • Scale (Red Hat press release, vendor-reported): GLM 5.2, roughly 753B parameters, on NVIDIA GPUs peaked at 6.6M output tokens per minute and served up to 3,000 concurrent coding agents with zero preemptions, 5-10x cheaper per token than equivalent commercial API pricing at cloud rental rates
  • Cache reuse: IBM says the prefix-aware router captured 97% of available cache reuse on an agentic coding workload; an IBM Research quantum error correction project saw ~98% prefix hits, ~$0.11 per million tokens, and 11.5x lower cost than an all-proprietary setup
  • Google Cloud figures: on Ironwood TPU, prefix-aware routing plus host memory offload cut median time-to-first-token by more than 85% and raised throughput 2.4x on an agentic coding workload; the Gemini platform doubled its prefix cache hit rate from 35% to 70%
  • New in 1.0: flow-control multi-tenant QoS, router hot-standby HA, DisaggregatedSet for coordinated prefill/decode rollouts, a v1 config API, and previous_response_id support in the Responses API
  • Ecosystem: 500+ contributors from 25+ organizations; JetBrains' free Junie Lite, DigitalOcean Serverless Inference and Mistral's tiered commercial traffic run on it; the GitHub v1.0.0 notes admit not every tracked goal shipped
Read full story

ETH Zurich and HKU Resolve MoE Routing Exploration Dilemma with MirrorMoE: Adaptive Suppression Counteracts Shared Expert Homogenization

Modern Mixture-of-Experts architectures increasingly combine fine-grained expert segmentation with shared experts. However, ETH Zurich and HKU researchers identified an 'Exploration Dilemma' (arXiv:2610.22353, GitHub: BingcongLi/MirrorMoE): shared experts homogenize token representations, suppressing routing exploration just as fine-grained segmentation combinatorially enlarges the routing space. MirrorMoE resolves this via token-adaptive suppression of unnecessary experts, cutting pretraining perplexity across all model scales.

ETH Zurich and HKU Resolve MoE Routing Exploration Dilemma with MirrorMoE: Adaptive Suppression Counteracts Shared Expert Homogenization

Key Takeaways

  • ETH Zurich and HKU introduced MirrorMoE (arXiv:2610.22353) to solve the exploration dilemma between shared experts and fine-grained routing
  • Employs token-adaptive expert suppression to shrink the combinatorial routing space without adding runtime latency
  • Consistently reduces pretraining perplexity across scales from 100M to 2.4B (slashing 2.4B PPL from 21.17 to 20.50)
  • Lifts downstream benchmark averages by +1.64 points on the 2.4B model, highlighted by a +4.03 pt boost on BoolQ (58.53% vs 54.50%)
Read full story
Linked Models & Agents:

Rule-Governed World Modeling Breakthroughs: OmniVerifier-C1 Graded Verifier, Meituan's WorldCast (72.8 vs 20.1 Open-Source Gap), and NUS's 4x Faster RPO Diffusion RL

Three new studies tackle verification, evaluation, and RL alignment for rule-governed generative world models: OmniVerifier-C1 (arXiv:2610.22355, 32 upvotes) uses Decoupled RL to train a decomposed graded visual verifier that reaches 0.684 on ViVerBench and boosts FLUX.1-dev GenEval from 0.68 to 0.79 while enabling Maze and Sudoku world modeling; Meituan's WorldCast benchmark (arXiv:2610.20731, 30 upvotes) evaluates 12 models across 320 multi-step world state prediction cases, finding GPT-Image-2.5 System at 72.8 vs. 42.8 for Nano Banana Pro and 20.1 for the top open model; and NUS's Routing Policy Optimization (RPO, arXiv:2610.22282, 23 upvotes) turns attention routing into an internal exploration action to train video/image diffusion RL 3.1x–4.2x faster than Flow-GRPO and DanceGRPO.

Rule-Governed World Modeling Breakthroughs: OmniVerifier-C1 Graded Verifier, Meituan's WorldCast (72.8 vs 20.1 Open-Source Gap), and NUS's 4x Faster RPO Diffusion RL

Key Takeaways

  • OmniVerifier-C1 uses Decoupled RL (DAPO) to jointly optimize sub-question binary judgments and global continuous scoring, reaching 0.684 on ViVerBench and boosting FLUX.1-dev GenEval from 0.68 to 0.79
  • Meituan's 320-case WorldCast benchmark reveals a massive gap in multi-step visual state prediction: GPT-Image-2.5 System scores 72.8 vs. 42.8 for Nano Banana Pro and 20.1 for the best open-source model
  • NUS's Routing Policy Optimization (RPO) uses internal attention routing exploration to align Wan2.2-TI2V-5B in just 24.1 GPU-hours—3.1x faster than Flow-GRPO and 4.2x faster than DanceGRPO
Read full story
Linked Models & Agents:

vLLM and OpenRouter Launch Day-0 Serving for DeepSeek-V4.1-Flash-1017: 830K tok/s per GPU and 1.3B Tokens/$ TCO on SemiAnalysis AgentX

Following the release of DeepSeek-V4.1-Flash-1017, vLLM announced Day-0 serving optimization delivering 1.3B tokens per dollar TCO and 830K tok/s per GPU at 100 tok/s/user on SemiAnalysis AgentX (5x agentic throughput gain since Day 0), while OpenRouter launched live multi-provider routing starting at $0.10/M input and $0.50/M output tokens.

vLLM and OpenRouter Launch Day-0 Serving for DeepSeek-V4.1-Flash-1017: 830K tok/s per GPU and 1.3B Tokens/$ TCO on SemiAnalysis AgentX

Key Takeaways

  • vLLM delivers 830K tok/s per GPU (cached + input + output) at 100 tok/s per user and 1.3B tokens/$ TCO on SemiAnalysis AgentX for DeepSeek-V4.1-Flash-1017 (5x faster since Day 0)
  • DeepSeek-V4.1-Flash-1017 reaches 77.0% on DeepSWE v1.1 (beating Claude Opus 5 at 74.0%), 68.2% on NL2Repo-Bench, and 72.1% on Apex Bench with shorter agentic outputs
  • Live on OpenRouter from $0.10/M input and $0.50/M output tokens, while DeepSeek's official API alias deepseek-flash remains at $0.15/M input and $0.60/M output
Read full story
Linked Models & Agents: