a16z's Charts of the Week for Oct 30 (by Moses Sternstein) looks at the "Rack Spread": GPU rental prices are rising while token prices fall. The official a16z X account summed it up on Oct 31: since May, average token prices are down 67% and token spend is up more than 120%. By the article's napkin math, Ramp data gives roughly 122%, and Ornn's blended price multiplied by OpenRouter volume gives roughly 160%. A separate post says that since mid-July B300 rentals are up 106%, B200 up 66% and H100 up 17%. a16z also stresses that these indexes come from small, biased samples and do not show true total demand.
Zoom
Key Takeaways
Prices and spend: since May, average token prices are down 67% and token spend is up more than 120% (a16z post on X, a16z figures)
Two napkin estimates: Ramp token-spend data shows about +122%; Ornn blended price times OpenRouter volume shows about +160% (Charts of the Week, described by the author as napkin math, not a rigorous method)
GPU rentals since mid-July: B300 +106%, B200 +66%, H100 +17%, while average token prices fell 58% (a16z post)
AI spend per employee: an Atlanta Fed survey puts the mean above $1,000, while Ramp's median is $15.70 a month ($188 a year)
Reach: the two posts have about 371 and 492 likes and 38K and 96K views (as of 18:15 SGT)
On Oct 28, Mixpanel founder Suhail Doshi announced his new company Particle on X, with its first model, Particle-1, live now. Particle's blog says it is a reinforcement-learning post-train of DeepSeek V4.1 Flash that uses 28.4% to 52.2% fewer output tokens and costs 34.4% to 51.6% less per task across four coding and data engineering benchmarks, with pass rates inside the base model's confidence intervals. It is available through the Particle API, OpenRouter and Vercel AI Gateway; Particle's docs list $0.25 input and $1.00 output per million tokens with a 1,048,576-token context window.
Zoom
Key Takeaways
Across four benchmarks (Terminal-Bench 2.1, ReactBench, ADE Bench, DeepSWE 1.1), 28.4% to 52.2% fewer output tokens and 34.4% to 51.6% lower cost per task (vendor-reported)
Particle's summary table: $0.2211 per task versus $0.3368 for DeepSeek V4.1 Flash and $0.6523 for GLM 5.3 Flash; 59,580 mean output tokens versus 83,246 and 98,155
Task pass rate 72.57% versus 71.15% for the base model and 66.82% for GLM 5.3 Flash; GPQA Diamond 90.40% (base 90.91%), MMLU 86.08% (base 85.98%)
Training: a rank-64 LoRA on the experts in all layers, about 1.25% of the model's parameters, trained with sequence-level GSPO in two RL stages
Pricing: Particle docs currently list $0.25 input, $0.03 cached input and $1.00 output per million tokens, a 1,048,576-token context and a 131,072-token output ceiling
On October 19 Inception released Mercury 3, which it calls the most capable diffusion LLM (dLLM) to date. Per the vendor, it is more than 40x the size of Mercury 1, outputs 1,068 tokens/s, expands context from 260K (Mercury 2.5) to 1M tokens, and beats GPT-6 Luna and Claude Haiku 5.5 (both medium) on GPQA Diamond, IFBench, SWE-Bench Atlas and more. List price is $0.20/M input and $0.70/M output, 50% off at launch. It is live on the Inception API, OpenRouter, Baseten and Vercel AI Gateway with an OpenAI-compatible API. All scores are vendor-reported and not yet independently reproduced.
Zoom
Key Takeaways
Speed: 1,068 tokens/s output; the vendor table lists GPT-6 Luna at ~130 and Haiku 5.5 at ~240 (vendor figures).
Scores: GPQA Diamond 89.23, IFBench 76.84, SWE-Bench Atlas 51.08, all above Luna and Haiku 5.5; but DeepSWE v1 at 46.31 trails Haiku 5.5 (51.62), and MRCR v2 also trails Haiku.
Scale and context: more than 40x the parameters of Mercury 1, with a 1M-token window (260K on Mercury 2.5).
Price: $0.20/M in and $0.70/M out list, 50% off at launch ($0.10/$0.35), cached input 90% off; new API keys get 100M free tokens.
Access: Inception API, OpenRouter, Baseten and Vercel AI Gateway, OpenAI-compatible as a drop-in replacement.
MiniMax confirmed that Space Bunny, the anonymous model that led usage rankings on OpenRouter and elsewhere, was an early version of M3.1 Flash, and launched it on its API on October 19. Pay-as-you-go pricing is $0.15 input, $0.60 output and $0.015 cache read per million tokens, with a 1M context, text, image and video input, and an effort setting for thinking depth. Per MiniMax, it costs one third of M3 and runs 3x faster, scoring 73.7% on DeepSWE and 62.2% on Automation-Bench; the weights went live on Hugging Face on schedule on October 23 (MiniMax Community license, MXFP8), with day-0 support from Together, Inferact (vLLM) and LightSeek TokenSpeed.
Zoom
Key Takeaways
Official pay-as-you-go pricing: $0.15 input / $0.60 output / $0.015 cache read per million tokens; per token that is half of M3 ($0.30/$1.20), and MiniMax does not explain how it gets one third of the cost.
1M context, text/image/video input, five effort levels from low to max, default max, thinking cannot be turned off.
Weights opened October 23 at Hugging Face MiniMaxAI/MiniMax-M3.1-Flash: ungated, MiniMax Community license (not MIT/Apache), MXFP8 quantized, native multimodal MoE with about 428B total and 23B active parameters per token, plus a DSpark speculative-decoding draft head.
Migration: M2.5 and M2.7 closed to new users on October 19 and retire on November 18; M Plan and Token Plan requests now route to M3.1 Flash automatically.
Day-0 serving: Together AI runs it fully managed, MiniMax is partnering with Inferact to contribute production-inference improvements back to vLLM, and LightSeek's TokenSpeed added day-0 support.
Following the release of DeepSeek-V4.1-Flash-1017, vLLM announced Day-0 serving optimization delivering 1.3B tokens per dollar TCO and 830K tok/s per GPU at 100 tok/s/user on SemiAnalysis AgentX (5x agentic throughput gain since Day 0), while OpenRouter launched live multi-provider routing starting at $0.10/M input and $0.50/M output tokens.
Zoom
Key Takeaways
vLLM delivers 830K tok/s per GPU (cached + input + output) at 100 tok/s per user and 1.3B tokens/$ TCO on SemiAnalysis AgentX for DeepSeek-V4.1-Flash-1017 (5x faster since Day 0)
DeepSeek-V4.1-Flash-1017 reaches 77.0% on DeepSWE v1.1 (beating Claude Opus 5 at 74.0%), 68.2% on NL2Repo-Bench, and 72.1% on Apex Bench with shorter agentic outputs
Live on OpenRouter from $0.10/M input and $0.50/M output tokens, while DeepSeek's official API alias deepseek-flash remains at $0.15/M input and $0.60/M output
Perplexity Developers (@perplexitydevs) announced the launch of pplx-decider-v1.2 (27B) and pplx-decider-flash-v1.2 (4B) across OpenRouter and Vercel AI Gateway. Taking both text and images as input, these open-weight multimodal decision models return typed answers with calibrated probabilities at just $0.02 per million input tokens for the 27B checkpoint and half the cost for the 4B Flash variant.
Zoom
Key Takeaways
Dual open-weight multimodal decision models: launches 27B pplx-decider-v1.2 and 4B pplx-decider-flash-v1.2 taking text or images and returning typed answers with probabilities
Ultra-low $0.02/1M token pricing: 27B checkpoint costs $0.02 per million input tokens, while 4B Flash runs faster at half the price
Live on OpenRouter and Vercel AI Gateway: provides drop-in rolling aliases for low-latency agent routing and multimodal classification
Liquid AI released d1-ultra on Oct 15: a multimodal decision model that takes text and images and, in one forward pass, returns probabilities for yes/no, choice and score questions without generating tokens; billed on input only. Artificial Analysis ranks it #1 on Decision Index v1.0 at 65, at least 8 points ahead, with an 85% non-hallucination rate. Available on the Liquid API, Vercel AI Gateway and OpenRouter.
Zoom
Key Takeaways
Artificial Analysis Decision Index v1.0: 65, ranked #1 and at least 8 points ahead (independent)
AA-Omniscience-Decision non-hallucination 85% vs 40% for runner-up Microsoft-Decision-1; more abstention drops accuracy to 32% vs d1's 41%
$0.042 per M input tokens, output free; a 1024x1024 image counts as 1,536 tokens
32,768-token context, 44 languages, leads all eight categories of Liquid's 112-eval suite (vendor)
Call via Liquid /decisions/v1/systemone with model d1-ultra, or liquid/d1-ultra on Vercel AI Gateway / OpenRouter
On 2026-10-15 Meta released weights for Muse Spark Open, a natively multimodal MoE (~816B total / ~42B active per token, 1M context) aimed at long-horizon coding and agents. Weights are on Hugging Face and peers; hosted APIs include Together, Fireworks, Groq and OpenRouter. Reported figures include DeepSWE 60%, MCP-Atlas 89.2% and JobBench 61.8% (see Meta’s evaluation methodology).
Zoom
Key Takeaways
Scale: ~816B total / ~42B active per token; 256 experts/layer with top-8 routing; 62 layers, hidden 8192; context up to 1,048,576 (HF blog)
Throughput (NVIDIA tech blog, agentic on GB300 NVL72): up to ~50K tok/s/GPU and ~600 tok/s/user; NVFP4 checkpoints published (NVIDIA blog)
Deploy: Muse Model License with limited commercial terms; SGLang / vLLM / TensorRT-LLM / MLX / llama.cpp; Meta demos two Apple M5 Ultra Mac Studios for on-desk local run
Third-party index: Artificial Analysis Intelligence Index ~38.9–39 (xhigh, as of 2026-10-07)—behind closed frontier, but a key open-weight reference (AA)
On 2026-10-15 Meta released Muse Spark Open open weights: an 816B-total / 42B-active MoE with a Perception Encoder, natively multimodal (text/image/video/audio in → text out), 1M context, aimed at coding and long-horizon agents. Weights are on Hugging Face (BF16/NVFP4), SageMaker, Model Garden and Foundry; APIs include OpenRouter, Together, Fireworks and Groq. Meta calls it the most capable US open-weight model to date and, with Apple, optimized it to run locally across two M5 Ultra Mac Studios via MLX.
Zoom
Key Takeaways
Specs: MoE 816B total / 42B active (256 experts, top-8), 62 layers, 1M context; text+image+video+audio in → text out (model card)
Local: NVFP4 needs ≥~700GB VRAM, BF16 ~2TB; or two Apple M5 Ultra Mac Studios via MLX (Meta blog)
Agentic benches (xhigh, official table): MCP-Atlas 89.2, JobBench 61.8, OSWorld Verified 80.4, Terminal-Bench 2.1 76.4, DeepSWE 1.1 60, AA Intelligence Index 39
On 2026-10-15 NVIDIA released Nemotron 3.5 Super VL, an open multimodal agent model with 120B total / 12B active parameters: a Mamba-2–Transformer hybrid MoE with a C-RADIO vision encoder, text/image/document/video input, up to 1M context, and built-in MTP plus LiLiCorr speculative decoding. Weights, datasets and recipes ship under OpenMDW-1.1 on Hugging Face, OpenRouter, ModelScope and NIM. Alongside it, NVIDIA published Bolt 74B-A9B 24GB, a natively 2-bit (QAT) local agent model for RTX/DGX Spark; the 4GB Bolt E4B follows on Oct 23.
Zoom
Key Takeaways
Scale: Super VL is 120B total / 12B active with up to 1M context; BF16 reference serving needs 4×H100 or 2×GB200, with an NVFP4 build for smaller footprints (model card)
Coding (vendor-reported): SWE-bench Verified 71.10, Terminal-Bench 2.1 66.70, LiveCodeBench v6 88.22; in the same table Qwen3.5-122B-A10B scores 71.9 / 48.70 / 76.7 and GLM-5.3-Flash 79.10 / 82.70 / 80.6
Agent evals: Hermes Bench 63.44% (Nemotron 3 Ultra 56.33%), LangChain Evals 87.05% (Ultra 84.3%); NVIDIA claims up to 5x single-GPU throughput vs open peers and 600+ TPS on Baseten (tech blog)
Bolt 74B-A9B 24GB: native 2-bit (IQ2 + QAT), 256K context; Terminal-Bench 2.1 50.98 and BrowseComp 66.75 vs 44.38 / 48.74 for Qwen3.6-35B-A3B; PinchBench 87.82 is slightly below Qwen3.6's 88.07 (Bolt card)
Serving: LiLiCorr speculative decoding gives up to 2.3x faster generation on DGX Spark; the Dynamo recipe serves 8x more users with 85% lower TTFT on 16×B200 at 50 tok/s/user
On 2026-10-14 Inworld AI opened Inworld Inference to every developer, positioning it as an inference service for consumer apps (companions, tutors, voice, interactive fiction). Instead of listing thousands of models it builds a per-model serving stack for open models such as Gemma 4, GLM 5.3 Flash, DeepSeek and MiMo: draft-and-verify (speculative) decoding, a rewritten split-KV verify attention kernel, KV-aware routing and eval-gated low precision. Vendor-reported: Gemma 4 26B p95 first token of 0.31 s, 7.8x faster than the median OpenRouter provider; up to 90% of prompts from cache; 99.999% success over 20M+ GLM 5.3 Flash requests in 30 days. The API is OpenAI-compatible with Flex/Default/Priority/Fast tiers per request.
Zoom
Key Takeaways
Gemma 4 26B first token: p50 0.17 s / p95 0.31 s, 3.6x / 7.8x faster than the median OpenRouter provider (blog, vendor-reported snapshot, not a matched workload)
Pricing: Gemma 4 26B at $0.07 input, $0.035 cached, $0.34 output per 1M tokens, nearly 40% below the median no-training provider
Rewritten verify attention: 3.4–7.5x faster per layer, attention share of a decode step 27.5%→16.5%, p50 TTFT 154→117 ms at equal eval scores
KV-aware routing: GLM 5.3 Flash cache hits 25%→47%, DeepSeek V4.1 Flash 11%→33%; cached MiMo 2.6 Pro input costs 99% less
Reliability: 98 failures across 20M+ GLM 5.3 Flash requests in 30 days (~4.8 per million); per-request service_tier of Flex/Default/Priority/Fast
On Oct 13 Liquid AI released LFM2.5-24B-A2B, the largest and final LFM2.5 model: 24B total parameters, ~2.45B active per token, 131K context, and always-on reasoning. By Liquid's own numbers it leads or matches Qwen3.6-35B-A3B on 5 of 8 tool-use/agent benchmarks (AutomationBench 27.69 vs 8.72) while trailing on SWE-Bench Verified, GPQA and other knowledge/coding tests. Open weights are on Hugging Face with GGUF, MLX, FP8 and ONNX builds; API pricing is $0.10 / $1.00 per million tokens and it is also on OpenRouter.
Agent benchmarks (vendor-reported, vs Qwen3.6-35B-A3B): AutomationBench 27.69 vs 8.72, LiveClaw 48.09 vs 38.19, ToolSandbox 80.71 vs 75.02, BFCLv4 65.01 vs 65.00
Disclosed weaknesses: SWE-Bench Verified (OpenCode) 51.00 vs 66.20, GPQA Diamond 74.42 vs 83.69, WebArena Verified 26.75 vs 37.62
Speed: 184 tok/s decode on Apple M5 Max, 114 tok/s on Ryzen AI Max+ 395, 21K+ output tok/s on one B300 with SGLang
Training: pretraining scaled 17T to 33T tokens; agentic RL inside OpenClaw, Hermes Agent, Pi and OpenCode; API $0.10 in / $1.00 out per M tokens
On Oct 13 ScaleDown launched Constrained Decoding Models (CDMs): each model is trained for a single task and only returns answers that task allows (a label or spans of the input) instead of free-form token-by-token generation. LangChain shipped a langchain-scaledown integration and LangSmith LLM Gateway support the same day, with middleware for history summarization, tool-result compression and structured extraction. ScaleDown claims 200x faster and 467x cheaper than frontier models on these narrow tasks; Extract and Summarize are on OpenRouter at $0.04 per million input tokens.
Zoom
Key Takeaways
Four single-task models (Summarize, Compress, Extract, Classify), each taking up to 1M input tokens; Extract, Summarize and Classify accept images and scanned PDFs
Vendor-reported: vs the average of GPT-5.5, GPT-5.4 Mini and GPT-5.4 Nano, Summarize is 41x cheaper and 2.7x faster, Compress 138x cheaper and 1.8x faster, both with higher accuracy
Classify vs Jev: Jev is text-only with 32K tokens of state per question; Classify takes 1M tokens plus images and adds a one-sentence reason per decision
LangChain's trace example compresses a docs page from 18,402 to 3,110 characters; if a ScaleDown call fails the middleware leaves history untouched and the agent keeps running
On OpenRouter, Extract and Summarize cost $0.04/M input tokens with 1M context; Summarize caps output at 4,000 tokens and neither supports tools or response_format
On Oct 9 Microsoft released Microsoft-Decision-1, a decision-scoring model that returns calibrated probabilities over fixed options instead of generating text, aimed at routing, classification, verification, agent guardrails and AI judging. Microsoft reports top accuracy across 36 blind benchmarks, ~35x lower P50 latency than GPT-6 Sol and a 1.3% decision-flip rate under perturbation. Available on Microsoft Foundry and OpenRouter, 32K context, $0.042/M input, free output.
Vendor-reported: highest accuracy across 36 blind benchmarks (~150K questions); 2.5x faster than runner-up H2O-Lightning-4B v1.1 and 35x faster than GPT-6 Sol
Robustness: 1.3% average decision flips across 8 perturbation types; zero flips when options are paraphrased, reversed or shuffled
Base: post-trained from Qwen3.5-9B for single-pass scoring; Microsoft plans to rebase on MAI and OpenAI models
Internal use: Xbox Research labeled 10K+ feedback items at near GPT-6 Sol quality, 14x+ faster and ~200x cheaper (vendor-reported)
On 2 Oct 2026 OpenRouter shipped Model Router Benchmarks: seven model routers (Auto, Jev, NVIDIA Switchyard, Unbiased Pareto, Sakana Fugu, and variants) compared on GPQA Diamond, Tau3 Banking, MMMU Pro Vision, SWE Atlas QA, Deep SWE, and Terminal Bench 2.1. The Router Index defaults to quality 60% / time 20% / cost 20%. openrouter/pareto-code and Fusion are not on this board.
Zoom
Key Takeaways
Announced 2026-10-02 22:08 UTC; blog by Brian Thomas dated 10/2/2026; live page openrouter.ai/benchmarks/routers
Seven router slugs in the public stats feed: openrouter/auto, typesafe/jev-router, nvidia/switchyard, unbiased/pareto, sakana/fugu-max, sakana/fugu-ultra, sakana/fugu-ultra-v2. pareto-code and Fusion were not benchmarked
Router Index default is quality 60% / time 20% / cost 20% on a 0-10 scale; it is not raw accuracy
Stats API 2026-10-04 SGT, Deep SWE n=113: Switchyard Flash+Opus 5.5 and Opus-only both 83/113 (~$279 / $285); Flash+GPT-6 Sol 74/113 at ~$55
Terminal Bench 2.1 n=89: GPT-6 Astra 77/89 ($66.50), Opus 5.5 74/89 ($24.00); one Pareto run 71/89 ($27.20)
-oOdUn61bDRSykk4H951SoDOHzqoOe1.png)
