SemiAnalysis Breaks Down 1.3B Tokens/$ AgentX Record: vLLM + Inferact Disaggregated Serving Beats Aggregated B300 by 4x, While Cline Highlights Multi-Harness RL

Core Background & Industry Pain Points

Scaling autonomous coding agents in production raises two critical questions across AI infrastructure and post-training engineering: First, why do traditional aggregated serving deployments—where a single GPU instance handles both compute-bound prompt prefill and memory-bandwidth-bound token decoding—suffer from severe hardware underutilization on next-generation accelerators like NVIDIA B300 during multi-turn agent loops? Second, how can open-weights models bridge the gap with proprietary frontier models like GPT-5.6-Sol on complex terminal engineering tasks without altering their underlying model architecture?

Architectural Highlights & Core Mechanisms

Today's hardware-software analyses from SemiAnalysis (@SemiAnalysis_) and open-source coding agent team Cline (@cline) highlight two decisive engineering levers:

• vLLM × Inferact Prefill/Decode Disaggregated Serving on AgentX: Recognizing that multi-turn agent workloads interleave massive shared-prefix prefills with short, latency-sensitive action decodes, @vllm_project and @inferact deployed a fully disaggregated serving architecture on the SemiAnalysis AgentX benchmark. Separating prefill instances from decode instances allows independent batching, parallelism, and KV-cache routing policies tailored to each phase's hardware bottleneck.
• Multi-Harness RL Scaling Without Algorithmic Changes: Analyzing DeepSeek-V4.1-Flash-1017's dramatic leap in coding agent reliability, @cline highlighted that the upgrade required "no algorithmic innovation" or backbone changes; instead, the gains came entirely from scaling post-training reinforcement learning across multiple distinct coding agent harnesses (DeepSeek Harness, Tencent CodeBuddy, and diverse terminal/editor tool loops), teaching the policy to emit shorter trajectories and generalize across tool schemas.

Authoritative Benchmarks & Empirical Comparisons

• 4x TCO Advantage Over Aggregated NVIDIA B300: Evaluated under hyperscaler volume pricing on SemiAnalysis AgentX, @vllm_project and @inferact achieved 1.3 billion tokens per dollar of TCO, outperforming the NVIDIA B300 aggregated serving configuration by 4x.
• 35% Terminal-Bench Gain at ~17x Lower Cost: As highlighted by @cline, DeepSeek-V4.1-Flash-1017 improves the previous DeepSeek-V4.1-Flash result on Terminal-Bench by 35% (reaching 42.3% on Terminal-Bench 4.0), surpassing GPT-5.6-Sol while running at ~17x lower inference cost.

Developer Practical Guide & Quickstart

Inference platform teams deploying B200/B300 or Hopper clusters for agentic workloads should transition from aggregated single-pool serving to vLLM disaggregated prefill/decode topologies to capture up to 4x TCO improvements. Meanwhile, developers using Cline (VS Code, JetBrains, CLI, or Desktop v0.0.48) can configure deepseek/deepseek-v4.1-flash-1017 via OpenRouter or DeepSeek API to run autonomous repository edits at a fraction of proprietary model costs.