vLLM Narrows Gap with DeepInfer to 6% on NVIDIA GB200, Reaching 1M Tokens/s per GPU on GB300

Background and the Problem

As DeepSeek-V4.1-Flash solidifies its status as a foundational backbone for high-throughput coding agents, the performance delta between vendor-specific serving stacks (such as DeepSeek's newly open-sourced DeepInfer) and open general-purpose runtimes (such as vLLM) ignited intense industry debate. Infrastructure teams feared proprietary engines, heavily optimized with specialized fused kernels and tailored scheduling, would outpace general frameworks by 30% to 50%, forcing vendor lock-in. Countering this narrative, the vLLM core engineering team published verified benchmark results demonstrating near-parity on October 21, 2026 (@vllm_project, Status #2112703311933747347).

Architecture and How It Works

1. Parity with Production Topologies:

• Evaluated under DeepSeek's identical configuration: full 131,072 (131K) sequence length with nextn=3 multi-token speculative decoding;

1. NVLink-Optimized Kernel Pipeline on GB200 & GB300:

• Re-architected Multi-Head Latent Attention (MLA) decoding and MoE expert dispatch kernels to saturate the NVLink fabric of NVIDIA GB200 and GB300 systems;
• Interleaves MoE all-to-all collectives directly behind dense GEMM execution, effectively eliminating inter-device communication bubbles;

1. Day-0 Multi-Model Ecosystem Enablement:

• Added immediate day-0 serving for FastH3-Omni (real-time video generation) and Spark-X2.6-4B (an Apache 2.0, 1M-context lightweight backbone).

Benchmarks and Measured Results

Disclosed in official comparative figures (Status #2112703311933747347):

• GB200 Production Throughput Delta:
• Across production concurrency windows spanning batch sizes 8 to 256, vLLM's end-to-end token output closed within 6% of DeepInfer's published throughput;
• GB300 Peak Ceiling:
• Single-GPU sustained throughput surpassed 1,000,000 tokens/second;
• Single-user streaming velocity surged to 826 tokens/second, enabling instant multi-thousand-line codebase transformations;
• Unit Economics:
• Based on DeepSeek's published off-peak API pricing tiers, a single GPU yields $38.96 per hour in gross serving revenue, proving unmatched ROI for infrastructure providers.

Getting Started for Developers

• Announcement Link: Access official thread at https://x.com/vllm_project/status/2112703311933747347;
• Runtime Upgrade: Upgrading to vLLM's latest release activates optimized MLA and MoE communication pipelines for DeepSeek models;
• Infrastructure Strategy: Enterprise teams operating large GPU estates can remain anchored on standard vLLM without incurring stack migration debt, capturing frontier-grade serving performance while preserving cross-hardware flexibility.