vLLM shipped v0.29.0 with Model Runner V2 as the default runner for all models (594 commits, 277 contributors). Highlights include another Kimi-K3 optimization pass, DeepSeek-V4 shared experts fused into MegaMoE, Mamba prefix caching that keeps internal prefill checkpoints (about 9–25% better TTFT), per-request speculative-decoding acceptance stats over the API, a sharded_rdt P2P backend for RL weight sync, and new model support such as Hy4-preview, Qwen3.8-Flash-Next, GraniteSWA, and NemotronH Omni Reasoning V3.
Key Takeaways
- ✓Model Runner V2 is now the default for all models.
- ✓Mamba prefix caching improves TTFT ~9–25%; speculative decoding exposes acceptance stats via API.
- ✓DeepSeek-V4 MegaMoE fusion, RL sharded_rdt P2P sync, and several new model backends.
⚡
TL;DR VerdictKey Decision Metrics at a Glance
🧭
Not enough VRAM? Compare cloud API and self-hosting costs
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Action HubReady to adopt this in production?
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Up NextvLLM v0.32.0: 814 commits, built-in /v1/systemone structured-decision endpoint, NVFP4 KV cache on more GPUs, and 43–85% more req/s with multiple API workers
Read Now→
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.