vLLM shipped v0.29.0 with Model Runner V2 as the default runner for all models (594 commits, 277 contributors). Highlights include another Kimi-K3 optimization pass, DeepSeek-V4 shared experts fused into MegaMoE, Mamba prefix caching that keeps internal prefill checkpoints (about 9–25% better TTFT), per-request speculative-decoding acceptance stats over the API, a sharded_rdt P2P backend for RL weight sync, and new model support such as Hy4-preview, Qwen3.8-Flash-Next, GraniteSWA, and NemotronH Omni Reasoning V3.

Key Takeaways

  • ✓Model Runner V2 is now the default for all models.
  • ✓Mamba prefix caching improves TTFT ~9–25%; speculative decoding exposes acceptance stats via API.
  • ✓DeepSeek-V4 MegaMoE fusion, RL sharded_rdt P2P sync, and several new model backends.
ADSponsored