High-throughput LLM inference engine SGLang released milestone v0.5.20, consolidating 713 merged PRs from 237 contributors. The update introduces native return_sampling_mask for reinforcement learning (RL) rollouts, returning exact token support and log-probabilities per decode step to eliminate post-hoc Top-k/Top-p reconstruction during PPO/GRPO/DPO training. Paired with overlap scheduling, decode throughput increases by 17% at batch size 1 and 52% at batch size 64 on Qwen3-8B. The release also implements a unified radix tree with branching-point caching for sliding window attention (SWA) and brings first-day support for GLM-5.3-Flash, Tencent Hy4-Preview, and Qwen3.8-Flash-Next.

Key Takeaways

  • ✓Massive release: 713 merged PRs from 237 contributors covering RL rollouts, overlap scheduling, and multi-model support
  • ✓Zero-reconstruction RL rollouts: return_sampling_mask returns token candidates and log-probs to streamline GRPO/PPO training
  • ✓52% decode throughput gain: overlap scheduling boosts Qwen3-8B decoding by 17% at batch 1 and 52% at batch 64
  • ✓Unified Radix Tree for SWA: enables branch-point caching across sliding-window and hybrid-attention model architectures
  • ✓Day-one frontier model support: out-of-the-box support for GLM-5.3-Flash, Tencent Hy4-Preview, Qwen3.8-Flash-Next, and SenseNova MoT
🔗

Project Links & Resources

Direct Access
Direct access to official project resources and documentation
🐙
GitHub Release Notes (sglang v0.5.20)
github.com/sgl-project/sglang/releases/tag/v0.5.20
↗
📖
SGLang Official Documentation
docs.sglang.io
↗
🐙
SGLang GitHub Repository
github.com/sgl-project/sglang
↗
🐙
GitHub: sgl-project/sglang
github.com/sgl-project/sglang/releases/tag/v0.5.20`
↗
🐙
GitHub: sgl-project/sglang
github.com/sgl-project/sglang/releases/tag/v0.5.20`.
↗
🔬

In-Depth Technical Analysis

Core Background & Industry Pain Points In the era of reinforcement learning post-training (PPO, GRPO, DPO) and test-time compute, inference engines must synchronize tightly with training frameworks. Standard engines only output final sampled tokens during rollouts, forcing trainers to reconstruct Top-k/Top-p truncated distributions at high computational cost. Furthermore, as hybrid-attention and sliding window attention (SWA) architectures proliferate, legacy Radix Tree caches fail across window boundaries, causing severe Time-to-First-Token (TTFT) latency penalties and VRAM fragmentation. ### Architecture Highlights & Internals SGLang v0.5.20 implements core infrastructural upgrades: 1. Sampling Masks for RL Rollouts: The return_sampling_mask parameter instructs decoders to output exact token candidate support sets and log-probabilities at each step, enabling lossless trajectory replay without probability reconstruction; 2. Overlap Pipeline Scheduling: Masks run under asynchronous overlap scheduling with --sampling-mask-max-tokens (default 4096), eliminating host-device synchronization stalls; 3. Unified Radix Tree for SWA: Re-architects cache branching for sliding window models, enabling shared prefix hits comparable to standard causal attention; 4. Broad Model Compatibility: Delivers first-day support for GLM-5.3-Flash, Tencent Hy4-Preview, Qwen3.8-Flash-Next, FastH3, and SenseNova MoT. ### Authoritative Benchmarks & Measured Scores - Decoding Throughput: On NVIDIA H100 with Qwen3-8B, overlap scheduling boosts decode throughput by 17% at batch size 1 and 52% at batch size 64; - RL Training Speedup: Integrated with Ray/DeepSpeed GRPO workflows, removing reverse probability reconstruction shortens single-round training latency by 28.4%; - SWA Prefix Hit Rate: Over 32K context evaluations, prefix hit rate jumps from 21.3% to 78.9%, slumping TTFT by 63%. ### Developer Hands-on Guide Upgrade via pip install --upgrade sglang and launch with --return-sampling-mask --sampling-mask-max-tokens 4096. Release notes: https://github.com/sgl-project/sglang/releases/tag/v0.5.20.