DeepSeek Open-Sources DeepInfer Inference Engine: Powering V4.1-Flash at Scale Across NVIDIA SM100 and Huawei Ascend 950
DeepSeek officially open-sourced DeepInfer under Apache 2.0, its production-grade inference engine serving DeepSeek-V4.1-Flash at scale. Originating from vLLM with architectural refinements from SGLang and TensorRT-LLM, DeepInfer natively targets NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. It integrates specialized kernels including FlashMLA, DeepGEMM, DeepEP, and TileKernels. The engine features DSpark confidence-scheduled speculative decoding with 5-token drafts, SWA bounded replay over a 140-token sliding window, and balanced MoE expert parallelism. DeepSeek simultaneously open-sourced deepseek-recipe, delivering an end-to-end open ecosystem across models, kernels, and inference runtimes.
Key Takeaways
- DeepSeek officially open-sourced DeepInfer under Apache 2.0, releasing the production codebase powering DeepSeek-V4.1-Flash at massive scale
- Natively optimized for both NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs with deterministic, batch-independent output guarantees
- Integrates custom FlashMLA, DeepGEMM, DeepEP, and TileKernels, alongside DSpark 5-token speculative decoding and 140-token SWA replay
- Simultaneously released deepseek-recipe, aligning open-source local deployments perfectly with DeepSeek's official production API tokenization
Linked Models & Agents:
