DeepSeek open-sources DeepInfer, the inference framework behind its V4.1-Flash production service, with speculative decoding, batch-invariant inference and structured outputs, plus the deepseek-recipe API encoding library

DeepSeek's open weights are popular, but new architectures such as V4.1-Flash (Engram, sliding-window attention) usually wait for vLLM or SGLang support, and community deployments rarely match the official API. On Oct 20 DeepSeek open-sourced DeepInfer, the framework behind its production service, and said inference code for future models will land there too. Per the GitHub repo, DeepInfer was originally built mainly on vLLM and uses FlashMLA, DeepGEMM, DeepEP, TileKernels and custom TileLang kernels. It supports Attention DP / balanced CP and MoE EP+EPLB, DSpark draft-token decoding with confidence-scheduled verification, SWA bounded replay to cut prefill work, a zero-overhead async scheduler, batch-invariant inference, deterministic sampling, and regex/JSON Schema/EBNF structured outputs. The companion deepseek-recipe library converts and encodes requests to match the official API. DeepSeek published only throughput-latency charts (4 GPUs for prefill, 64 for decode) with no head-to-head numbers versus vLLM or SGLang; its reference V4.1 scores (vendor-reported) are AIME 2026 99.9%, GPQA Diamond 93.4% and HLE-Full 42.6%. To try it you need NVIDIA SM100 or Ascend 950 hardware, Linux, Python 3.12, GCC 13+ and CUDA 13 or CANN 9.2; set DEEPINFER_ACCELERATOR, run ./build.sh, then examples/offline_inference.py. The KV cache system, prefill/decode disaggregation and load balancing are not included, so production serving still needs your own pieces or vLLM/SGLang.