Release16d ago
DeepSeek ships V4.1-Flash-1017: MIT open weights, live on web, app and API at unchanged prices, DeepSWE 77.0 (vendor-reported), plus public Search/Fetch APIs
On Oct 17 DeepSeek upgraded V4.1 Flash to V4.1-Flash-1017 across web, app and API (deepseek-flash, same call and price) and published MIT-licensed weights on Hugging Face and ModelScope. Vendor-reported DeepSWE v1.1 rises from 74.2 to 77.0 and Terminal-Bench 4.0 from 31.2 to 42.3, with shorter runtimes and outputs. It also opened Search/Fetch APIs backed by its in-house search engine at $0.75 per 1,000 calls.
Tooling12d ago
vLLM and Ant Group open-source Humming, one quantized-inference kernel library for many formats and every NVIDIA GPU since Turing, lifting DeepSeek-V4.1-Flash end-to-end throughput by 14-43%
On Oct 21 the vLLM community and Ant Group open-sourced Humming (Apache 2.0), a quantized-inference library that JIT-compiles GEMM kernels on demand. It supports every NVIDIA architecture from Turing on plus T-Head's Zhenwu 810E, and covers weight and activation combinations from INT1-INT8 and FP4-FP8 for dense and MoE workloads. In vLLM's microbenchmarks it runs 2-5x faster than vLLM Marlin, FlashInfer or DeepGEMM in some cases; on 8 H20 GPUs, switching DeepSeek-V4.1-Flash MoE from Marlin W4A16 to Humming W4A8 raised throughput 14-43%. vLLM 0.29.0 ships it built in.
Inference12d ago
vLLM and Ant Group Open-Source Humming: Up to 5x Kernel Acceleration and +43.3% MoE Throughput
The vLLM project team and Ant Group officially open-sourced Humming (humming-kernels), a specialized high-performance kernel library designed for serving quantized MoE and dense LLMs at massive scale. Humming natively supports WnA16, W8A8, W4A8, and W4A4 across INT1–INT8 and FP4–FP8 formats, spanning NVIDIA architectures from Turing through Blackwell as well as T-Head Zhenwu 810E silicon. In production benchmarks, Humming kernels outpace cuBLAS by 3–4x at small batch sizes and surpass Marlin by 2x on H200 clusters. Serving DeepSeek-V4.1-Flash across 8x H20 GPUs, Humming drives a +43.3% throughput gain under 64 concurrent requests while reducing time-to-first-token (TTFT) by 32%, pre-integrated directly into vLLM v0.29.0+.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.