AI Coding Live News

The vLLM project team and Ant Group officially open-sourced Humming (humming-kernels), a specialized high-performance kernel library designed for serving quantized MoE and dense LLMs at massive scale. Humming natively supports WnA16, W8A8, W4A8, and W4A4 across INT1–INT8 and FP4–FP8 formats, spanning NVIDIA architectures from Turing through Blackwell as well as T-Head Zhenwu 810E silicon. In production benchmarks, Humming kernels outpace cuBLAS by 3–4x at small batch sizes and surpass Marlin by 2x on H200 clusters. Serving DeepSeek-V4.1-Flash across 8x H20 GPUs, Humming drives a +43.3% throughput gain under 64 concurrent requests while reducing time-to-first-token (TTFT) by 32%, pre-integrated directly into vLLM v0.29.0+.

Zoom

On Oct 21 the vLLM community and Ant Group open-sourced Humming (Apache 2.0), a quantized-inference library that JIT-compiles GEMM kernels on demand. It supports every NVIDIA architecture from Turing on plus T-Head's Zhenwu 810E, and covers weight and activation combinations from INT1-INT8 and FP4-FP8 for dense and MoE workloads. In vLLM's microbenchmarks it runs 2-5x faster than vLLM Marlin, FlashInfer or DeepGEMM in some cases; on 8 H20 GPUs, switching DeepSeek-V4.1-Flash MoE from Marlin W4A16 to Humming W4A8 raised throughput 14-43%. vLLM 0.29.0 ships it built in.

Zoom

Hours after DeepSeek open-sourced DeepInfer, the vLLM project team released official benchmark parity data serving DeepSeek-V4.1-Flash. Utilizing DeepSeek's identical decoding setup (131K context window, nextn=3 speculative decoding), vLLM closed its throughput performance delta to within 6% of DeepInfer across batch sizes 8 through 256 on NVIDIA GB200 systems. On GB300 hardware, vLLM attained up to 1,000,000 tokens/second per GPU and 826 tokens/second per user, generating $38.96 in hourly GPU revenue at DeepSeek off-peak rates. vLLM simultaneously shipped day-0 runtime support for FastH3-Omni and Spark-X2.6-4B.

Zoom

DeepSeek officially open-sourced DeepInfer under Apache 2.0, its production-grade inference engine serving DeepSeek-V4.1-Flash at scale. Originating from vLLM with architectural refinements from SGLang and TensorRT-LLM, DeepInfer natively targets NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. It integrates specialized kernels including FlashMLA, DeepGEMM, DeepEP, and TileKernels. The engine features DSpark confidence-scheduled speculative decoding with 5-token drafts, SWA bounded replay over a 140-token sliding window, and balanced MoE expert parallelism. DeepSeek simultaneously open-sourced deepseek-recipe, delivering an end-to-end open ecosystem across models, kernels, and inference runtimes.

Zoom

On Oct 20 (UTC) DeepSeek open-sourced DeepInfer (Apache 2.0), the in-house inference stack that serves DeepSeek-V4.1-Flash at scale. The release covers offline inference with Attention DP/CP and MoE EP+EPLB parallelism, DSpark speculative decoding, SWA bounded replay, batch-invariant inference, deterministic sampling and structured outputs on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. The KV cache system, prefill/decode disaggregation and load balancing are not included. DeepSeek also released deepseek-recipe (MIT), which converts Chat Completions, Messages and Responses requests and encodes them the same way the official API does.

Zoom

On Oct 19 (UTC) Unsloth released dynamic GGUF quants and a run-locally guide for DeepSeek-V4.1-Flash-1017. Per Unsloth, the 1-bit UD-IQ1_M quant is 250.4 GB (vs 508.4 GB for lossless Q8), keeps 88.2% top-1 agreement with the official weights on long17k, and runs on 128 GB+ RAM machines, because the model's 110.6–202.8 GB Engram n-gram table can stay on SSD via mmap. Running it currently requires Unsloth's llama.cpp PR build or Unsloth Desktop.

Zoom

Researchers introduced BlenderWorld (arXiv:2610.23986), the first comprehensive benchmark evaluating visual agents creating and editing 3D models from real-world references in Blender. Spanning 223 tasks across procedural geometry, Python scripting, material shaders, spatial layouts, and lighting, GPT-6 Astra leads the benchmark with a composite Score of 79.9 (and a 42.2% Perfect completion rate), followed by Opus-5 at 73.0. Open-weight flagship DeepSeek-V4.1-Flash claims 65.7, matching proprietary models while relying on active execution-and-inspection tool loops.

Zoom

The SGLang team released v0.5.22 featuring 950 PRs from 245 contributors: DeepSeek-V4.1 delivers up to 2.3x more throughput per GPU on NVIDIA H200 and 1.9x on AMD MI350X/MI355X via DSpark. The new InstantTensor loader cuts model loading time by 10x. B200 decodes Qwen3.8-Flash-Next 12% faster, while B300 speeds up GLM-5.3-Flash by 10% and yields 1.85x decode throughput for MiMo-V2.6-Pro. SGLang also adds native Apple Silicon PyTorch support and ComfyUI integration.

Zoom

Arena announced a major update to Code Arena: WebDev: DeepSeek-V4.1-Flash-20261017 (Max) clinched the #1 position among open models with 1670 points, surpassing Kimi K3 (Max) at a 10x lower cost. Ranking #12 overall, it outperforms Claude Opus 5 (High) and narrows the gap with closed-source frontier GPT-6 Sol (Max) to roughly 10 points, resetting the Pareto efficiency curve at $0.30 input / $1.20 output per million tokens.

Zoom

vLLM and NVIDIA unveiled a unified CuTe DSL decode megakernel (~12,000 LoC authored by NVIDIA CentML's Benjamin Chislett) for Moonshot AI's frontier Kimi K3 model on NVIDIA GB300 NVL72. By fusing attention, MoE, and cross-GPU communication, the optimization cuts per-layer latency from 130 µs (23 kernels) to 69 µs (13 kernels), delivering 842 tokens/s at batch=1 with DSpark speculative decoding (1.73x faster than stock vLLM) and up to 1.8x higher P90 per-user interactivity (403.2–423 tok/s/user) on SemiAnalysis AgentX, with DeepSeek-V4.1-Flash (PR #61868) and GLM-5.3 support coming next.

Zoom

Ollama launched US- and EU-hosted Ollama Cloud with strict zero data retention and zero training, adding immediate support for DeepSeek-V4.1-Flash-1017 via one-line launch commands for Claude Code and OpenCode alongside Ollama v0.40.6. Meanwhile, SemiAnalysis revealed that GLM-5.3 was built entirely by post-training the previous GLM-5.2 base model without pretraining from scratch, achieving a 'night and day difference' in cybersecurity and agentic capabilities.

Zoom

SemiAnalysis published the architectural breakdown behind the 1.3B tokens per dollar TCO record on AgentX: vLLM and Inferact utilized Prefill/Decode disaggregated serving at hyperscaler volume pricing to achieve 4x better cost-efficiency than an aggregated NVIDIA B300 configuration. Concurrently, open-source coding agent Cline reported that DeepSeek-V4.1-Flash-1017 beats GPT-5.6-Sol on Terminal-Bench at ~17x lower cost (+35% over prior V4.1-Flash) purely by scaling RL training across multiple coding agent harnesses.

Zoom

Following the release of DeepSeek-V4.1-Flash-1017, vLLM announced Day-0 serving optimization delivering 1.3B tokens per dollar TCO and 830K tok/s per GPU at 100 tok/s/user on SemiAnalysis AgentX (5x agentic throughput gain since Day 0), while OpenRouter launched live multi-provider routing starting at $0.10/M input and $0.50/M output tokens.

Zoom

On 2026-10-15 Microsoft AI published a technical post and PDF saying it has finished pre-training its in-house base model MAI-Base-2: MoE in every layer (1 shared + 7-of-512 routed experts), 80B active / 2.8T total, 102 layers, native encoder-free vision, a 50T-token mixture and a March 2026 knowledge cutoff. Per MAI, it beats every open-weight model tested on held-out BPB and its recipe is 4.2–12.3x more pre-training-FLOPs-efficient than DeepSeek-V4-Pro-Base. It is still in post-training: no API, no weights and no release date yet; Microsoft plans to license weights to trusted partners via Foundry.

Zoom