AI Coding Live News — Real-Time 𝕏/Twitter Updates & Breakthroughs

@vllm_project@vllm_project·Collected 17d ago

👀99

🔥96

vLLM and OpenRouter Launch Day-0 Serving for DeepSeek-V4.1-Flash-1017: 830K tok/s per GPU and 1.3B Tokens/$ TCO on SemiAnalysis AgentX

Following the release of DeepSeek-V4.1-Flash-1017, vLLM announced Day-0 serving optimization delivering 1.3B tokens per dollar TCO and 830K tok/s per GPU at 100 tok/s/user on SemiAnalysis AgentX (5x agentic throughput gain since Day 0), while OpenRouter launched live multi-provider routing starting at $0.10/M input and $0.50/M output tokens.

Key Takeaways

vLLM delivers 830K tok/s per GPU (cached + input + output) at 100 tok/s per user and 1.3B tokens/$ TCO on SemiAnalysis AgentX for DeepSeek-V4.1-Flash-1017 (5x faster since Day 0)
DeepSeek-V4.1-Flash-1017 reaches 77.0% on DeepSWE v1.1 (beating Claude Opus 5 at 74.0%), 68.2% on NL2Repo-Bench, and 72.1% on Apex Bench with shorter agentic outputs
Live on OpenRouter from $0.10/M input and $0.50/M output tokens, while DeepSeek's official API alias deepseek-flash remains at $0.15/M input and $0.60/M output

Read full story

Linked Models & Agents:

#vLLM#DeepSeek-V4.1-Flash#OpenRouter#AgentX#LLM-Serving#Coding-Agent

LM Studio@lmstudio·Collected 45d ago

👀118

🔥94

View on 𝕏 ↗

LM Studio ships Inco Splash day 0: ~144 tok/s Qwen3.8-27B on M5 Max

LM Studio announced a day-0 partnership with Inco to bring the Splash inference engine into LM Studio. They cite up to about 144 tok/s for Qwen3.8-27B on M5 Max, with setup docs at lmstudio.ai/blog/splash-engine. Splash was already open-sourced by Inco; this puts the Apple-silicon-optimized stack inside LM Studio so local desktop users do not have to assemble the engine separately.

Key Takeaways

LM Studio integrates Inco Splash on day 0 for one-click local desktop use.
Claims ~144 tok/s for Qwen3.8-27B on M5 Max, with an official setup post.
Moves Apple-silicon-optimized inference into a mainstream local client distribution path.

Read full story

Linked Models & Agents:

⚡ Qwen3.8-27B 🌐 lm-studio 🌐 inco

#LM Studio#Splash#Inco#Qwen3.8#Apple Silicon#local inference#M5 Max

Inco AI@inco_ai·Collected 45d ago

👀200

🔥93

View on 𝕏 ↗

Inco open-sources Splash: ~144 tok/s for Qwen3.8-27B on M5 Max

Inco AI open-sourced Splash, a local inference engine built for Apple silicon and models like Qwen3.8-27B. On an M5 Max MacBook Pro it reports about 144 tok/s—up to 3× Ollama, 2× oMLX, and nearly 4× when an agent fans out sub-agents. LM Studio shipped day-0 support under Settings > Runtime > Experimental backends.

Key Takeaways

Open-source Splash: local inference tuned for Apple silicon and Qwen3.8-27B.
Demo: ~144 tok/s on M5 Max; up to ~3× Ollama / 2× oMLX, nearly 4× with agent fan-out.
LM Studio day-0 support; repo github.com/incoai/splash; blog lmstudio.ai/blog/splash-engine.

Read full story

Linked Models & Agents:

⚡ Qwen3.8-27B 🌐 inco 🌐 Qwen (通义千问)🌐 lmstudio

#Inco#Splash#Qwen3.8-27B#Apple Silicon#M5 Max#LM Studio#local inference#open source