AI Coding Live News

llama.cpp and ggml Ship v1.0.0: First Major Version in 3.5 Years Adopts Semantic Versioning, With Breaking C API Changes Now Forcing a Major Bump; Release Drops 35 Deprecated Functions and Adds FP8 KV Caches, a MoE Expert Cache and /v1/audio/speech

On November 13, ggml-org published llama.cpp v1.0.0 on GitHub and announced on the Hugging Face blog that both the llama.cpp and ggml libraries are now at v1.0.0. From here on, breaking changes to the public C API in ./include bump the major version, backward-compatible features, backends and model support bump the minor version, and pure fixes bump the patch version. The release removes 35 deprecated functions, promotes the NextN/speculative decoding API into llama.h, adds OCP FP8 KV caches and a MoE expert cache with a configurable CPU share, supports new architectures such as Kimi-K3.1-Pro, hardens llama-server against SSRF, CORS and DNS rebinding issues, and adds an OpenAI-compatible /v1/audio/speech endpoint. Creator Georgi Gerganovs release post has about 4,180 likes.

llama.cpp and ggml Ship v1.0.0: First Major Version in 3.5 Years Adopts Semantic Versioning, With Breaking C API Changes Now Forcing a Major Bump; Release Drops 35 Deprecated Functions and Adds FP8 KV Caches, a MoE Expert Cache and /v1/audio/speech

Key Takeaways

  • Versioning rules: MAJOR = breaking change to the public C API in ./include; MINOR = backward-compatible feature, backend, model support or API addition; PATCH = fix with no API change
  • Breaking changes: llama_set_warmup, 35 deprecated functions and --defrag-thold are gone, and llama_set_embd_state_src is renamed llama_set_ctx_state_src, so apps embedding libllama need code changes before upgrading
  • New capabilities: FP8 KV caches (-ctk/-ctv f8_e4m3), a MoE expert cache (moe_cache_size, moe_cache_cpu_share) and loading external PLE shards via --shard/--ple
  • llama-server is hardened against SSRF, CORS, DNS rebinding and MCP tool cancellation issues, gains an OpenAI-compatible /v1/audio/speech endpoint and a redesigned web UI (GitHub release)
  • Distribution channels include Docker, Debian, Homebrew, conda-forge, MacPorts, Winget and XCFramework; the project supports more than 150 architecture variants
Read full story
Linked Models & Agents:

Zyphra Open-Sources ZONOS3-Mini TTS: 800M-Parameter MoE With 100M Active, Apache 2.0, Real-Time in a Phone Browser, Word-Level Delivery Control and Voice Cloning

Zyphra released ZONOS3-Mini, an open-source text-to-speech model, on November 9, with weights on Hugging Face under Apache 2.0. It is a sparse MoE with 800 million total and about 100 million active parameters per token, trained on 112 H100s in 2.5 days. By Zyphra's numbers, time to first audio is under 570 ms in the browser on a Pixel 9 and under 150 ms on an M3 MacBook Air, and it ranks first among open models on the capability index of its new ZTTS2-Eval benchmark (35 languages of read speech plus 20 in-the-wild), ahead of Fish S2 Pro and MOSS TTS v1.5. It supports voice cloning, age, accent and gender steering, and word-level delivery control, with WebGPU, SGLang, vLLM, GGML and MLX runtimes.

Zyphra Open-Sources ZONOS3-Mini TTS: 800M-Parameter MoE With 100M Active, Apache 2.0, Real-Time in a Phone Browser, Word-Level Delivery Control and Voice Cloning

Key Takeaways

  • Scale: 800M total, about 100M active parameters (64 experts, top-2 routing), versus 8B for the previous ZONOS2; open weights under Apache 2.0
  • Speed (Zyphra figures): under 570 ms time to first audio in a Pixel 9 browser and under 150 ms on an M3 MacBook Air; 80 ms median at 8 concurrent requests and peak 457 seconds of audio per second on one H100
  • The new ZAC-1 codec runs at 15 Hz versus 86 Hz for DAC and cuts word error rate about 10x under similar data; one generation can produce up to 10 minutes of 44.1 kHz audio
  • Evaluation: ZTTS2-Eval covers 35 languages of read speech plus 20 in-the-wild; on its 420-item capability index ZONOS3-Mini ranks first among open models, followed by Fish S2 Pro and MOSS TTS v1.5 (Zyphra's own benchmark, not yet independently reproduced)
  • Get started: try it in the browser via WebGPU with no install, or run sgl-omni serve --model-path Zyphra/ZONOS3-Mini for an OpenAI-compatible /v1/audio/speech endpoint
Read full story
Linked Models & Agents:
⚡ ZONOS3-Mini🌐 Zyphra
g
ggml-org@AICoder·
👀226
🔥84

llama.cpp v0.6.0 ships GLM-5.3-Flash (320B) and Qwen4Exp MTP speculative decoding, a new llama_batch_ext API, and up to ~3x faster few-row mat-mul on Apple GPUs

ggml-org released llama.cpp v0.6.0 on Oct 5 (v0.5.0 was Sep 23). It brings GLM-5.3-Flash (GLM5-Next, a 320B text+vision hybrid MoE) into a stable tag, adds MTP speculative decoding for Qwen4Exp (~1.5x decode on DGX Spark), introduces the llama_batch_ext / llama_process() API for mixed token and embedding batches, adds few-row MMA mat-mul kernels on Metal (up to ~3x for speculative and batched decoding), and bumps ggml to v0.26.0. Session file formats are bumped too.

Key Takeaways

  • New models: GLM-5.3-Flash (GLM5-Next, 320B KDA/DSA hybrid text+vision MoE), Clef decision model, Ling 3.0 VL, LFM2.5-Encoder
  • Speed: Qwen4Exp MTP speculative decoding ~1.5x decode on DGX Spark; Metal few-row MMA mat-mul up to ~3x
  • API: llama_batch_ext + llama_process() for mixed token/embedding batches; server and examples migrated
  • Compat: LLAMA_SESSION_VERSION 11 and LLAMA_STATE_SEQ_VERSION 4, so old session/state files must be regenerated
  • Server: /v1/models reports modalities, /v1/embeddings takes typed vision/audio/video input, new /v1/systemone endpoint
Read full story
Linked Models & Agents:

llama.cpp ships Decision Models: /v1/systemone scores options in one forward pass (Julia-1, Laya, Kev-4B, lev, OpenJev)

On 2026-10-02 ggml-org announced llama.cpp server support for Decision Models via /v1/systemone (PR #29818, merged 2026-10-02T09:56Z). Send a state plus typed questions; get option probabilities in one forward pass. Day-0 models: Julia-1, Laya, Kev-4B, lev, OpenJev (vision). Cloudflare Clef is planned next.

llama.cpp ships Decision Models: /v1/systemone scores options in one forward pass (Julia-1, Laya, Kev-4B, lev, OpenJev)

Key Takeaways

  • Official post 2026-10-02 + merged PR #29818 (2026-10-02T09:56Z, +2139/−15)
  • Endpoint POST /v1/systemone with choice/score/noul; probabilities + confidence; output_tokens=0
  • Day-0 GGUFs: Julia-1 / Laya / Kev-4B / lev / OpenJev; median ~3–43 ms/question on RTX PRO 6000 (blog table)
  • Distinct from Ollama SystemOne & Cloudflare Clef hosting; Clef listed as next in llama.cpp
  • Start: llama serve -hf ggml-org/Kev-4B-GGUF then curl /v1/systemone
Read full story
Linked Models & Agents:
⚡ Julia 1⚡ laya⚡ Kev 4b⚡ lev⚡ openjev🌐 Ggml org🌐 Huggingface
ADSponsored

llama.cpp v0.5.0: CUDA conv2d implicit GEMM, Metal MoE/SSM fusion, MiMo-V2.6 & HunyuanOCR & Mimir 1B

llama.cpp v0.5.0 (2026-09-23) accelerates CUDA conv2d via implicit GEMM, adds Metal MoE and SSM_CONV fusion, lets the server bind multi-address/--host lists and UNIX sockets, and adds HRM-Text (DFM Mimir 1B), MiMo-V2.6, and HunyuanOCR (DFlash) conversion plus ggml 0.25.0 backends.

llama.cpp v0.5.0: CUDA conv2d implicit GEMM, Metal MoE/SSM fusion, MiMo-V2.6 & HunyuanOCR & Mimir 1B

Key Takeaways

  • Shipped v0.5.0 on 2026-09-23 — major step past the prior v0.4.1 line
  • Perf: CUDA conv2d implicit GEMM; Metal MoE + SSM_CONV fusion; CUDA graphs for MTP drafting
  • Models: HRM-Text/Mimir 1B, MiMo-V2.6 convert, HunyuanOCR DFlash, Qwen4Exp sparse FA, Nemotron MTP/H
  • Server: multi-address/--host lists + UNIX sockets; input_image on function-call outputs; router race fixes
  • API: LoRA from FILE*; sampler env vars; llama-bench --version
Read full story
Linked Models & Agents:
⚡ qwen⚡ deepseek⚡ nemotron🌐 Ggml org

llama.cpp v0.4.1 adds Maple/Hy4/Spark2.5, qwen3-coder parser fixes, and ggml 0.24.0

Official @ggml_org shipped llama.cpp v0.4.1: adds Maple 20B-A1B, Tencent Hy 4, and Spark2.5 support; improves Kimi-K3 rollback, JSON schema handling, qwen3-coder complex-type parsing, and MCP/multimodal fixes; updates ggml to v0.24.0. Release notes: ggml-org/llama.cpp v0.4.1.

Key Takeaways

  • Official llama.cpp v0.4.1 adds Maple 20B-A1B, Tencent Hy 4, and Spark2.5.
  • Improves qwen3-coder parsing, Kimi-K3 rollback, JSON schema, and multimodal/server stability.
  • ggml bumps to v0.24.0; notes on ggml-org/llama.cpp v0.4.1.
Read full story
Linked Models & Agents: