AI Coding Live News

Zyphra Open-Sources ZONOS3-Mini TTS: 800M-Parameter MoE With 100M Active, Apache 2.0, Real-Time in a Phone Browser, Word-Level Delivery Control and Voice Cloning

Zyphra released ZONOS3-Mini, an open-source text-to-speech model, on November 9, with weights on Hugging Face under Apache 2.0. It is a sparse MoE with 800 million total and about 100 million active parameters per token, trained on 112 H100s in 2.5 days. By Zyphra's numbers, time to first audio is under 570 ms in the browser on a Pixel 9 and under 150 ms on an M3 MacBook Air, and it ranks first among open models on the capability index of its new ZTTS2-Eval benchmark (35 languages of read speech plus 20 in-the-wild), ahead of Fish S2 Pro and MOSS TTS v1.5. It supports voice cloning, age, accent and gender steering, and word-level delivery control, with WebGPU, SGLang, vLLM, GGML and MLX runtimes.

Zyphra Open-Sources ZONOS3-Mini TTS: 800M-Parameter MoE With 100M Active, Apache 2.0, Real-Time in a Phone Browser, Word-Level Delivery Control and Voice Cloning

Key Takeaways

  • Scale: 800M total, about 100M active parameters (64 experts, top-2 routing), versus 8B for the previous ZONOS2; open weights under Apache 2.0
  • Speed (Zyphra figures): under 570 ms time to first audio in a Pixel 9 browser and under 150 ms on an M3 MacBook Air; 80 ms median at 8 concurrent requests and peak 457 seconds of audio per second on one H100
  • The new ZAC-1 codec runs at 15 Hz versus 86 Hz for DAC and cuts word error rate about 10x under similar data; one generation can produce up to 10 minutes of 44.1 kHz audio
  • Evaluation: ZTTS2-Eval covers 35 languages of read speech plus 20 in-the-wild; on its 420-item capability index ZONOS3-Mini ranks first among open models, followed by Fish S2 Pro and MOSS TTS v1.5 (Zyphra's own benchmark, not yet independently reproduced)
  • Get started: try it in the browser via WebGPU with no install, or run sgl-omni serve --model-path Zyphra/ZONOS3-Mini for an OpenAI-compatible /v1/audio/speech endpoint
Read full story
Linked Models & Agents:
⚡ ZONOS3-Mini🌐 Zyphra

Tencent Hunyuan open-sources Hy-SimulMT, a Chinese-English real-time simultaneous interpreting model with ~3s lag and live voice cloning

On Oct 26 Tencent Hunyuan (Tencent Hy) open-sourced Hy-SimulMT, a real-time simultaneous interpreting model for bidirectional Chinese-English speech-to-speech translation. It translates while the speaker talks, outputs both translated text and speech, and can clone the speaker's voice in real time. Six checkpoints landed on Hugging Face (30B-A3B translation-only and source-plus-translation Interleave variants, each in BF16, FP8 and GGUF), with the model card listing Apache-2.0; GitHub ships the SimulInfer serving stack and SimulTrain training tools. Vendor-reported: about 3s StreamLAAL on RealSI long-form audio, the lowest among compared systems.

Tencent Hunyuan open-sources Hy-SimulMT, a Chinese-English real-time simultaneous interpreting model with ~3s lag and live voice cloning

Key Takeaways

  • Size and variants: 30B-A3B MoE (about 3B active), six checkpoints: translation-only and Interleave (source plus translation), each in BF16, FP8 and GGUF (Hugging Face collection)
  • Latency (vendor-reported): about 3s StreamLAAL on RealSI long-form audio, the lowest among compared systems, with a GEMBA quality score above the compared open models and 4 of 5 commercial APIs
  • Short-form (vendor-reported): FLEURS 90.3, CoVoST2 75.5, ACL6060 81.2, RealSI-Sentence 84.7, the last being the top score in the table; Qwen3.8-LiveTranslate scores 88.4/82.1/80.5/84.4 in the same table
  • Human comparison: 18 real Chinese and English recordings of about 6 minutes each against 6 CATTI Level 1/2 interpreters; Tencent says quality was on par with often lower latency, and notes the result only covers this small test
  • Deployment: one SimulInfer deploy.sh command starts vLLM (>=0.31) plus a web/WebSocket service; speech output and voice cloning need a separate TTS service; llama.cpp support is not yet merged upstream (SimulInfer docs)
Read full story
Linked Models & Agents:
⚡ Hy-SimulMT-30B-A3B⚡ Hy-SimulMT-30B-A3B-Interleave⚡ Qwen3.8-LiveTranslate🌐 Tencent

Mistral Releases Voxtral 3 Audio Model Suite and Real-Time Voice-Agent API with Sub-500ms Latency

Mistral AI unveiled Voxtral 3, a production audio intelligence suite paired with a real-time Voice-Agent API. The lineup features the 4B-parameter Mini Transcribe, Small Transcribe 3, and a turn-key voice agent SDK built on LiveKit. Supporting transcription across 50 languages with diarization, multi-lingual TTS with emotional tags across 35+ languages, and 30-minute voice cloning, Voxtral 3 redefines voice AI economics. In Artificial Analysis independent benchmarks, Small Transcribe 3 secured rank #3 globally with a 2.1% error index (outperforming ElevenLabs Scribe v2), while Realtime Transcribe delivers sub-500ms latency at $0.04/min for full voice agent turns.

Mistral Releases Voxtral 3 Audio Model Suite and Real-Time Voice-Agent API with Sub-500ms Latency

Key Takeaways

  • Mistral AI launched the Voxtral 3 suite spanning transcription, emotional TTS, studio-grade voice cloning, and a LiveKit-powered Voice-Agent SDK
  • Artificial Analysis independently verified Small Transcribe 3 at a 2.1% error index, outperforming ElevenLabs Scribe v2 (2.2%) and Grok Voice 2.0 (2.3%)
  • Realtime Transcribe 3 achieves sub-500ms latency, delivering sub-second full conversational voice-agent turns
  • Aggressive pricing: Mini Transcribe starts at $0.0035/min, and managed voice agent sessions cost just $0.04/min paired with Mistral Small 4
Read full story

Mistral launches the Voxtral 3 speech model family and a voice agent SDK: transcription in 50+ languages, TTS at $13 per 1M characters, EU-hosted or self-deployed

On Oct 20 Mistral released Voxtral 3: the Voxtral Small/Mini Transcribe 3 speech-to-text models, Voxtral Mini Transcribe Realtime 3, the Voxtral Mini TTS 3 (Voxtral Speech 3) text-to-speech model, and a LiveKit-based voice agent SDK that streams listening, reasoning with tool calls, and speaking as one pipeline, with each turn completing in under a second per Mistral. Transcription covers 50+ languages with diarization and context biasing; TTS covers 35+ languages and 50+ dialects, plus Pro Voice cloning from 30 minutes of audio. It is live on the Mistral API (EU, US and global endpoints) and can be self-deployed.

Mistral launches the Voxtral 3 speech model family and a voice agent SDK: transcription in 50+ languages, TTS at $13 per 1M characters, EU-hosted or self-deployed

Key Takeaways

  • Multilingual transcription (vendor-reported): macro WER over 45 FLEURS languages is 4.1% (language forced) / 4.0% (inferred) for Voxtral Small Transcribe 3, tied with Microsoft AI Transcribe 2 and ahead of Scribe v2 (4.9%) and Whisper large-v3 (16.0%/14.9%)
  • Diarization (vendor-reported): cpWER of 18.4% for Small and 18.6% for Mini, vs 30.9% for the previous Voxtral Mini Transcribe 2 and 38.4% for Gemini 3.5 Transcribe
  • Pricing: Voxtral Mini Transcribe 3 is 4B parameters at $0.0035/min (docs list it as Public Preview with 32k context); Voxtral Speech 3 costs $13 per 1M characters
  • Realtime: Voxtral Mini Transcribe Realtime 3 runs below 500 ms latency; in Mistral's 35-language FLEURS chart the 480 ms setting scores 6.4% WER, still behind Gemini 3.5 Realtime Transcribe at 4.2%
  • The voice agent SDK is built on LiveKit with SIP telephony and WebRTC; each stage (LLM, voice, language) is swappable, and Pro Voice clones a voice from 30 minutes of audio
Read full story
Linked Models & Agents:
⚡ Voxtral Small Transcribe 3⚡ Voxtral Mini Transcribe 3⚡ Voxtral Mini Transcribe Realtime 3⚡ Voxtral Mini TTS 3🤖 Voxtral voice agent SDK🌐 Mistral AI🌐 LiveKit
ADSponsored

Ant Group Open-Sources Ming-3.0-tts: 93.2% Pronunciation Accuracy Leading Global Open Speech Models

In the latest independent Text-to-Speech (TTS) evaluations published by Artificial Analysis, Ant Group's open-source Ming-3.0-tts achieved a 93.2% pronunciation accuracy score, ranking #1 among all 31 evaluated models (outperforming commercial leader ElevenLabs v4 at 91.7%). Offering both a dense 0.5B parameter variant and a 7.9B mixture-of-experts (MoE) model, it delivers voice cloning and preset voices with a Controlled Voice Elo score of 1,069 (#1 among open-weight models), fully released under the permissive MIT license.

Ant Group Open-Sources Ming-3.0-tts: 93.2% Pronunciation Accuracy Leading Global Open Speech Models

Key Takeaways

  • Ant Group open-sourced Ming-3.0-tts, achieving a chart-topping 93.2% pronunciation accuracy on Artificial Analysis speech benchmarks
  • Surpasses commercial leader ElevenLabs v4 (91.7%) and ranks #1 among open-weight models on the Controlled Voice leaderboard (1,069 Elo)
  • Provides both a lightweight 0.5B dense model for edge inference and a 7.9B MoE flagship with zero-shot voice cloning
  • Released under the permissive MIT license for unrestricted commercial deployment, hosted on Hugging Face
Read full story
Linked Models & Agents:

Alibaba Qwen launches Qwen3.8-LiveTranslate for 60-language real-time interpretation

Qwen announced Qwen3.8-LiveTranslate, its next-generation real-time simultaneous interpretation model. An Interleave architecture improves faithfulness, fluency, and conciseness while cutting average lagging (LAAL) from 2.8s to 2.3s across 60 languages. New capabilities include real-time speaker diarization with more stable voice cloning, synchronized bilingual display, and long-context disambiguation for names and terminology.

Key Takeaways

  • Interleave architecture cuts average lagging (LAAL) from about 2.8s to 2.3s across 60 languages.
  • Real-time speaker diarization keeps each speaker's voice more stable in multi-party speech.
  • Adds synchronized bilingual display and long-context term disambiguation; try it on QwenCloud.
Read full story
Linked Models & Agents:
⚡ Qwen3 8 Livetranslate🌐 alibaba🌐 Qwen (通义千问)