Tencent Hunyuan open-sources Hy-SimulMT, a Chinese-English real-time simultaneous interpreting model with ~3s lag and live voice cloning
On Oct 26 Tencent Hunyuan (Tencent Hy) open-sourced Hy-SimulMT, a real-time simultaneous interpreting model for bidirectional Chinese-English speech-to-speech translation. It translates while the speaker talks, outputs both translated text and speech, and can clone the speaker's voice in real time. Six checkpoints landed on Hugging Face (30B-A3B translation-only and source-plus-translation Interleave variants, each in BF16, FP8 and GGUF), with the model card listing Apache-2.0; GitHub ships the SimulInfer serving stack and SimulTrain training tools. Vendor-reported: about 3s StreamLAAL on RealSI long-form audio, the lowest among compared systems.
Key Takeaways
- Size and variants: 30B-A3B MoE (about 3B active), six checkpoints: translation-only and Interleave (source plus translation), each in BF16, FP8 and GGUF (Hugging Face collection)
- Latency (vendor-reported): about 3s StreamLAAL on RealSI long-form audio, the lowest among compared systems, with a GEMBA quality score above the compared open models and 4 of 5 commercial APIs
- Short-form (vendor-reported): FLEURS 90.3, CoVoST2 75.5, ACL6060 81.2, RealSI-Sentence 84.7, the last being the top score in the table; Qwen3.8-LiveTranslate scores 88.4/82.1/80.5/84.4 in the same table
- Human comparison: 18 real Chinese and English recordings of about 6 minutes each against 6 CATTI Level 1/2 interpreters; Tencent says quality was on par with often lower latency, and notes the result only covers this small test
- Deployment: one SimulInfer deploy.sh command starts vLLM (>=0.31) plus a web/WebSocket service; speech output and voice cloning need a separate TTS service; llama.cpp support is not yet merged upstream (SimulInfer docs)
Linked Models & Agents:
