NVIDIA open-sources Nemotron 3.5 Transcribe, a 30B MoE speech model with about 4B active parameters that handles transcription in 40 languages, speaker attribution and translation, and tops open models on the Open ASR English leaderboard

Background and the problem

Speech models that write down every word are no longer rare. The hard part comes after: who said what in a meeting, how to translate it, and how to spell domain terms correctly. That usually means chaining separate ASR, diarization and translation models, each deployed and failing on its own. Nemotron 3.5 Transcribe, which NVIDIA open-sourced on Oct 28, aims to handle all of it in one checkpoint you steer with plain-language instructions.

Architecture and how it works

• NVIDIA calls it a speech-augmented language model (SALM): a speech encoder and a speaker encoder extract features in parallel and feed a Mamba-2/Transformer hybrid MoE language model, built on Nemotron 3.5 Lightning 30B-A3B.
• Speaker cues reach the language model together with the acoustic features, so it labels each turn while generating instead of aligning speakers afterwards.
• Only about 4B parameters are active per token (4.3B of 34B in the model card, counting the encoders), so it runs on a single NVIDIA GPU from DGX Spark to Blackwell data-center systems; context is 65,536 tokens and recordings can run up to an hour.
• Instructions switch the task, for example "transcribe this recording and label each speaker", "translate this audio into English" or "what decisions were made in this meeting?", and prompts can list candidate terms.

Benchmarks

Figures below come from NVIDIA and Artificial Analysis. On the Hugging Face Open ASR English leaderboard it is first among open models at 5.33% average WER; it posts 4.62% mean WER across 7 languages and 10.98% on long-form audio, the lowest among open models in the model card's comparison. Artificial Analysis measured a 2.8% AA-WER, tied for first among open models with Voxtral Small (24B active, 2.77%) at about one-sixth the active parameters, and 1.8% on its AA-AgentTalk voice-agent set versus 2.3% for the next-best open model. For speaker attribution, NVIDIA says it ranks first on David AI's 21-language leaderboard and in VoiceArena's 14-system comparison. Artificial Analysis also measured DeepInfra's hosted endpoint at a 75x speed factor and $2.00 per 1,000 minutes.

Getting started

1. Download BF16, FP8 or NVFP4 weights from the Hugging Face model page, and try a recording in the official live demo first.
2. Start from the prompt presets in the model card; for unusual terms, list candidates in the prompt before reaching for fine-tuning.
3. For domain adaptation, fine-tune with Granary v2 data and NeMo Speech tools, as described in NVIDIA's Hugging Face blog post.
4. Language support differs by task and audio Q&A is trained in English only; NVIDIA also advises spot-checking names, speaker changes and answers against the recording.