On 2026-10-23 (15:41 UTC) the official @NVIDIAAI account announced Nemotron 3.5 Bolt E4B 4GB, a small open model for local assistants, interactive characters and robots that can hold a conversation and use tools. The official Hugging Face model card says it is post-trained from Gemma-4-E4B-it, has 8B total parameters (4B effective), ships in Q4_K_M quantization, supports up to 64K input and 64K output tokens, and has a switchable thinking mode, plus a DFlash speculative-decoding drafter and a BF16 version. Weights, data and recipes are open under the OpenMDW-1.1 license, with commercial use allowed. On the model card the 4GB build scores 55.60 on MultiChallenge (Gemma4 E4B-it at the same quantization 39.36, Qwen3.5 4B 52.23), but trails Qwen3.5 4B on Tau2-Bench (52.32) and BFCL-v3 (67.31).
Key Takeaways
- ✓NVIDIA officially released Nemotron 3.5 Bolt E4B 4GB for local assistants, interactive characters and robots
- ✓Post-trained from Gemma-4-E4B-it: 8B total parameters (4B effective), Q4_K_M quantization, 64K context, switchable thinking mode
- ✓Open weights, data and recipes under the OpenMDW-1.1 license with commercial use allowed, plus a DFlash speculative-decoding drafter
- ✓Model card results: 55.60 on MultiChallenge leads its size class, but it trails Qwen3.5 4B on Tau2-Bench and BFCL-v3
Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Background and pain point
Local voice assistants, game characters and robots need fast replies on devices with very little memory, while still handling multi-turn conversation and tool calls. Large models don't fit, and generic small models often stumble on tool use and instruction following. On October 23 (15:41 UTC) NVIDIA AI announced Nemotron 3.5 Bolt E4B 4GB, the smallest member of the Nemotron 3.5 Bolt family (the 24GB Bolt 74B-A9B was open-sourced in mid-October).
How it works
According to the official Hugging Face model card, Bolt E4B is post-trained from Google's Gemma-4-E4B-it, with 8B total parameters and 4B effective. The 4GB build uses Q4_K_M quantization and supports up to 64K input and 64K output tokens. NVIDIA focused on multi-turn tool use, instruction following and input grounding, and the chat template exposes switches for thinking mode and conversational mode, with a runtime thinking budget. NVIDIA also ships a DFlash speculative-decoding drafter and a full-precision BF16 version. Weights, training data and recipes are open under the OpenMDW-1.1 license, and commercial use is allowed.
Benchmarks
These are the model card results for the Q4_K_M build: MultiChallenge 55.60 (Gemma4 E4B-it 39.36, Qwen3.5 4B 52.23); RoleBench 71.21 (60.51, 52.35); IHEval 82.40 (66.61, 62.87). The weak spots are listed too: Tau2-Bench 52.32 versus 61.74 for Qwen3.5 4B, BFCL-v3 67.31 versus 70.56, and RULER 64K 85.76 versus 95.37. Overall it is strong at conversation, role-play and instruction hierarchy, but behind Qwen3.5 4B on complex tool chains and long-context retrieval.
What developers can do now
The model card includes a llama.cpp Docker recipe: download Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf from Hugging Face, optionally add the DFlash drafter for speculative decoding, start the OpenAI-compatible server, and toggle reasoning with enable_thinking. It suits local voice assistants, NPCs and robot conversation front ends; if your workload is mostly complex multi-step tool calling, compare it with Qwen3.5 4B on your own tasks first.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.