Thinking Machines Details End-to-End 4-Bit Training for Inkling 2 Small: NVFP4 Forward Passes in Expert Layers From Pretraining Through RL, 8% Faster Pretraining and 20% Faster RL, With the 594B Model Pretrained in 68 Hours
In a Connectionism blog post, Thinking Machines explains how Inkling 2 Small used a native 4-bit (NVFP4) forward pass from the first pretraining step through the final RL rollout, with MXFP8 backward passes using stochastic rounding and quantized communications. The lab calls it the largest model it knows of trained end-to-end with a native 4-bit forward pass, with an 8% wall-clock speedup in pretraining and 20% in RL over full MXFP8 training.
Key Takeaways
- NVFP4 is used only for forward passes in the MoE expert layers; backward passes use MXFP8 with re-quantization and stochastic rounding to limit gradient bias
- The same numerics run from pretraining through SFT and RL, so the quantized checkpoint needs no calibration at the end of training
- Versus full MXFP8: 8% faster pretraining wall-clock and 20% faster RL; pretraining costs about 2% compute efficiency and RL shows no discernible reward difference
- Expert-parallel communication became the new bottleneck, so the team quantizes the combine step after a 32-dimensional random Hadamard transform into a block-scaled format
- The lab says a similar-size run took several weeks months ago; Inkling 2 Small was pretrained in 68 hours on GB300 NVL72 racks
Linked Models & Agents:
