On 2026-10-15 Microsoft AI published a technical post and PDF saying it has finished pre-training its in-house base model MAI-Base-2: MoE in every layer (1 shared + 7-of-512 routed experts), 80B active / 2.8T total, 102 layers, native encoder-free vision, a 50T-token mixture and a March 2026 knowledge cutoff. Per MAI, it beats every open-weight model tested on held-out BPB and its recipe is 4.2–12.3x more pre-training-FLOPs-efficient than DeepSeek-V4-Pro-Base. It is still in post-training: no API, no weights and no release date yet; Microsoft plans to license weights to trusted partners via Foundry.
Key Takeaways
- ✓Scale: 80B active / 2.8T total, 102 layers, hidden 8704, MoE in every layer (1 shared + 7 of 512 routed); MAI-Base-1 was 35B/1T, 78 layers, alternating dense/MoE (PDF)
- ✓Compute: 16K GB200s for 38 days (Aug–Sep 2026), 21% average MFU (vs 2.5 PFLOPS BF16), 89% goodput; the Muon step is 2.1% (165 ms) of iteration time
- ✓Efficiency (vendor estimate via scaling-law fits): 4.2–12.3x fewer FLOPs than DeepSeek-V4-Pro-Base to match its BPB; 6.3–12.7x better than the June MAI-Base-1 recipe
- ✓Data: 50T-token mixture, 18% vision-language (4.5T unique), ~36% more unique tokens than Base-1, described as organic data with no distillation
- ✓Availability: still in post-training, no API/weights/date; weights to be licensed to trusted partners through Foundry; MAI-Base-3 prep underway (blog)
Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Background
Microsoft has relied heavily on OpenAI models; its in-house track began with MAI-Base-1 (35B active / 1T total) and MAI-Thinking-1 in June, still below frontier scale. On 2026-10-15 Microsoft AI published a technical blog and PDF announcing that MAI-Base-2 has finished pre-training.
Architecture and training
- MoE in every layer: 80B active / 2.8T total, 102 layers, hidden size 8704, 1 shared + 7-of-512 routed experts. A multi-round dropless MoE keeps memory fixed under any routing imbalance without dropping tokens, raising MoE-layer MFU from 24% to 35%.
- Muon at scale: momentum is resharded for Newton–Schulz orthogonalisation; the optimizer step costs 2.1% (165 ms) of iteration time. Activation gradient clipping replaces purely global clipping and also surfaced a preprocessing bug.
- Scaling science: "source scaling" keeps data repetition constant across ladder rungs; fitted laws such as LR×WD = 17.04/steps set the final hyperparameters.
- Run: 16K GB200s, 38 days in Aug–Sep 2026, 21% average MFU (normalised to 2.5 PFLOPS BF16), 89% goodput; 50T-token mixture with 18% vision-language data (encoder-free VLM), March 2026 cutoff.
Benchmarks (vendor-reported)
Only pre-training bits-per-byte on held-out internal code and vendor-commissioned STEM/Math/QA sets: MAI-Base-2 is lower than DeepSeek-V4-Pro-Base, Kimi K3, Qwen3.8-2.4T-A95B and MiMo-V2.6-Pro-RL (the latter three are post-trained checkpoints, a caveat MAI notes). Scaling-law fits put the recipe at 4.2–12.3x more FLOPs-efficient than DeepSeek-V4-Pro-Base and 6.3–12.7x better than MAI-Base-1. No downstream or third-party evals yet; press reports of beating an OpenAI base model come from private tests.
What developers can do
Nothing to call or download yet: the model is in post-training, and Microsoft plans to license weights to trusted partners via Foundry with no date. Training teams can borrow the dropless-MoE scheduling, the (log LR, log LR×WD) sweep grid and activation gradient clipping from the PDF; app builders should wait for post-training results and a Foundry listing.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.