On 2026-11-06 Reflection's official account released the Beam model technical report (a 156-page PDF), detailing how its 501.3B-total / 23.4B-active open-weight MoE was trained: 23.8T pretraining tokens on 6,144 GB300 GPUs in under four weeks, then a fully asynchronous RL run on about 10.5K GB300s for four weeks that produced over 100M rollouts. The report says FP8 and NVFP4 checkpoints keep 99.93% and 98.98% of median benchmark scores, and the 1.93B DFlash 2 drafter speeds single-request decoding 1.96–3.03x (vendor figures).
Key Takeaways
- ✓Scale: 501.3B total / 23.4B active, 52 layers, 256 experts per layer with 8 active per token, 151,936-token vocabulary
- ✓Pretraining: 23.8T tokens on 6,144 GB300s in under four weeks, 92.3% goodput late in the run and 21% steady-state MFU
- ✓RL: about 10.5K GB300s for four weeks, 100M+ rollouts, 110K concurrent rollouts and 12-second weight updates; the report sees no plateau as RL compute grows
- ✓Serving: FP8 / NVFP4 keep 99.93% / 98.98% of median scores; DFlash 2 speeds single-request decoding 1.96-3.03x
- ✓Safety: 88.9% refuse/pivot on harmful requests and 81.5% compliance on harmless ones in in-house ISE; Limited in all four systemic-risk categories
Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Background and Industry Pain Point
Beam's weights went out under Apache 2.0 in October, but what the open ecosystem lacked was reusable training detail. On November 6 Reflection published a 156-page technical report covering data, midtraining, RL and cluster operations, which is rare for a Western open lab.
Architecture and Mechanism
Sparse MoE: 52 layers (3 dense + 49 MoE), 256 experts per layer with 8 routed per token and no shared experts; local and global attention interleaved 5:1 with a 512-token sliding window and NoPE on global layers. Pretraining used 23.8T tokens on 6,144 GB300s in under four weeks; a two-stage midtraining extended context to 1M. Post-training was a fully asynchronous RL run on about 10.5K GB300s for four weeks with over 100M rollouts, followed by multi-teacher on-policy distillation merging the agentic and alignment teachers.
Benchmarks
Report tables (vendor figures): SWE-bench Verified 81.6%, Terminal-Bench 2.1 80.1%, SWE-Bench Pro v1 66.1%, GPQA Diamond 90.6%, HLE (no tools) 36.0%. In the same table GLM 5.3 and Kimi K3 post 88.2% and 88.3% on Terminal-Bench 2.1, so Beam does not lead everywhere; its pitch is lower inference compute. FP8 / NVFP4 keep 99.93% / 98.98% of median scores.
Developer Guide
For self-hosting, read sections 4.5-4.6: NVFP4 targets Blackwell, and DFlash 2 (1.93B, 7 drafted tokens) gives 1.96-3.03x single-request and 1.89-2.75x 32-concurrency decode speedups on four B300s. Weights are on Hugging Face. Teams building RL infrastructure should focus on section 4.4 and chapter 6 for the async design and fault recovery.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.