Thinking Machines Opens Inkling 2 Small Weights on Hugging Face Under Apache 2.0: 594B-Total / 17B-Active MoE, Native NVFP4 Runs on Two B300s, Day-0 Support from SGLang, vLLM, Cloudflare and More

Background and the Problem It Solves

On November 5, Thinking Machines Lab, the company founded by Mira Murati, launched Inkling 2 Small, the first model of its second generation. At launch it was only available for inference and fine-tuning on the lab's own Tinker platform, with a promise to release the weights under Apache 2.0 after external safety testing. A week later the lab delivered. Over the past year open-weight models from US labs have generally trailed Chinese releases in size and capability, so a frontier lab publishing a nearly 600B-parameter multimodal model in full matters; many in the community are calling it the strongest US open model so far (a community view, not an official ranking). The safety section of the model card says the team ran internal and external testing on CBRN, cyber and loss-of-control risks and concluded the model does not present material uplift beyond what the open-weight ecosystem already offers.

Architecture and How It Works

Per the model card, Inkling 2 Small is a 48-layer decoder-only transformer with a sparse MoE feed-forward backbone: each token is routed to 8 of 384 experts, for 594B total and 17B active parameters (Cloudflare's page says 18B; the model card is used here). Attention mixes local and global layers, and images are encoded by a vision transformer and projected into the same hidden space as text, with a context window of up to 1M tokens. The unusual part is precision: the model was pretrained, midtrained and post-trained in NVFP4, so NVFP4 is its native format and the main checkpoint is about 394GB. An MXFP8 checkpoint is also published, but the lab says its 1-2 point eval gain is not meaningful in practice.

Benchmarks and Measured Results

All figures below are vendor-reported numbers from the model card. On third-party indexes it scores 30.6 on the Artificial Analysis Intelligence Index v5.0 and 56.1% on Vals Index v2.2. For coding it posts 72.1% on DeepSWE v1.1, 25.4% on FrontierCode v2.0 and 61.9% on SWE-Atlas v1.1. For search and tools it reaches 90.2% on BrowseComp and 85.4% on MCP Atlas. In the same table DeepSeek-V4.1-Flash still leads on AA Index (32.6) and DeepSWE (77.0%), and the closed Claude Haiku 5.5 reaches 36.5 on AA Index; Inkling's edge is mainly BrowseComp, JobBench (70.5%) and enterprise operations tasks. The SGLang team reports 576 tokens per second per user at batch 1 on NVIDIA Vera Rubin (framework-reported).

Getting Started for Developers

Self-hosting: download the NVFP4 weights from Hugging Face and start them with the official SGLang or vLLM recipe on at least two B300s or four H200s; Unsloth and TokenSpeed also have recipes. For further training use Tinker, Miles, SkyRL or Snowflake Cortex Training. If you would rather not run it yourself, call @cf/thinkingmachines/inkling-2-small on Cloudflare Workers AI, which requires the Workers Paid plan and costs $0.30 per million input tokens, $1.00 per million output tokens and $0.10 per million cached input tokens; OpenRouter charges the same and offers a free variant, and Together, Baseten, Nebius, Modal, Fireworks and NVIDIA are live too. Read the lab's acceptable use policy before deploying.