NVIDIA Releases Dynamo Deployment Recipes for Four Korean Open Models: A.X K2, K-EXAONE 2.0, Solar Open 2 and Motif-3 on B200, Claiming Up to 2x Inference Performance

Background and Industry Pain Points

Korea's AI for All program backs a set of homegrown foundation models, but MoE models with hundreds of billions of parameters do not fit on one GPU. They must be split across many GPUs, with request routing, KV cache transfer and parallelism tuned by hand, and many teams stall between open weights and production. At NVIDIA AI Day Seoul on November 9, NVIDIA released Dynamo-optimized deployment recipes for SK Telecom's A.X K2, LG AI Research's K-EXAONE 2.0, Upstage's Solar Open 2 and Motif Technologies' Motif-3, alongside an expanded sovereign AI collaboration with Korea's Ministry of Science and ICT (MSIT).

Architecture Highlights

Dynamo is NVIDIA's open-source distributed inference-serving framework that handles routing and communication across nodes. All four recipes sit under recipes/ in the ai-dynamo/dynamo GitHub repo and use NVFP4 weights, FP8 KV cache and a vLLM backend. Each ships aggregated and disaggregated (prefill/decode split) Kubernetes manifests, a model-cache download job and an AIPerf benchmark job. Details vary by model: Motif-3 uses TP2 with expert parallelism and MTP2 multi-token prediction; A.X K2 uses TP4 with EAGLE3 speculative decoding and adds a 32x H200 option; Solar Open 2 enables KV-aware routing in aggregated mode.

Benchmarks and Comparisons

NVIDIA's headline is up to 2x inference performance out of the box on B200, with no baseline named. The Dynamo docs give concrete numbers (chat trace, SLA of at least 50 tok/s per user and TTFT p50 under 5 s): Motif-3 on two B200s at 293.96 output tok/s per GPU with 388 ms TTFT; Solar Open 2 with two replicas at 216.88 tok/s per GPU with 308 ms TTFT; K-EXAONE 2.0 on four GPUs at 87 tok/s per GPU with 291 ms TTFT; A.X K2 on eight GPUs at 76.49 tok/s per GPU with 0.525 s TTFT. Motif-3 and A.X K2 results use synthetic speculative-decoding acceptance and do not measure real quality. Independent third-party results have not been published.

Developer Hands-on Guide

Teams with B200 clusters can clone ai-dynamo/dynamo, open the matching folder under recipes/ (ax-k2, k-exaone-2.0, solar-open2-250b, motif-3), run the model-cache job to fetch weights, apply the manifests under vllm/, then reproduce the benchmarks with the jobs in perf/. Per the docs, pick aggregated serving for low concurrency and fast first tokens, and disaggregated serving when prefill and decode must scale separately; for K-EXAONE 2.0 the disaggregated target matches aggregated throughput per GPU while using twice the GPUs. Solar Open 2 defaults to high reasoning effort, so a small max_tokens can cut it off mid-reasoning, especially for structured output.