DiffuSpace, founded by the HKU Dream team, open-sources Clover-35B-A3B, a diffusion LLM hitting 680 tokens/s on one GPU and 73.6 on SWE-bench Verified

Background and the problem

Almost every mainstream LLM today is autoregressive and emits one token at a time. For agents, every step waits for the model to finish its output token by token, and that latency adds up across a task. Diffusion language models (dLLMs) take a different approach: they start with a block of masked tokens and refine them in parallel, which can in principle be much faster. The catch has been that open diffusion models were small and weak. In April 2025 the HKU NLP group released Dream 7B, showing a diffusion model could compete with autoregressive models at small scale. On Oct 28 DiffuSpace, a company started by members of that team, released its first model, Clover-35B-A3B-Preview, to test whether diffusion models keep their quality and speed edge at larger scale.

Architecture and how it works

Clover is initialized from Qwen3.6-35B-A3B and keeps its hybrid Gated DeltaNet (GDN) and full-attention MoE backbone: 40 layers, 256 routed experts with top-8 routing and one shared expert per layer, 35B total and about 3B active parameters, and a 128K context window. It generates block by block (default block size 32). Each block starts fully masked, positions whose confidence exceeds a threshold (default 0.95) are filled in parallel, and a finished block becomes the prefix for the next one. The team rebuilt GDN to be bidirectional: earlier context is compressed in the GDN recurrent state, tokens inside the current block interact through bidirectional softmax attention, and a learnable block-size calibration plus a zero-initialized right-side convolution branch were added. During denoising, tokens in the same block can also share a subset of experts, which cuts the number of experts touched.

Benchmarks

All figures below are vendor-reported, with no independent replication yet. On six agent benchmarks DiffuSpace says Clover scores highest among the models compared: 73.6 on SWE-bench Verified, 51.7 on Terminal-Bench 2.1 and 65.3 on tau3 Overall, against the diffusion models LLaDA2.2 flash and Mercury 2.5 and the same-size autoregressive Qwen3.6-35B-A3B and Qwen3.5-35B-A3B. General results include 83.8 on GPQA Diamond, 95.8 on ZebraLogic, 72.0 on AA-LCR and 84.8 on MMLU-Pro. On speed, it reaches 680 tokens/s overall in internal agent workloads on a single NVIDIA GB300 at BF16 and batch size 1, and 515 tokens/s on a single H200 versus 365 tokens/s for a same-size Qwen with DFlash2 acceleration. Across 49 WebMCP tasks, median agent execution time was 1.52 seconds. Community members have asked how much of the agent gain comes from diffusion decoding itself versus extra post-training on top of Qwen3.6, which DiffuSpace has not broken down.

Getting started

Weights are Apache 2.0 on Hugging Face, ModelScope and GitHub. Running it in Transformers needs trust_remote_code, and the BF16 weights are about 69GB, so a single large GPU is required; DiffuSpace recommends its own SGLang implementation for serving and evaluation, and the GitHub repo also ships a llama.cpp port and self-speculative decoding. For local use, an INT4 build runs on a 16GB RTX 5070 Ti at 101.8 tokens/s on MATH-500. The model card warns that very long generations can repeat and ships default repetition and end-of-response controls. The team says a decision model, an agent harness and a desktop coding client will follow, and online access is by application at diffuspace.ai.