SenseTime open-sources SenseNova-Loop-9B-Preview, a 9B looped Transformer with about 14.6B inference parameters for long-context and agentic work

Background and pain point

Small models that need to handle long agent tasks usually get there by adding parameters or switching to MoE, which raises memory and serving costs. Looped Transformers take another route: run the same layers more than once to buy compute depth without storing more weights. There has been plenty of research on looped models, but few open-weight ones are ready for agent workflows. SenseTime's SenseNova-Loop-9B-Preview, open-sourced on Oct 30, turns that idea into a deployable multimodal model aimed at long context, document understanding, coding, and tool use.

Architecture

The model has 32 distinct decoder layers. Input goes through layers 0 to 3, then the block of layers 4 to 27 runs twice with shared weights, then layers 28 to 31 and the output head. That is 56 layer computations per forward pass with only 32 layers of weights stored; SenseTime lists about 9B stored parameters and about 14.6B at inference. Inside the loop, two-stream mHC (manifold-constrained hyper-connections) carries the residual state, with stream mixing constrained by 20 Sinkhorn iterations. Attention is hybrid: three linear-attention layers and one full-attention layer per four-layer group, with loop boundaries aligned to those groups.

Benchmarks

All figures below come from SenseTime's own chart; there is no third-party replication and the technical report is not out. It leads on BIRD-Dev 71.38 (Qwen3.5-27B 69.84), WorkBuddy Bench 66.89 (Qwen3.6-35B-A3B 61.98), AIDABench 58.87 (57.19), TeamBench 48.89 (44.44), and AutomationBench 13.72 (10.17). It trails on Terminal Bench 2.1 at 40.45 versus 47.19 for Qwen3.6-27B, and on tau3-Banking at 18.56 versus 19.59 for Qwen3.5-27B. RaccoonPPTBench at 70.70 is an internal SenseTime test set.

Getting started

Weights are on Hugging Face under Apache 2.0. The checked environment is Transformers 5.18.0 with PyTorch 2.11 on CUDA 13.0; load with trust_remote_code=True, and set enable_thinking=True for thinking mode. For serving, use LightLLM (the official example uses 4 GPUs) or an SGLang build from PR 47281; fine-tuning scripts are in OpenSenseNova/torchtitan. This is a preview, and the custom model code should be reviewed before you run it; test it on your own agent tasks before production use.