On Oct 28 Cartesia chief scientist Albert Gu (co-author of Mamba) and team published two research posts on a new generation of hierarchical networks (H-Nets). In a controlled scaling ladder, swapping BPE tokenization for learned dynamic chunking alone gave up to 3-5x compute efficiency. A larger two-stage byte-level H-Net (3.67B parameters), compute-matched per byte against SmolLM3-3B, reached SmolLM3’s final average downstream score after about 1T tokens versus 11T (vendor-reported). The new architecture is not released yet; Cartesia says it plans to release it over time.
Key Takeaways
- ✓Controlled ladder (vendor-reported): a 1-stage H-Net as a drop-in replacement for BPE gave about 1.3x compute efficiency on validation loss, about 2x on code, and up to 3-5x overall
- ✓Larger run: a 3.67B two-stage byte H-Net vs 3.08B SmolLM3, compute-matched within 2% FLOPs per byte, matched SmolLM3’s six-task average after about 1T tokens instead of 11T
- ✓Code gains are larger: 1.49x-2.40x (median 2.04x) on 15 in-distribution languages, median 3.10x on 10 out-of-distribution languages, up to 5.37x on Prolog
- ✓Training mix is 70% English, 20% code, 5% math and 5% multilingual; the main network sees about one position per 7.6 bytes, and the outer encoder and decoder take about 43% of FLOPs
- ✓No weights or code yet; Cartesia says it will release some of these architectures over time; the original H-Net paper is arXiv 2507.07955
Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Background and the problem
Almost every LLM today relies on a hand-designed tokenizer such as BPE that splits text into a fixed vocabulary before training. Tokenizers behave unevenly on code, many languages and unusual formats. Albert Gu, Cartesia’s chief scientist and co-author of Mamba, posted on X that learning language end-to-end from bytes is a central test of general learning methods.
Architecture and mechanism
H-Nets (hierarchical networks) were introduced in 2025 (arXiv 2507.07955); this is a much improved version. The model has five parts: an encoder, learned dynamic chunking, a main network, dechunking and a decoder, so bytes are compressed into chunks inside the model and the process can be nested. The large model uses two stages: Mamba-2 layers on the outside, a sliding-window attention block in the second stage, and a 40-layer Transformer main network that sees about one position per 7.6 bytes.
Benchmarks and comparisons
All figures are Cartesia’s own and have not been independently reproduced. In the controlled scaling ladder, H-Nets gave about 1.3x compute efficiency on validation loss, about 2x on code, and up to 5.37x on out-of-distribution code (Prolog). Against SmolLM3-3B (11.1T training tokens), a compute-matched 3.67B H-Net reached its final six-task average after about 1T tokens.
What developers can do now
There are no public weights, code or API yet, so for now developers can read the two blog posts and the original paper. Cartesia says it plans to release some of these architectures over time and will use the approach for speech models that learn directly from raw audio. Things to watch: gains on code and multilingual data, and whether third parties reproduce the results at larger scale.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.