Katelyn Lesse, Anthropic's head of platform engineering, said on the Latent Space podcast (published Oct 27) that a Rust version of the Messages API fully rolled out the previous week. The old system was a CPU-hungry Python monolith, the team started with "not a single Rust developer", humans designed and verified the system, Claude wrote the implementation, and shadow traffic compared outputs between the old and new versions. Her ballpark figures are roughly 40-50% less CPU and 10-20% better time to first token. A clip posted by Somi (@somi_ai) drew 1,000+ likes and about 140K views on X and was quoted by swyx and others, with the debate centered on whether "never do the big rewrite" is now outdated advice. These are X discussion views and spoken estimates; Anthropic has not published a written engineering report.
Key Takeaways
- ✓The first-hand source is Katelyn Lesse speaking on the Latent Space episode "How Anthropic ships Claude with Claude", in the "56:05 Rebuilding the Messages API in Rust" segment
- ✓Lesse said the move was driven by CPU becoming a bottleneck as usage surged; CPU now has to be capacity-planned in advance, and research sandboxes, product sandboxes and CI all draw from the same pool inside Anthropic
- ✓The team forked the entire API in Rust in one go, ran shadow traffic to confirm the Python and Rust outputs matched, then turned it on gradually, deliberately adding no new features during the port
- ✓Her ballpark is about 40-50% CPU savings and 10-20% better time to first token, with larger gains on longer-running workloads; these are spoken estimates from the podcast, not a formal benchmark
- ✓On X, Somi called "never do the big rewrite" dead advice and swyx said Anthropic "vibecoded their entire backend to Rust"; the latter is an exaggeration, since Lesse only discussed the Messages API
Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Background and the problem
This is an X discussion item, not an official Anthropic announcement. It starts from a Latent Space podcast episode published Oct 27 with Katelyn Lesse, Anthropic's head of platform engineering, and Angela Jiang, head of product for the platform. Lesse said that when she joined a little over a year ago, the Messages API was a Python monolith running in a single GCP region. As usage surged, CPU became as scarce as accelerators: capacity that used to scale on demand now has to be planned in advance.
How it works
According to Lesse, the team fully rolled out a Rust version of the Messages API the previous week. The team had "not a single Rust developer in the room", so humans owned the distributed-systems design and verification while Claude wrote the implementation and helped engineers learn and test ideas. They forked the entire API in Rust in one go, sent shadow traffic to both the Python and Rust systems, checked that outputs matched, and then turned the new version on gradually. She described it as a pure port with no new features added, with static analysis flagging when other teams added Python-side features that also needed Rust-side work; user-facing re-architecture comes next.
Benchmarks and results
The only figures in the episode are Lesse's spoken ballpark estimates: roughly 40-50% CPU savings and roughly 10-20% better time to first token, with larger gains on longer-running workloads. Anthropic has not published a written engineering report or formal benchmark, so no further numbers are given here.
What developers can take from it
On X, a clip posted by Somi (@somi_ai) drew 1,000+ likes and about 140K views, arguing that the old "never do the big rewrite" advice no longer holds; swyx quoted it saying Anthropic "vibecoded their entire backend to Rust", which overstates it, since Lesse only discussed the Messages API. For teams considering an AI-assisted rewrite, the reusable lessons are to focus effort on design and verification, use shadow-traffic comparison plus feature flags for a gradual cutover, and freeze feature scope during the port. Whether it fits a given system still depends on test coverage and the ability to compare live traffic.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.