Anthropic Launches Message Threads in Public Beta on the Messages API: Server-Side Conversations Cut Claude Code TTFT About 35% on 500K-1M Token Requests

Background and Industry Pain Point

Long-running agents re-upload the whole conversation every turn, and the API re-validates and re-tokenizes all of it. Once a conversation reaches hundreds of thousands of tokens, requests run to hundreds of KB and time to first token keeps growing. On November 2 Anthropic added message threads to the Messages API as a public beta to address this.

Architecture Highlights and Mechanism

The first request sends the full conversation, system and tools as usual, plus "thread": {"type": "create"}; the API stores the conversation under the response id. Later requests use "type": "continue" with previous_message_id, send only new messages, and must omit system and tools. Threads are append-only and each id can be continued once; changing a fixed field such as model or thinking returns thread_fingerprint_mismatch. Threads live encrypted in memory, are never written to disk, expire after 5 idle minutes and are deleted within 1 hour. They do not cache prompts for you; you still set cache breakpoints, at most 4.

Benchmarks and Measured Results

Per Anthropic: threads are on by default for most Claude Code users, and median time to first token on 500K-1M token requests is about 35% lower; in long conversations the median request fell from about 800 KB to 1.8 KB. No independent third-party measurement yet.

Developer Quick Start

Add the message-threads-2026-08-12 beta header and the thread parameter. The Python, TypeScript and other SDKs offer a client.beta.messages.thread helper that continues threads and replays automatically on a 404. Your app should still keep its own full copy of the conversation. Pricing and rate limits are unchanged, but Message Batches, token counting and on-demand compaction are unsupported, and organizations with HIPAA readiness or Inference hooks enabled cannot use threads.