On Oct 20 ElevenLabs released Eleven Realtime, end-to-end speech-to-speech models where each model is a complete voice agent, removing the need to wire up speech recognition, an LLM and text-to-speech yourself. v1 Mini targets low-cost natural conversation and v1 Max targets support and multi-step tool use. Both cover 90+ languages and 17,000+ voices. Citing independent Artificial Analysis evals, Max answers 98.8% of Big Bench Audio correctly and starts speaking in 0.57s on average. The API is OpenAI Realtime compatible, so migrating means changing the endpoint and model name.
Key Takeaways
- ✓Speech reasoning (independent Artificial Analysis eval cited by ElevenLabs): on 1,000 Big Bench Audio questions, v1 Max scores 98.8% and v1 Mini 75.9%
- ✓Speed (vendor-stated): Max starts speaking in 0.57s on average and Mini in 0.62s; the launch post says "half a second", slightly faster than the blog figures
- ✓Conversation dynamics and preference: Full-Duplex-Bench Mini 95.8%, Max 94.1%; Speech Agent Arena blind tests Max 998, Mini 988
- ✓Tool-use task completion: Max 87.1% vs Mini 60.3%, which is why ElevenLabs recommends Max for heavy tool calling
- ✓90+ languages and 17,000+ voices; the Realtime API is WebSocket-only at
wss://api.elevenlabs.io/v1/convai/realtimeand works with the OpenAI SDK; pricing was not in the launch post
Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Background
Building a natural voice agent usually means choosing speech recognition, voice activity detection, turn-taking, an LLM and text-to-speech, then tuning how they fit. ElevenLabs argues developers shouldn't need to become configuration experts and folds the pipeline into one model. Mistral's Voxtral 3 voice agent SDK, released the same day, takes the opposite, chained-pipeline approach.
How it works
Eleven Realtime models listen, think and speak directly. v1 Mini targets assistants, tutors and characters and is enough for most cases per ElevenLabs; v1 Max reasons before replying for process-driven support and lead qualification. Both support 90+ languages and 17,000+ library voices and follow the OpenAI Realtime spec.
Benchmarks
Figures below are independent Artificial Analysis evals cited by ElevenLabs. Big Bench Audio: Max 98.8%, Mini 75.9%. Average time to first audio: Max 0.57s, Mini 0.62s, slightly slower than the "half a second" in the launch post. Full-Duplex-Bench: Mini 95.8% edges Max at 94.1%. Speech Agent Arena: Max 998, Mini 988. Tool-use completion: Max 87.1%, Mini 60.3%. Pricing was not in the launch post.
Getting started
ElevenLabs recommends picking a Realtime model from the dropdown in ElevenAgents to get phone, web, WhatsApp and CCaaS channels, knowledge base and monitoring. Apps on OpenAI Realtime can point the WebSocket to wss://api.elevenlabs.io/v1/convai/realtime and set the model to eleven_realtime_v1_mini. It is WebSocket-only, so WebRTC integrations must move first, and the standalone API lacks ElevenAgents channels and knowledge base.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.