LMSYS Arena published a coding-agents harness study comparing Claude Code, Codex CLI, and Pi. Across 21 rigorously tested model-harness pairs, the harness tax mattered far less than many assume—model quality still dominated. Details are on the Arena blog.

Key Takeaways

  • ✓Compared Claude Code, Codex CLI, and Pi as native coding harnesses for agents
  • ✓21 model-harness pairs showed a smaller harness tax than many assume
  • ✓Model quality still dominated; more Arena findings are forthcoming
⚡

Key Decision Metrics at a Glance

TL;DR Verdict
🏆 Benchmark RankRank
No model benchmark in this story · see current rankings
Full Leaderboard→
💰 Cost & PlansCompare
API & Subscription
Compare Plans↗
🛠️ Ecosystem & IDERules
Claude Code (Anthropic)
Setup IDE Rules↗
🧭

Heavy Claude Code use: compare subscription limits and API bills

Compare 40 dev plans & simulate token costs vs $20/mo subscriptions

⚔️ Compare Model→💰 Compare plans & limits↗
Action HubReady to adopt this in production?

Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.

⚔️ Launch VS Compare→💳 View Plans↗
Up NextOpenAI Codex CLI 0.166.0: keyboard transcript selection, GPT-6.1 Terra in Vertex AI and Bedrock catalogs, codex doctor network checks
Read Now→