Benchmark23h ago
Artificial Analysis Releases HealthBench & MLCR-AA: Exposing Severe Verbosity Bias in Medical AI Rankings
Independent AI evaluation platform Artificial Analysis officially released HealthBench Professional alongside the length-normalized MLCR-AA index (@ArtificialAnlys). The benchmark revealed a pervasive industry blindspot: LLM-as-a-Judge evaluations suffer from acute 'verbosity bias', disproportionately rewarding longer generations regardless of medical conciseness. In raw HealthBench scoring where Anthropic outputs were 4–6x longer, Claude Opus 5.5 led at 71.4%. However, once length bias was scientifically normalized via MLCR-AA, GPT-6 Astra (max) tied Claude Sonnet 5.5 for #1, while Claude Opus 5.5 plummeted from #1 down to #6, setting a new rigorous standard for medical model evaluation.
Benchmark3d ago
Artificial Analysis launches HealthBench and HealthBench Professional leaderboards: GPT-6 Astra leads clinician conversations at 66.0%, while Claude Opus 5.5 tops the raw score but falls to sixth after the length penalty
On October 19 independent evaluator Artificial Analysis launched two health-conversation leaderboards built on OpenAI's HealthBench (5,000 realistic health conversations) and HealthBench Professional (525 clinician conversations), graded against physician-written rubrics and reported as both raw and OpenAI-style length-adjusted scores. On Professional, GPT-6 Astra (max) leads at 66.0% length-adjusted, ahead of GPT-6.1 Sol (65.1%) and Claude Sonnet 5.5 (63.0%); Claude Opus 5.5 has the top raw score (71.4%) but drops to sixth (59.9%) because its ~5,900-character answers are penalized. On HealthBench, Claude Sonnet 5.5 leads at 62.1%, with DeepSeek V4.1 Flash close behind at 60.9% for about 1/16 of the cost per task.
Benchmark3d ago
Devin Fusion with an Opus 5.5 lead and GPT-6 Luna sidekick tops the Artificial Analysis Coding Agent Index at 68.1, at what Cognition says is 31% lower cost per task than Claude Code with Opus 5.5
Cognition said on October 19 that Devin Fusion, pairing Claude Opus 5.5 as the lead with GPT-6 Luna as the sidekick, scored 68.1 on the Artificial Analysis Coding Agent Index v1.5. Artificial Analysis's current data puts it first, but only 0.1 ahead of Muse Code with Muse Wonder Full at 68.0, effectively a tie; Antigravity CLI with Gemini 4 Argon is third at 63.8. Cognition says the pairing costs 31% less per task than Claude Code with Opus 5.5 (vendor-reported). Fusion ships in Devin CLI and Desktop.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.