Artificial Analysis Releases HealthBench Medical Evaluation: Verbosity Penalties Reshuffle Ranks as GPT-6 Astra Takes #1

Background and the Problem

Clinical consultation and medical decision support represent high-stakes domains demanding absolute factual integrity, rigorous diagnostic reasoning, and concise communication. Existing benchmarks (such as MedQA) rely primarily on multiple-choice questions, failing to reflect multi-turn clinical dialogue. Crucially, open-ended medical evaluations have suffered from models "gaming" evaluation metrics by outputting thousands of characters of generic disclaimers and encyclopedic fluff to obscure ambiguous diagnostic claims. In real-world emergency care, verbose outputs cost critical clinical time. Artificial Analysis introduced HealthBench and HealthBench Professional to measure medical conversational intelligence under strict verbosity constraints.

Architecture and How It Works

1. Multi-Turn Clinical Scenario Simulation:

• Spans hundreds of medical dialogues encompassing emergency differential diagnosis, chronic medication conflict analysis, and rare disease reasoning;

1. Strict Length Penalty for Redundant Verbosity:

• HealthBench Professional enforces an information-density length penalty. Models must present evidence-based clinical reasoning succinctly; padding answers with boilerplate disclaimers incurs steep penalties;

1. Independent Third-Party Verification:

• Evaluated independently via standardized API calls, verified against physician gold standards.

Benchmarks and Measured Results

According to official Artificial Analysis leaderboard telemetry (Figure 1, HealthBench Disclosures):

• HealthBench Professional (Length-Penalized):
• GPT-6 Astra (Max Effort) secures #1 globally with 66.0%;
• GPT-6.1 Sol (Max Effort) claims #2 at 65.1%;
• Claude Sonnet 5.5 demonstrates remarkable conciseness and diagnostic accuracy, placing #3 at 63.0%;
• The Opus Verbosity Penalty: Claude Opus 5.5 logged the highest raw unpenalized score at 71.4%, but because its answers averaged ~5,900 characters, the length penalty dropped it to 59.9% (#6 overall);
• HealthBench Standard:
• Claude Sonnet 5.5 leads at 62.1%;
• DeepSeek V4.1 Flash takes #2 at 60.9%, outperforming larger proprietary models;
• GPT-6.1 Sol secures #3 at 58.5%.

Getting Started for Developers

• Leaderboard Access: Accessible at artificialanalysis.ai;
• Engineering Recommendation for Healthcare AI: This benchmark proves that in clinical AI systems, information density trumps verbose disclaimers. Engineers building hospital triage systems or physician assistants should enforce strict character limits and structured output schemas in system prompts, prioritizing immediate differential diagnoses over filler prose.