Benchmark18d ago
Artificial Analysis launches AA-Robotics v0.1: coding agents drive a real robot arm, GPT-6 Astra leads at 69.4
Artificial Analysis released AA-Robotics v0.1 on 2026-10-14, a research preview in which frontier models running inside coding-agent harnesses (Codex, Claude Code, Antigravity) control a 6-axis robot arm over an HTTP API across eight tabletop tasks. GPT-6 Astra scores 69.4, ahead of GPT-6.1 Sol (64.6), Gemini 4 Argon (55.3), Claude Opus 5.5 (48.8) and Claude Fable 5.1 (46.3). Models take about 30 minutes per task versus roughly 20 seconds for a human.
Benchmark11d ago
Artificial Analysis Releases HealthBench & MLCR-AA: Exposing Severe Verbosity Bias in Medical AI Rankings
Independent AI evaluation platform Artificial Analysis officially released HealthBench Professional alongside the length-normalized MLCR-AA index (@ArtificialAnlys). The benchmark revealed a pervasive industry blindspot: LLM-as-a-Judge evaluations suffer from acute 'verbosity bias', disproportionately rewarding longer generations regardless of medical conciseness. In raw HealthBench scoring where Anthropic outputs were 4–6x longer, Claude Opus 5.5 led at 71.4%. However, once length bias was scientifically normalized via MLCR-AA, GPT-6 Astra (max) tied Claude Sonnet 5.5 for #1, while Claude Opus 5.5 plummeted from #1 down to #6, setting a new rigorous standard for medical model evaluation.
Benchmark46d ago
Vals Live Terminal-Bench 4.0: GPT-6 Astra Leads at 57.1%
Vals AI launched Terminal-Bench 4.0: 66 new end-to-end terminal tasks (ship a service, prove a theorem, train a GPU kernel, write a forensic report, and more). Median expert effort is about 4 hours; grading is strict (full verifier suite or zero), and scores are avg@3. GPT-6 Astra won at 57.1%, 7.6pp ahead of Claude Fable 5.1 (49.5%) and 11.6pp ahead of Claude Opus 5 (45.5%). No other model cleared 30%; fourteen of 27 models scored zero on both hardware and media. Version 4.0 spans seven categories—software, science, ML, operations, hardware, security, and media—with roughly three-quarters of tasks outside traditional software work and no overlap with v2.1.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.