Artificial Analysis released AA-Robotics v0.1 on 2026-10-14, a research preview in which frontier models running inside coding-agent harnesses (Codex, Claude Code, Antigravity) control a 6-axis robot arm over an HTTP API across eight tabletop tasks. GPT-6 Astra scores 69.4, ahead of GPT-6.1 Sol (64.6), Gemini 4 Argon (55.3), Claude Opus 5.5 (48.8) and Claude Fable 5.1 (46.3). Models take about 30 minutes per task versus roughly 20 seconds for a human.
Key Takeaways
- ✓Scores (out of 100): GPT-6 Astra 69.4, GPT-6.1 Sol 64.6, Gemini 4 Argon 55.3, Claude Opus 5.5 48.8, Claude Fable 5.1 46.3 (AA article)
- ✓Speed: ~30 min per task for models (Argon ~52 min) vs ~20 s for a human
- ✓Cost: GPT-6.1 Sol cheapest at $1.93/task, GPT-6 Astra $16.36/task; ~70% of spend is input context
- ✓Grasp success: Astra 45%, Fable 5.1 44%, Opus 5.5 40%, Argon 36%, Sol 28%
- ✓Measure-the-box (within 2 mm) tops out at 17/100 across all five models
Key Decision Metrics at a Glance
Heavy Claude Code use: compare subscription limits and API bills
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Artificial Analysis launched AA-Robotics v0.1, a research preview measuring how well frontier language models understand, plan and act in the physical world. Each model runs zero-shot inside a coding-agent harness in an isolated container (GPT models in Codex, Claude in Claude Code, Gemini 4 Argon in Antigravity, all at high effort), reaches a Hiwonder NexArm 6-axis arm through a simple HTTP API, sees the scene through four cameras, and writes code to plan movements. Eight tabletop tasks (press a button, place a toy, pour a drink, tidy blocks, shape matching, write hello world, build a tower, measure a box) are scored by a human judge with partial credit.
Results (AA-reported): GPT-6 Astra 69.4, GPT-6.1 Sol 64.6, Gemini 4 Argon 55.3, Claude Opus 5.5 48.8, Claude Fable 5.1 46.3. Models average about 30 minutes per task (Argon ~52) versus ~20 seconds for humans; Sol is cheapest at $1.93 per task and Astra costs $16.36, with ~70% of spend on input context. Grasp success ranges from 45% (Astra) to 28% (Sol), and no model exceeds 17/100 on measuring a box to within 2 mm. AA's takeaway: coding agents can now drive a robot arm for simple tasks, but are roughly 100x slower and 10x costlier than a human.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.