Independent evaluator Artificial Analysis reran VIALS-AA, its life-sciences visual reasoning benchmark, in a one-shot format with one request and no tools. Tool use lifted the cross-model average from about 25% to about 47%. Leader Claude Fable 5.2 (max) scores 67% with tools at $4.20 per task but 44% without them at $0.73, while Claude Haiku 5.5 (max) reaches 49% with tools at $0.07 per task. The same day AA also reported that across the Claude lineup, moving up a model tier usually beats maxing out effort on intelligence per dollar.
Key Takeaways
- ✓Tools lift the VIALS-AA cross-model average from ~25% to ~47% (AA figures)
- ✓Fable 5.2 (max): 67% at $4.20/task with tools vs 44% at $0.73 without, a 5.8x cost gap
- ✓Haiku 5.5 (max) scores 49% at $0.07/task with tools; Opus 5.5 without tools scores only 34% at $0.33
- ✓On 23 cell-counting tasks, Fable falls from 65% to 9% and Opus 5.5 (max) from 57% to 4% without tools
- ✓Claude lineup: Opus 5.5 (xhigh, 57) beats Sonnet 5.5 (max, 55) at 40% lower cost per task and about half the time
Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Background and Industry Pain Point
Teams choosing a model for scientific image analysis often look only at the headline leaderboard score and assume bigger is better. But the same model can behave very differently inside an agent harness (writing code, cropping images, looking again) versus answering once. VIALS is Handshake AI's benchmark for interpreting life-sciences images, and VIALS-AA is Artificial Analysis' implementation, which by default lets models work as agents in a sandbox. AA has now rerun the same tasks in a one-shot, no-tools format, putting a number on what tools are worth in points and dollars.
Architecture and Mechanism
With tools, a model can crop images, read pixel intensities and separate small objects in code before counting them; without tools it has to estimate by eye. AA found that the 10 models scoring below 31% without tools average about 42 tool calls per task, versus about 23 for the 7 models above that line. MiMo-V2.6-Pro gains the most, from 5% to 43% at roughly 49 calls per task. GPT-6 Astra and GPT-6.1 Sol make the fewest calls, about 8, and gain only 13 and 16 points.
Benchmarks and Measured Results
All figures are AA's own: Fable 5.2 (max) scores 67% with tools (the top of AA's leaderboard, ahead of Gemini 4 Carbon at 64%) and 44% without. On 23 cell-counting and quantification tasks, Fable drops from 65% to 9% and Sonnet 5.5 (max) from 65% to 13%, while Astra holds up better, falling from 61% to 35%. On phylogenetics tasks Astra actually scores higher without tools (83% vs 78%). On cost, tools make Qwen3.8 Max 28x more expensive per task and Gemini 3.8 Flash 20x.
Developer Guide
The practical takeaway: for charts, microscopy and other tasks that need exact counts, a small model with a code sandbox is often both more accurate and cheaper than a large model answering once. Start with a Haiku-class model plus a Python execution tool as a baseline before upgrading. AA's same-day Claude analysis adds sizing advice: Opus 5.5 (xhigh) scores 57, above Sonnet 5.5 (max) at 55, at 40% lower cost per task, but at the Haiku tier every step up costs more. Full data is on AA's VIALS-AA page and its Anthropic provider page.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.