Benchmark38d ago
Vals Live Terminal-Bench 4.0: GPT-6 Astra Leads at 57.1%
Vals AI launched Terminal-Bench 4.0: 66 new end-to-end terminal tasks (ship a service, prove a theorem, train a GPU kernel, write a forensic report, and more). Median expert effort is about 4 hours; grading is strict (full verifier suite or zero), and scores are avg@3. GPT-6 Astra won at 57.1%, 7.6pp ahead of Claude Fable 5.1 (49.5%) and 11.6pp ahead of Claude Opus 5 (45.5%). No other model cleared 30%; fourteen of 27 models scored zero on both hardware and media. Version 4.0 spans seven categories—software, science, ML, operations, hardware, security, and media—with roughly three-quarters of tasks outside traditional software work and no overlap with v2.1.
Benchmark10d ago
Artificial Analysis launches AA-Robotics v0.1: coding agents drive a real robot arm, GPT-6 Astra leads at 69.4
Artificial Analysis released AA-Robotics v0.1 on 2026-10-14, a research preview in which frontier models running inside coding-agent harnesses (Codex, Claude Code, Antigravity) control a 6-axis robot arm over an HTTP API across eight tabletop tasks. GPT-6 Astra scores 69.4, ahead of GPT-6.1 Sol (64.6), Gemini 4 Argon (55.3), Claude Opus 5.5 (48.8) and Claude Fable 5.1 (46.3). Models take about 30 minutes per task versus roughly 20 seconds for a human.
Agents8d ago
SpecAgent & LEASH: Trajectory-Level Speculative Execution Halves Agent Latency while Latent-Uncertainty Escalation Beats Always-On Claude Opus 5 on OSWorld
Two new frameworks re-architect how weak and strong models collaborate in long-horizon agents: SpecAgent (arXiv:2610.21463) lets a small model draft multi-step read-only tool trajectories grounded in real observations while holding environment-modifying actions, allowing the strong model to verify the entire segment in a single parallel prefill pass—halving wall-clock time and memory traffic on τ²-Bench. Complementarily, LEASH (arXiv:2610.22280) uses training-free latent-state uncertainty to escalate and supervise computer-use agents, reaching 73.66% on OSWorld with Qwen3.5-35B-A3B—beating always-on Claude Opus 5 (72.90%) while cutting inference cost by 19.2%.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.