Published at 11/7/2026, 6:10:00 PM · 19 views
📰 Stanford Audit of Terminal-Bench 2.0: 28 of 89 Tasks Defective; Repairs Ship as 2.1 with Up to ~12-Point Agent Score Shifts
A Stanford/Cornell team (E. Kelly Buchanan, Crystal Zhou, Scott W. Linderman, Christopher Ré) posted arXiv 2611.08417 (2026-11-05) on "benchmark maintenance debt": even Terminal-Bench 2.0, which had human and LM red-teaming before release, still had defects in about 30% of tasks…
💬 What's your take? Join the discussion in the comments.
Read the original Chinese version
📰 斯坦福等审计 Terminal-Bench 2.0：89 题中 28 题有缺陷，修复后发布 2.1，单 Agent 分数最多变动约 12 分
斯坦福 / 康奈尔团队（E. Kelly Buchanan、Crystal Zhou、Scott W. Linderman、Christopher Ré）于 arXiv 2611.08417（2026-11-05）提出「benchmark maintenance debt」：即使用发布前人工+模型红队审过的…
💬 你怎么看？欢迎在评论区讨论。
#Terminal-Bench #Terminal-Bench 2.1 #benchmark audit
0 Comments