Mid-Harness Scales Action Compute at Model-Harness Boundary for Terminal Agents, Driving Pass@1 from 50% to 68%

核心背景与行业痛点

Autonomous coding and terminal agents operating in Linux environments navigate stateful systems where actions exhibit irreversible side effects. A single botched command—such as an incompatible dependency overwrite, malformed compiler flag, or unintended directory deletion—corrupts the environment irreversibly, dooming the remainder of the trajectory regardless of subsequent model capability. Prevailing test-time compute methods resort to unconstrained trajectory scaling (e.g., Best-of-N full rollouts), which inflates token expenses by orders of magnitude while leaving step-level command fragility unaddressed.

架构亮点与底层机制

Researchers introduce Mid-Harness, allocating test-time compute precisely at the interface between model generation and the harness environment:

1. Action-Level Boundary Interception: Leaves both the base generator and terminal harness unmodified, intercepting and sampling alternative command candidates concurrently before physical execution.
2. Verification-Gated Execution: Evaluates candidate actions using pairwise verification mechanisms, filtering destructive syntax anomalies and command hallucinations.
3. Verifier Distillation to Compact Models: Demonstrates that capable verifiers (such as GPT-5.6 Sol) extract superior commands from frozen generators; distilling verifier preferences into a compact 9B model unlocks self-contained pairwise filtering.
4. Joint Action-Trajectory Scaling: Combines step-level action pruning with macro-level trajectory scaling, outperforming naive trajectory search at a fraction of token cost.

权威 Benchmark 与实测跑分对比

Benchmarked on TerminalBench-Lite across diverse bash tasks:

1. 18-Point Jump in Pass@1: With a TMAX-9B action generator, 8 candidate samples elevate Pass@1 from 50.00% to 68.03% under capable verification.
2. Token-Efficient Robustness: Outperforms full trajectory Best-of-N baselines under matched token budgets, boosting first-try command success by over 32%.
3. Broad Transfer Across Harnesses: Delivers consistent gains across multiple underlying LLM backbones and terminal sandbox configurations.

开发者实战落地与开箱指南

Mid-Harness provides an immediately actionable architecture for real-world coding agents like OpenHands, Cline, and Claude Code. Engineering teams can insert action sampling and pairwise verification into tool-calling middleware, establishing an execution pre-filter that blocks destructive terminal actions and dramatically improves autonomous task completion.