Tsinghua and Z.AI open-source HarnessPro: agents rewrite their own harness, reaching 80.8% on Terminal-Bench 2 at 34% lower cost per task

Background and pain point

A coding agent's results depend not only on the model but on its harness: the instructions, tools and runtime policies wrapped around it. Recent recursive self-improvement methods let agents rewrite their harness from past runs, but the paper notes they usually buy higher pass rates with higher cost. With GPT-5.5 and Codex on SWE-Bench Pro, RHO raised mean cost per task from $1.88 to $2.68, up 42.6%. HarnessPro, from researchers at Tsinghua University, Z.AI (Zhipu) and collaborators, aims to raise scores and cut cost at the same time.

How it works

There are three steps. Specialized agents diagnose execution traces to find failure patterns, redundant tool calls and wasted computation. They then plan changes under different performance-cost priorities and implement candidate harnesses, touching instructions, execution profiles, tool-use policies and verification helpers. Candidates are scored with LLM-synthesized tests, without ground-truth answers or official tests, and a normalized weighted squared-distance rule keeps Lite (cost), Medium (balanced) and Heavy (performance) harnesses for the next iteration.

Benchmarks

All figures are the authors' own and have not been independently reproduced. With GPT-5.5 and Codex on Terminal-Bench 2, Heavy scores 80.8% at $0.72 per task versus RHO at 76.3% and $1.09; Medium scores 76.3% at $0.56 and Lite 72.9% at $0.43. Averaged over three benchmarks, Heavy beats RHO by 2.0 points at 26.1% lower cost. With GLM-5.2 and Claude Code, Heavy reaches 81.4% at $0.62 on Terminal-Bench 2, versus vanilla at 74.6% and $0.92. Each benchmark uses only 10 training tasks, with 59, 100 and 100 held-out test tasks.

Getting started

The pzs19/HarnessPro repository is MIT-licensed and ships Lite, Medium and Heavy harnesses for all three benchmarks. It needs Linux, Python 3.12 or 3.13, uv, Docker Compose and Codex CLI (the paper used 0.147.0); clone it, run uv sync and the prepare_runtime script, then supply benchmark data and model keys yourself. A practical first step is to swap the Medium harness into an existing Codex setup and check whether pass rate holds while the bill drops, then try Lite for cost-sensitive batch work. These harnesses were evolved for specific benchmarks, so validate them on your own tasks before relying on them.