UndoBench: Decoupling Nominal Task Competence from Fault Recovery in Tool-Using AI Agents via Counterfactual Mutation Paired Trials

Background and the Problem

Tool-using AI agents are rapidly assuming critical enterprise workflows across cloud orchestration, payments, and system infrastructure. Existing agent evaluations measure nominal completion in undisturbed conditions, presuming APIs never disconnect and networks never drop packets. In production, partial failures and lost acknowledgments are standard occurrences. When facing unconfirmed API responses, default LLM behavior relies on unconstrained retries, repeatedly dispatching mutations that duplicate side effects, corrupt ledgers, and violate state consistency.

Architecture and How It Works

UndoBench (arXiv:2610.05622) formalizes the first dedicated benchmark isolating fault recovery capability from nominal planning:

1. Counterfactual Paired Trials: Executes mirrored trajectories under identical random seeds across nominal and fault-injected runs, mathematically isolating recovery skills from raw task difficulty.
2. Wire-Level Effect and Environment State Oracles: Intercepts network payloads and system state changes, verifying external mutations directly rather than trusting LLM self-reporting.
3. Phase-Dependent Execution Analysis: Categorizes failures into three operational phases: Before Mutation, During Partial Mutation, and After Commit Before ACK, uncovering where architectural recovery protocols fail.

Benchmarks and Measured Results

Benchmarked across 36 workflows and 36 fault topologies spanning eight business verticals over 5,760 executions:

1. Severe Competence-Recovery Disconnect: Models exhibit an 83.54% nominal completion rate, yet conditional recovery success rate (CRSR) plunges to 46.72% under injected lost-acknowledgment faults.
2. Catastrophic Side Effects: Naive retries triggered duplicate external state mutations in 53.33% of trials.
3. Vulnerability During Partial Mutations: Standard client-side idempotency and logging collapse during partial multi-service mutations, demonstrating that safe recovery demands transactional compensation protocols.

Getting Started for Developers

UndoBench is openly available at GitHub (tradertanmay/undobench) with mock enterprise environments and fault injectors. Engineering teams deploying production-grade agents can benchmark their orchestration runtimes against UndoBench, replacing naive try-catch prompt patterns with verifiable saga compensation patterns.