SWE-at-Scale: Reverse-Engineering Historical Bug Fixes to Exponentially Synthesize Verifiable Software Engineering Tasks

Background and the Problem

As coding agents advance rapidly, both reinforcement learning with verifiable rewards (RLVR) and rigorous benchmarking demand a massive, diverse supply of executable software engineering (SWE) tasks. However, real-world tasks are bottlenecked by the natural rate of GitHub issue reporting and expensive manual Docker curation, while existing synthetic methods rely on naive heuristic prompting that fails to capture the complex mechanics of authentic software failures.

Architecture and How It Works

To exponentially scale realistic task synthesis, researchers from Shanghai Jiao Tong University introduced SWE-at-Scale:

1. Mechanistic Fix-to-Bug Inversion: Built on the insight that historical bug-fix commits encapsulate deep mechanistic knowledge of how real bugs form, SWE-at-Scale inverts real-world repair patches to transform repair supervision into grounded bug-induction supervision;
2. Cross-Context Bug Pattern Transfer: Transfers learned failure typologies to novel code contexts and repositories, repeatedly sampling diverse, executable bug instances paired with verifiable test suites.

Benchmarks and Measured Results

Extensive empirical evaluations demonstrate superior task realism and scaling behavior:

• Robust Diversity up to 200 Generations per File: Generates distinct, non-repetitive bug typologies from a single code context, with bug diversity scaling robustly up to 200 generations per file;
• Higher Challenge for Frontier Coding Agents: Poses substantially greater difficulty and diagnostic realism to state-of-the-art coding agents than heuristic bug-injection baselines.

Getting Started for Developers

The synthesis framework is open-sourced at https://github.com/cslsolow/SWE-at-scale:

• Scale Private RLVR Code Datasets: Run SWE-at-Scale over internal git commit histories to turn finite historical bug fixes into thousands of executable debugging environments for training repository-specialized coding agents.