NUS, USTC, and Alibaba Qwen Introduce Steering Agent: Dual-Path RL Framework Solves Goal Drift, Boosting Multi-Hop QA by +7.4 Points with 26% Fewer Turns

Background and the Problem

As LLM-based autonomous agents and multi-agent systems (MAS) execute increasingly complex, long-horizon software engineering and web workflows, they frequently suffer from goal drift—gradually straying from the user's primary objective through recursive tool invocations and subtask proliferation. Existing remedies like static pre-planning or post-hoc verification introduce heavy latency overheads and cannot dynamically modulate intervention frequency. To resolve this, researchers from the National University of Singapore (NUS), USTC, and Alibaba's Qwen Business Unit introduced Steering Agent (arXiv:2610.22316), an online runtime supervisory framework that detects drift and steers agents back on track.

Architecture and How It Works

1. Dual-Path Fast/Slow Steering Architecture:

• Fast Path (Low-Cost): Continuously observes the interaction context to make lightweight binary decisions on whether the system has drifted, deciding when to intervene;
• Slow Path (High-Cost): Triggered selectively upon drift alerts to diagnose failure points and generate grounded redirect instructions, determining how to intervene.

1. Joint Reinforcement Learning via Pairwise Ranking & GRPO:

• The fast path is trained using pairwise ranking objectives to maximize drift sensitivity, while the slow path is optimized via Group-Relative Policy Optimization (GRPO) to synthesize concise, high-efficacy corrective prompts. This decouples supervisory intelligence from the executor, enabling seamless steering of black-box LLMs without executor retraining.

Benchmarks and Measured Results

• In-Domain Multi-Hop QA Benchmarks (Table 1):
• Compared to the unsteered baseline (42.70 avg accuracy, 6.4 turns) and static RL planning (45.91, 8.8 turns), Steering w/ RL achieves 50.10 avg accuracy (+7.40 points gain) while cutting executor turns from 8.8 to 6.5 turns (-26.1% reduction);
• Delivers marked gains on 2Wiki (50.1 vs 36.9, +13.2 pts) and HotpotQA (53.7 vs 46.9, +6.8 pts).
• Out-of-Domain Online Search Benchmarks (Table 2):
• On the Frames benchmark with live search APIs, lifts Avg@4 from 26.46% to 36.32% (+9.86 pts) and Pass@4 from 44.17% to 59.71% (+15.54 pts);
• On GAIA, boosts Avg@4 from 16.75% to 23.79% (+7.04 pts) and Pass@4 to 39.81%; reaches 72.00% Pass@4 on xBench-DS (vs 65.00%).

Getting Started for Developers

In production multi-agent architectures (such as autonomous repository refactoring or deep research loops):

1. Deploy the fast-path steering classifier as an event-driven sidecar filter between agent tool turns to monitor trajectory drift scores;
2. Configure dynamic intervention thresholds so that slow-path redirection prompts are only generated when drift probability exceeds tolerance, pruning unproductive execution branches without inflating token costs.