Alibaba Unveils QwenGyre: Elastic Reinforcement Learning Framework for 700K-Token xLong-Horizon Agents

核心背景与行业痛点 / Background & Pain Points As autonomous coding agents transition from isolated single-bug patches to repository-scale software development, task horizons have expanded by orders of magnitude (xLong-Horizon). Real-world autonomous workflows routinely span hours, hundreds of tool interactions, and nearly 1M tokens per rollout trajectory. Existing on-policy RL architectures (Colocate or decoupled Async) collapse under this regime: execution latency variance produces massive GPU idle bubbles during rollout waits, while non-linear trial-and-error branching creates intractable trajectory redundancy that cripples backward training passes. ### 架构亮点与底层机制 / Architectural Highlights Alibaba's Qwen team developed QwenGyre, an end-to-end framework specialized for xlong-horizon online RL: 1. Elastic Non-Interruptive GPU Reallocation: Seamlessly shifts GPU compute between rollout workers and gradient training nodes in real time without pausing or resetting active long-running environment sandboxes; 2. Branching Trajectory Processor: Reconstructs execution trees, prunes redundant trial-and-error execution loops, and assigns granular partial-progress credit to uncompleted trajectories to constrain memory footprints; 3. Megatoken Sequence Parallelism: Integrates ultra-long context attention distributed primitives capable of processing 700K+ token backpropagation updates per step on dense and MoE architectures. ### 权威 Benchmark 与实测跑分对比 / Benchmark & Evaluation Scaled to Alibaba's flagship Qwen 3.8 (2.4T parameters) with rollouts averaging 700,000 tokens: - NL2RepoBench Breakthrough: Achieves a +6.0% absolute accuracy leap (52.5% to 58.5%) on repository synthesis in only 48 RL steps; - Massive Training Speedup: Outperforms traditional Colocate architectures by 1.85x and decoupled Async pipelines by 1.78x across heterogeneous training domains; - Compute Efficiency: Maintains sustained GPU cluster computational efficiency above 88% despite extreme task length disparities. ### 开发者实战落地与开箱指南 / Developer Practical Guide - Technical Reference: Detailed algorithms are available in arXiv preprint 2609.33848; - Paradigm Shift: Establishes that trillion-parameter foundation models can undergo stable online RL across megatoken horizons; - Ecosystem Integration: Core scheduling and trajectory deduplication components are being prepared for open-source integration within the Qwen ecosystem.