Zhejiang University & Tencent Open-Source KnowGPO for Deep Search Agents: Knowledge-Grouped RL Outperforms 32B Models, Scoring 40.2 on GAIA

Background and the Problem

Deep search agents solve knowledge-intensive tasks through long-horizon, multi-turn interactions where assigning credit to individual search decisions is notoriously difficult. Outcome-based reinforcement learning (such as GRPO) provides only a single scalar terminal reward for an entire trajectory. Recent process-supervision methods (GiGPO, IGPO) attempt fine-grained credit assignment, but anchor rewards to surface-level text or model activations—both of which fail in deep search where retrieved documents are noisy and lengthy. Researchers from Zhejiang University and Tencent introduced KnowGPO (arXiv:2610.22331, GitHub: ZJU-REAL/KnowGPO) to group trajectories by objective knowledge acquisition.

Architecture and How It Works

1. Atomic Knowledge State Decomposition:

• For each query, KnowGPO decomposes the target answer derivation into a set of atomic knowledge conditions. At each step, an agent's state is represented by the subset of satisfied conditions rather than raw textual observations;

1. Group-Relative RL on Incremental Knowledge Gains:

• Trajectories sharing the same knowledge state are clustered into groups for within-group advantage computation. The incremental knowledge gained between consecutive steps acts as an environment-grounded process reward, effectively eliminating reward hacking caused by noisy search snippets.

Benchmarks and Measured Results

• BrowseComp-ZH and GAIA Benchmarks (Table 1):
• Starting from Qwen3-8B-SFT (30.1% / 31.1%):
• + GRPO reaches 32.9% / 35.0%; + GiGPO achieves 37.3% / 35.0%; + IGPO reaches 32.2% / 37.9%;
• + KnowGPO reaches 38.8% on BrowseComp-ZH and 40.2% on GAIA, outperforming all RL baselines and gaining +5.9 / +5.2 percentage points over standard GRPO;
• The 8B KnowGPO agent surpasses 32B systems like WebSailor-32B (25.5%) and ASearcher-Web-32B (15.6%), closely rivaling DeepMiner-32B-RL (40.1%) and proprietary GLM-4.5 (37.5%).
• Xbench Leaderboard (Table 2):
• Achieves 48.0% on Xbench-2505 and 36.0% on Xbench-2510, setting new state-of-the-art results among open-source 8B search agents.

Getting Started for Developers

• Open-Source Repository: Code and implementation details are available at https://github.com/ZJU-REAL/KnowGPO.
• Engineering Takeaways: When post-training search and research agents, developers can integrate atomic knowledge condition evaluators into the RL loop, substituting outcome-only signals with KnowGPO's state-grouped advantage estimates to dramatically improve tool-search efficiency on compact 8B models.