Tencent Hunyuan in collaboration with Fudan University's NLP group has open-sourced ExplorationBench (arXiv: 2609.30199, Project: explorationbench.com), establishing a rigorous framework to quantify an AI agent's genuine scientific exploration and hypothesis-testing capacity. To prevent models from cheating via pre-training memorization, the benchmark introduces 'Alien Worlds'—computational sandbox environments (AlienCode and AlienLogic) governed by strictly executable rules that directly contradict terrestrial intuition. Agents must actively frame hypotheses, design probe experiments via API tools, and revise causal models under feedback.

Key Takeaways

  • ✓Pioneers 'Alien Worlds' executable sandboxes whose rules contradict earthly knowledge, preventing pre-training data leakage.
  • ✓Comprises two distinct discovery sandboxes—AlienCode and AlienLogic—spanning 55 discovery targets and 140 scientific tasks.
  • ✓Furnishes agents with intentionally flawed documentation, environmental feedback, and dedicated tool-calling interfaces.
  • ✓Benchmarking 10 frontier models reveals that while leading models discover novel rules, extended exploration frequently stalls or degrades early gains.
  • ✓Project portal, sandbox execution runtimes, and evaluation suites released openly on GitHub.
🔗

Project Links & Resources

Direct Access
Direct access to official project resources and documentation
📝
ExplorationBench 官方项目主页与交互沙箱
www.explorationbench.com/
↗
🌐
ArXiv 论文预印本 (2609.30199)
arxiv.org/abs/2609.30199
↗
📝
Hugging Face Daily Papers 讨论主页
huggingface.co/papers/2609.30199
↗
🔬

In-Depth Technical Analysis

Core Background & Industry Pain Points True scientific discovery requires formulating hypotheses, designing probe experiments, and refining causal models in the face of counter-intuitive evidence. However, existing LLM evaluations (GPQA, GSM8K) draw from established human repositories, confounding genuine hypothesis generation with pre-training memorization. Crucially, static benchmarks cannot test whether an agent can iteratively debug flawed assumptions when dropped into an unfamiliar system. ### Architecture Highlights & Internals Tencent Hunyuan and Fudan University present ExplorationBench, an evaluation harness constructed upon verifiable 'Alien Worlds'. Environments operate on mathematically deterministic rules that intentionally conflict with conventional syntax and logic (e.g., inverted boolean precedence, counter-intuitive pointer mechanics). The benchmark provides two distinct sandboxes: AlienCode (runtime code mechanics) and AlienLogic (symbolic causality), supplying agents with an intentionally incomplete manual and programmatic observation probes to formulate, test, and isolate unknown invariants. ### Authoritative Benchmarks & Measured Scores Evaluated across 10 frontier architectures: leading reasoning models autonomously discover up to 61.4% of counter-intuitive synthetic invariants through interactive experimentation. However, evaluations uncover a fundamental failure mode in long-horizon scientific agents: excessive exploration introduces context noise, causing agents to unlearn or contradict verified discoveries, resulting in late-stage accuracy degradations of 18.2% to 26.5%. ### Developer Hands-on Guide Researchers can explore interactive sandbox terminals at https://www.explorationbench.com/, download evaluation drivers, and review the 140 scientific discovery targets on arXiv and Hugging Face Papers.