Benchmark14d ago
OpenAI open-sources MultiLangBench and reports low-resource language gains: GPT-6 Astra averages 87.4% building SQLite across 20 languages, GPT-6.1 Sol reaches up to 97.9% on COBOL migration
On October 19 OpenAI said its latest models are much better at low-resource languages such as COBOL, Fortran, Visual Basic and OCaml, and open-sourced MultiLangBench, which asks a model to build a SQLite-compatible database engine from scratch in each of 20 languages. GPT-6 Astra averages an 87.4% test pass rate (GPT-5.6 Sol: 26.8%), above 83% in 19 of 20 languages, with COBOL hardest at 72.1%. On a separate COBOL migration eval requiring byte-for-byte output parity after porting to C++, Java, Kotlin, Python or Rust, GPT-6.1 Sol scores 86.6% at low effort (GPT-5.6 Sol: 5.4%) and 97.9% at higher effort. All figures are OpenAI's own.
Research15d ago
Tongji University & Tencent Introduce GeoState: View-Decoupled 3D World Modeling Prevents Camera Scale Drift Across Waymo and KITTI Rollouts
Modern 3D world models typically use view-dependent features from geometry foundation models (GFMs) as predictive states, entangling camera-induced variations with physical scene dynamics. Tongji University and Tencent introduced GeoState (arXiv:2610.22343). By aggregating multi-view GFM features into a view-decoupled 3D scene state, GeoState models physical scene evolution independently of camera viewpoints, maintaining a single context-derived scale throughout rollouts without per-frame scale realignment.
Open Source15d ago
Zhejiang University & Tencent Open-Source KnowGPO for Deep Search Agents: Knowledge-Grouped RL Outperforms 32B Models, Scoring 40.2 on GAIA
Deep search agents struggle with credit assignment across long, noisy retrieved contexts. Zhejiang University and Tencent open-sourced KnowGPO (arXiv:2610.22331, GitHub: ZJU-REAL/KnowGPO), an RL framework that groups search trajectories by atomic knowledge conditions rather than surface observations. Qwen3-8B fine-tuned with KnowGPO achieves 38.8% on BrowseComp-ZH and 40.2% on GAIA, outperforming GRPO/GiGPO baselines and rivaling 32B open-source search systems.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.