Tongji University & Tencent Introduce GeoState: View-Decoupled 3D World Modeling Prevents Camera Scale Drift Across Waymo and KITTI Rollouts

Background and the Problem

In autonomous driving and robotics, 3D world models are essential for forward planning and counterfactual simulation. Recent approaches adopt latent features from geometry foundation models (GFMs) as world representations. However, because GFM features are inherently view-dependent, camera-induced transformations are entangled with true scene dynamics. This entanglement leads to severe scale drift and geometric instability during autoregressive rollouts, forcing previous models to rely on heuristic per-frame scale alignments. Researchers from Tongji University and Tencent introduced GeoState (arXiv:2610.22343) to decouple physical scene dynamics from sensor viewpoints.

Architecture and How It Works

1. View-Decoupled Scene Tokenizer:

• Aggregates multi-view GFM embeddings into a global, viewpoint-independent 3D scene state representation, isolating scene geometry from individual camera projections;

1. Single Context Scale Autoregression & Ray-Based Decoding:

• An autoregressive world model predicts future scene state transitions independently of camera movement. A lightweight geometry decoder subsequently queries the predicted state using arbitrary camera rays, preserving a single continuous scale across the entire rollout without per-frame realignment.

Benchmarks and Measured Results

• Waymo Long-Horizon Depth Forecasting (Table 1):
• Across forecasting horizons from $t+0.2\text{ s}$ up to $t+2.0\text{ s}$, GeoState consistently outperforms DINO-Foresight, VGGT-World, and FR3D on AbsRel and $\delta_1$ accuracy metrics without needing artificial per-frame scale alignment;
• Zero-Shot Transfer Across KITTI and nuScenes (Table 2):
• Generalizes zero-shot to KITTI ($t+0.6\text{ s}$ to $t+2.0\text{ s}$) and nuScenes ($t+0.75\text{ s}$ to $t+2.5\text{ s}$), demonstrating superior cross-sensor physical generalization.

Getting Started for Developers

• Project Resources: Project demos are available at https://hongchang-liu.github.io/GeoState-project-page/.
• Engineering Takeaways: When developing 3D simulation or driving world models, decoupling camera observation parameters from internal geometric state evolution prevents long-horizon drift and enables arbitrary novel-view rendering of future rollouts.