Tencent ARC & Tsinghua Open-Source Looking Ahead: Lookahead Attention Eradicates Autoregressive Video Drift

Background and the Problem

In generative video, diffusion architectures and autoregressive (AR) models represent competing paradigms. Autoregressive models offer compelling architectural harmony with frontier LLMs, native streaming rollout capabilities, and direct alignment with Transformer scaling laws. However, existing AR video systems are fundamentally crippled by Cumulative Error Drift. Under unidirectional causal attention, microscopic prediction errors introduced in early tokens propagate exponentially across subsequent frames. Within seconds, generations degenerate into visual amnesia, topological collapse, and severe motion jitter. To eradicate this cumulative decay, Tencent ARC and Tsinghua University introduced and open-sourced Looking Ahead in arXiv:2610.29989.

Architecture and How It Works

1. Lookahead Attention Architecture:

• Transcends rigid, unidirectional historical causal attention;
• Simultaneously calculates a low-resolution planning stream that pre-computes macroscopic temporal keypoints and camera trajectory bounds;

1. Dual-Stream Cross-Coupling:

• Microscopic token synthesis is conditioned concurrently on historical causal context and future temporal anchors via cross-attention;
• Mirrors human visual planning—sketching global scene silhouettes prior to populating granular texture details;

1. Low-Overhead Streaming Compatibility:

• The lightweight temporal planning stream incurs less than an 8% compute overhead, preserving the streaming efficiency and memory frugality of native autoregressive Transformers.

Benchmarks and Measured Results

Evaluated rigorously across extended multi-frame benchmark suites (Figure 2, Table 1):

• Temporal Stability & Coherence:
• Maintained an unprecedented 94.8% structural consistency score across 100+ frame generations, compared to just 61.2% for baseline AR models;
• Motion Fidelity (FVD):
• Improved Fréchet Video Distance (FVD) by 28.6% over competing open-source AR video architectures on dynamic motion datasets;
• Benchmarking AR Parity:
• Successfully demonstrated that autoregressive video generators can match the extended structural stability of premier diffusion systems without sacrificing autoregressive streaming throughput.

Getting Started for Developers

• Paper & Source Repository: Accessible via arXiv:2610.29989 and GitHub at TencentARC/Looking-Ahead;
• System Design Blueprint: Researchers engineering unified multimodal foundations and interactive world simulators should deploy Looking Ahead's dual-stream lookahead planning to eliminate long-horizon autoregressive error divergence.