LeCun and Balestriero turn the Platonic Representation Hypothesis into a theorem: a 109M text JEPA aligns with vision models better than 72B LLMs

Background

The Platonic Representation Hypothesis (PRH) says that models trained on different modalities converge on the same representation of the world as they scale. It explains why LLM text features line up with vision models, but it has been an empirical conjecture: it neither guarantees when alignment happens nor says how to get there efficiently, so the default answer has been more parameters and data.

How it works

The paper frames the question with JEPAs (joint-embedding predictive architectures). The authors prove that if positive pairs are two observations of one scene, such as two captions of a photo or two frames of a video, then JEPAs trained separately on any number of modalities learn the same representation up to one rotation. If positive pairs are only two uninformative corruptions of one observation, each encoder spends dimensions on its own modality's noise. They argue next-token prediction is exactly this kind of "bad view" JEPA (a prefix and its continuation), so it approaches alignment only slowly through scale.

Benchmarks

All figures are author-reported. On the PRH benchmark, the 109M text JEPA scores 0.184 alignment with DINOv2, above Qwen2.5-72B (0.154) and Llama-3-70B (0.151); with the CLIP image encoder it reaches 0.23 versus a previous best of 0.21. Good views need 42x fewer samples than next-token prediction for the same alignment. A captioner built from one rotation scores CIDEr 0.838 on COCO, and on a pair-free benchmark it reaches 64% top-5 on ImageNet-100 versus at most 56% for language models up to 8B. There is no independent reproduction yet.

Getting started

Code is on GitHub at galilai-group/prh-proof, and the "Captions Without Pairs" demo on Hugging Face lets you caption your own photos or videos with no image-caption pairs. In replies the authors note that generating good views for text automatically is still an open problem. For teams building multimodal retrieval or cross-modal alignment, the takeaway is that designing positive pairs may save more data than scaling up a language model.