Latent-MOPD: Carnegie Mellon Pioneers Representation-Level Multi-Teacher On-Policy Distillation for LLMs

核心背景与行业痛点

On-Policy Distillation (OPD) trains student LLMs on rollouts generated from their own policy distributions, effectively eliminating exposure bias and outperforming conventional offline Supervised Fine-Tuning (SFT) in complex mathematical and coding reasoning. When aggregating diverse specialized capabilities—such as synthesizing math solvers, code generators, and logical reasoners into a unified student—multi-teacher OPD has emerged as a premier paradigm. However, existing multi-teacher OPD methods remain restricted to 'token-only' probability alignment, calculating KL-divergences strictly over output vocabulary logits. This surface-level supervision discards rich geometric concept abstractions embedded within specialists' internal hidden states, leaving students to mimic superficial token styles without assimilating underlying deductive mechanisms.

架构亮点与底层机制

Researchers from Carnegie Mellon University and Georgia Tech introduce Latent-MOPD (arXiv:2610.02381), the first representation-level multi-teacher OPD framework:

1. Dual-Channel Representation & Logit Supervision: Integrates specialist teachers by pairing their output probability distributions with internal intermediate hidden states across generation steps without requiring teacher parameter updates.
2. Shared Projections & Late-Layer Target Selection: Resolves dimensional disparities between heterogeneous teacher-student hidden widths via shared linear projections, strategically aligning deep late-layer activations that encapsulate task abstractions.
3. Domain-Pure Grouped Updates: Identifies that interleaving latent gradients across disparate specialist domains within single batches triggers representation collapse; enforces domain-segregated gradient updates to ensure stable internal feature consolidation.
4. Progressive Annealing Curriculum: Initializes distillation with intensive hidden-state representation alignment to sculpt the student's semantic latent space, progressively shifting supervisory focus toward vocabulary token predictions during later epochs.

权威 Benchmark 与实测跑分对比

Evaluated across nine standardized benchmarks spanning mathematics, coding, and deductive reasoning:

1. Universal Dominance Across All 9 Benchmarks: Latent-MOPD outperforms token-only distillation, representation-only variants, and uniform-averaging baselines across every single evaluated test suite.
2. Equal-Sized Student Outperforms Best Individual Teachers: Holding student parameter counts strictly equal to individual specialist teachers, the unified student surpasses the domain-best teacher on a majority of the benchmarks, proving true capability synergy.
3. Resilient Cross-Family Transfer: Successfully generalizes to larger, independently trained cross-family teacher models, consistently eclipsing single-channel baselines without destabilizing internal representations.

开发者实战落地与开箱指南

Latent-MOPD establishes a practical blueprint for consolidating disparate open-source specialist checkpoints into a unified, high-performing foundation model. ML infrastructure teams post-training coding and reasoning agents can integrate Latent-MOPD's shared projection heads and domain-pure gradient batching into existing OPD workflows, extracting deep conceptual representations beyond superficial logit mimicry.