TRL v1.14.0 (2026-09-25) removes `trl.losses` fused-loss forks: DPO/KTO/GRPO stream via `_ChunkedLogProbFunction` into each trainer's own loss, fixing formula drift and plain-DDP missing all-reduce. Ships an in-tree Triton logprob+entropy kernel (~14× vs selective_log_softmax+entropy_from_logits). On 1×H100, Qwen3-0.6B, bs=4, seq 512: chunked+precompute_ref_log_probs median step 0.1971s (−12.1% vs v1.13 fused) and 4.83GB peak (−31%). Removes six low-use experimental trainers; vLLM window moves to ≤0.30.0.

Key Takeaways

  • ✓Shipped: GitHub v1.14.0 = PyPI trl 1.14.0; docs huggingface.co/docs/trl
  • ✓Architecture: remove trl.losses; DPO/KTO/GRPO via _ChunkedLogProbFunction into existing losses; use_liger_kernel still enables Liger model kernels
  • ✓Perf (official, 1×H100 / Qwen3-0.6B / bs=4 / seq512): 0.1971s (−12.1%) and 4.83GB (−31%) with chunked+precompute_ref_log_probs vs v1.13 fused
  • ✓Kernel: in-tree Triton logprob+entropy ~14× (0.89ms vs 12.8ms) on the stated H100 shape; default on CUDA/ROCm/XPU
  • ✓Breaking: six experimental trainers removed; plain-DDP KTO all-reduce fixed; vLLM window >=0.20.0,<=0.30.0
🔗

Project Links & Resources

Direct Access
Direct access to official project resources and documentation
🐙
GitHub Release v1.14.0
github.com/huggingface/trl/releases/tag/v1.14.0
↗
📖
PyPI trl 1.14.0
pypi.org/project/trl/1.14.0/
↗
📖
TRL documentation
huggingface.co/docs/trl
↗
🐙
TRL GitHub Repository
github.com/huggingface/trl
↗
🔬

In-Depth Technical Analysis

Core Background & Industry Pain Points

Post-training (DPO/KTO/GRPO) must save memory without duplicating loss math or breaking distributed semantics. After TRL v1.13 vendored Liger chunked_loss into trl.losses, formulas drifted (GRPO grad_norm, KTO+PEFT reference ignore, dropped DPO label_smoothing). Worst: under plain DDP the fused path ran on the unwrapped model, so the reducer never armed and ranks never all-reduced.

Architecture Highlights & Internals

v1.14.0 (2026-09-25) deletes trl.losses. Trainers now do backbone → _ChunkedLogProbFunction → existing loss, streaming selected-token log-probs without materializing full logits. use_liger_kernel=True still enables Liger model kernels; DPO/KTO/GRPO losses use TRL's chunked path. In-tree trl.kernels Triton fused logprob+entropy is default on CUDA/ROCm/XPU. Six low-use experimental trainers are removed with a written experimental exit policy.

Authoritative Benchmarks & Measured Scores

Official 1×H100, Qwen3-0.6B, bs=4, seq 512: v1.13 fused inline-ref 0.2243s / 7.05GB; v1.14 chunked+precompute_ref_log_probs=True 0.1971s (−12.1%) / 4.83GB (−31%) — a config the fused path previously rejected. Microbench (bf16, V=151936, H=4096, 8192 tokens): selective_log_softmax+entropy_from_logits 12.8ms / 2.32GiB → fused kernel 0.89ms (~14×).

Developer Hands-on Guide

pip install -U trl==1.14.0. Drop from trl.losses import FusedLinear*; prefer precompute_ref_log_probs=True for the memory/step win. See release notes for still-refused combinations. vLLM extra window is >=0.20.0,<=0.30.0. Docs: huggingface.co/docs/trl.