MLX-LM-LoRA v5.8.6: Apple Silicon fine-tuning library jumps from 3.1.3, adds DSLA preference alignment and critic-free KLPO RL
Gökdeniz Gülmez shipped MLX-LM-LoRA v5.8.6 on Oct 5 (GitHub Release and PyPI), the first release after 3.1.3. It adds DSLA with DPO/ORPO/CPO objectives and critic-free KLPO with token/sequence routes and several KL estimators, improves GRPO stability and microbatching for Online DPO, XPO, RLHF REINFORCE and PPO, and adds fast VJP for gated-delta layers. Minimums rise to mlx>=0.32.3, mlx_lm>=0.32.0 and Python>=3.11; license standardized on Apache 2.0.
Key Takeaways
- Jump from v3.1.3 (Sep 16) straight to v5.8.6, merging nine PRs (#64, #94, #96–#102).
- Two new modes:
--train-mode dsla(--dsla-loss dpo|orpo|cpo, default--latent-weight 0.1) and--train-mode klpo(--klpo-route token|sequence, estimators mc/binary/topk/full). - KLPO skips GRPO group normalization, PPO ratio clipping and the reference model, reusing GRPO reward callbacks and generation/scoring.
- Raised minimums:
mlx>=0.32.3,mlx_lm>=0.32.0, Python>=3.11. - QAT now covers DSLA; new GitHub Pages site with a Python API reference.