Hugging Face engineer @RisingSayak highlighted 🤗 Kernels (huggingface.co/docs/kernels): when you find a faster kernel, replace a supported layer’s forward() instead of rewriting the model. You can map different kernels for inference vs training and per device type, register choices globally or locally, then keep what benchmarks help. Hub dynamic loading ships via the kernels Python package (torch≥2.5 + CUDA), with community kernels in kernels-community.

Key Takeaways

  • ✓Swap optimized kernels by replacing a layer’s forward()—no model rewrite.
  • ✓Map different kernels for inference vs training and per device type, globally or locally.
  • ✓Hub loading via the kernels package (torch≥2.5/CUDA); docs and kernels-community available.
ADSponsored