Developer Taro L. Saito (@taroleo) followed a TypeSafe Jev-style approach—dropping full text/JSON generation and specializing on decisions—to distill DeepSeek V4 Flash judgments (~157GB weights) into a 4B model with about 26 hours of training on a DGX Spark. The student is roughly 1/20 the size, beats the teacher’s instant-answer mode, and runs about 22ms per decision, showing a fast local-LLM decision engine is practical.

Key Takeaways

  • ✓Jev-style decision-only specialization distills DeepSeek V4 Flash judgments into a 4B model.
  • ✓About 26 hours on a DGX Spark; roughly 1/20 the teacher size and ~22ms per decision.
  • ✓Claims to beat the teacher's instant-answer mode—useful for local routing/gating agents.
ADSponsored