Google's Gemma team says Gemini 4 Argon wrote vLLM optimizations that speed up gemma-4-E2B-it 2.85x on an A100
On Oct 26, Google's official @googlegemma account said a set of vLLM optimizations built with Gemini 4 Argon lifted gemma-4-E2B-it throughput on a single A100 from about 37,900 to 108,000+ tokens/sec, roughly 2.85x, with 100% token parity. The PRs were filed by Luciano Martins of Google DeepMind, and most are still under review.
Key Takeaways
- Google says gemma-4-E2B-it throughput on an A100 rose about 2.85x (about 37.9K to 108K+ tokens/sec) with 100% token parity
- The PRs come from Luciano Martins of Google DeepMind and cover fused Triton kernels, MTP speculative decoding and KV-sharing fast prefill
- The 2.85x figure is Google's claim for the full set; PR #60674 alone self-reports up to 1.90x prefill and about 1.06x decode
- As of Oct 27, only 3 of the 10 PRs in the linked list are merged; the rest are still in review
- It is a public example of using a frontier coding model for inference-kernel performance work, still awaiting reproduction by the vLLM community
Linked Models & Agents: