AI Coding Live News — Real-Time 𝕏/Twitter Updates & Breakthroughs

Google Cloud@googlecloud·Collected 11d ago

👀60

🔥97

Google Cloud Open-Sources TorchTPU: Native PyTorch on Google TPU Hardware with Day-Zero vLLM and SGLang Support

Google Cloud officially open-sourced TorchTPU, enabling native, zero-compilation-barrier PyTorch execution directly on Google Cloud TPU hardware (TPU v5p and TPU v6e). Overcoming the legacy friction of PyTorch/XLA's rigid graph compilation and dynamic-shape stalls, TorchTPU supports native PyTorch eager execution and custom Triton/C++ kernels. Leading high-throughput serving engines vLLM and SGLang announced immediate Day-0 support. Concurrently, Google Cloud introduced OpenAI models and Codex into its Vertex Model Garden, becoming the first hyperscaler to offer all three top frontier closed models.

Key Takeaways

Google Cloud open-sources TorchTPU, delivering native, compilation-free PyTorch execution on TPU v5p and TPU v6e hardware
Eliminates PyTorch/XLA graph-compilation overhead, natively supporting PyTorch eager mode and custom Triton/C++ kernels
vLLM and SGLang announce immediate Day-0 TorchTPU releases, unlocking substantial throughput boosts on TPU infrastructure
OpenAI frontier models and Codex join Google Cloud's Model Garden, solidifying Google Cloud as the premier multi-model hyperscaler

Read full story

Linked Models & Agents:

⚡ Codex⚡ Gemini 4 Argon 🤖 vLLM 🤖 SGLang 🌐 Google Cloud 🌐 OpenAI

#TorchTPU#PyTorch#TPU#Google Cloud#vLLM#SGLang#Inference

PyTorchPyTorch·Collected 13d ago

👀83

🔥87

View on 𝕏 ↗

Google and Meta open-source TorchTPU: native PyTorch on TPUs, where porting is as simple as .to("cuda") to .to("tpu"), with native TPU backends for vLLM and SGLang

At PyTorch Conference 2026 on Oct 20, Google open-sourced TorchTPU on GitHub, co-developed with Meta's PyTorch team, making TPUs a native PyTorch device alongside GPUs and CPUs: eager mode and torch.compile, about 450 ATen ops, and torch.distributed collectives. Google also released vLLM-TorchTPU and SGLang-TorchTPU engines, integrations with TorchTitan, Ray, VeRL and Hugging Face Transformers v5 (device_map="tpu"), and a preview TPU agent suite for coding assistants such as Claude Code and Antigravity. AWS shipped the comparable TorchNeuron a day earlier.

Key Takeaways

Porting (official PyTorch blog): existing code runs by changing model.to("cuda") to model.to("tpu") and the distributed backend to tpu_dist; the team notes peak TPU performance still needs TPU-aware shapes and sharding
Coverage (vendor-reported): 98% of about 4,917 popular Hugging Face models (4,821) run without crashing, and 97% of 657 architecture representatives (638) match GPU numerics
Training scale (vendor-reported): Qwen3 MoE 235B on TorchTitan scales from 1,024 to 8,192 TPU v7 chips with MFU roughly flat at 27-29%; on 64 TPU v7 chips it reaches 1,828 TPS versus 1,937 TPS for JAX/MaxText
Inference cost (Google citing SemiAnalysis InferenceX): TPU v7 with vLLM-TorchTPU hits 2.01 cents per million tokens on 8K/1K, down from 3.6 cents at its September debut
Four repos open-sourced: torch_tpu, vllm-torchtpu, sglang-torchtpu and a TPU TorchTitan; the TPU agent suite is in preview for TPU v6e and TPU7x

Read full story

Linked Models & Agents:

⚡ Qwen3 MoE 235B⚡ Llama 3 405B⚡ Muse Glimmer 30B⚡ Qwen 3.5🤖 TPU agent suite 🤖 Claude Code 🤖 Google Antigravity 🌐 Google 🌐 Google Cloud 🌐 Meta 🌐 Hugging Face 🌐 Inferact 🌐 RadixArk 🌐 Anyscale

#TorchTPU#PyTorch#TPU#vLLM#SGLang#TorchTitan#PyTorch Conference

SemiAnalysis@SemiAnalysis_·Collected 14d ago

👀74

🔥93

View on 𝕏 ↗

SemiAnalysis Discloses Google & Inferact TPUv7 Submission: TorchTPU vLLM Megakernels Drive 2.8T Kimi K3, Surpassing GB200 NVL72 and B300 in High-Interactivity TCO

SemiAnalysis revealed the first Google TPUv7 benchmark submission for the 2.8T Kimi K3 MoE model, led end-to-end by Inferact (the creators of vLLM) and Google using the TorchTPU vLLM stack. At a hardware rental cost of just $1.21/chip-hour (vs $2.31 for GB300 and $1.86 for GB200), TPUv7 delivers ~2.2M tokens per $1 TCO in the ultra-high interactivity regime (~390 tok/s/user), outpacing NVIDIA GB200 NVL72, B300, and AMD MI355X on cost-performance.

Key Takeaways

Inferact (creators of vLLM) and Google submitted their first preliminary TPUv7 benchmark for the 2.8T Kimi K3 MoE model
TPUv7 chip rental sits at $1.21/hour, significantly below GB300 ($2.31), B300 ($2.26), and GB200 ($1.86)
Delivers ~2.2M tokens per $1 TCO at ~390 tok/s/user, beating NVIDIA GB200 NVL72, B300, and AMD MI355X in the ultra-high interactivity regime
Built with fused TorchTPU vLLM megakernels and the TPU-Sync KVCache offload library for production PD disaggregation

Read full story

Linked Models & Agents:

#SemiAnalysis#TPUv7#vLLM#Kimi K3#InferenceX#Google Cloud

Inferact@inferact·Collected 48d ago

👀50

🔥92

View on 𝕏 ↗

Google Cloud and Inferact partner to make TPU first-class in vLLM with open-source serving work

Inferact (@inferact) announced a Google Cloud partnership to make TPU first-class in vLLM: one engineering roadmap for production serving/kernels, a native PyTorch path via TorchTPU, and day-0 frontier-model support, plus a community program with shared TPU capacity and maintainer review — all open source.

Key Takeaways

Google Cloud and Inferact are making TPU first-class in vLLM.
Roadmap covers production serving/kernels, TorchTPU, and day-0 frontier models.
Community shared TPU capacity; all collaboration work is open source.

Read full story

Linked Models & Agents:

🌐 Google DeepMind

#Google Cloud#Inferact#vLLM#TPU#TorchTPU#open source#Serving#agentic inference