AI Coding Live News

Google Cloud officially open-sourced TorchTPU, enabling native, zero-compilation-barrier PyTorch execution directly on Google Cloud TPU hardware (TPU v5p and TPU v6e). Overcoming the legacy friction of PyTorch/XLA's rigid graph compilation and dynamic-shape stalls, TorchTPU supports native PyTorch eager execution and custom Triton/C++ kernels. Leading high-throughput serving engines vLLM and SGLang announced immediate Day-0 support. Concurrently, Google Cloud introduced OpenAI models and Codex into its Vertex Model Garden, becoming the first hyperscaler to offer all three top frontier closed models.

Zoom

At PyTorch Conference 2026 on Oct 20, Google open-sourced TorchTPU on GitHub, co-developed with Meta's PyTorch team, making TPUs a native PyTorch device alongside GPUs and CPUs: eager mode and torch.compile, about 450 ATen ops, and torch.distributed collectives. Google also released vLLM-TorchTPU and SGLang-TorchTPU engines, integrations with TorchTitan, Ray, VeRL and Hugging Face Transformers v5 (device_map="tpu"), and a preview TPU agent suite for coding assistants such as Claude Code and Antigravity. AWS shipped the comparable TorchNeuron a day earlier.

Zoom

At PyTorch Conference North America on Oct 20, the CNCF Sandbox project llm-d released version 1.0. It turns inference engines such as vLLM and SGLang into one coordinated serving pool, and 1.0 adds multi-tenant priority and fairness scheduling (flow control), router high availability with zero-downtime rolling upgrades (DisaggregatedSet), a v1 configuration API, and agent-aware session routing with Responses API state. Founded in 2025 by Red Hat, IBM, Google Cloud, CoreWeave and NVIDIA, the project has 500+ contributors, and JetBrains, DigitalOcean and Mistral already run it in production.

Zoom

On October 19 Amazon announced TorchNeuron, which uses PyTorch's standard PrivateUse1 device-backend mechanism to register Trainium as a native "neuron" device, with eager mode, torch.compile and native distributed APIs such as FSDP, DTensor and DDP on Trn2 and Trn3. Amazon says existing PyTorch code runs with a single device configuration change, and it will be open-sourced under Apache 2.0 at PyTorch Conference North America on October 20. The official docs still list it as a closed beta for now.

Zoom

The SGLang team released v0.5.22 featuring 950 PRs from 245 contributors: DeepSeek-V4.1 delivers up to 2.3x more throughput per GPU on NVIDIA H200 and 1.9x on AMD MI350X/MI355X via DSpark. The new InstantTensor loader cuts model loading time by 10x. B200 decodes Qwen3.8-Flash-Next 12% faster, while B300 speeds up GLM-5.3-Flash by 10% and yields 1.85x decode throughput for MiMo-V2.6-Pro. SGLang also adds native Apple Silicon PyTorch support and ComfyUI integration.

Zoom

In reinforcement learning with verifiable rewards (RLVR), rollout trajectories are generated by high-throughput inference engines while backward gradients are computed by training engines, introducing subtle floating-point probability discrepancies that trigger unbounded variance under exact importance sampling. Researchers identified the invariance of logit displacement distributions and introduced Calibrated Importance Sampling (CIS), featuring confidence-aware truncation that replaces unbounded second moments with constant bounds, establishing top scores across five mathematical benchmarks.

Zoom

Shanghai AI Lab’s LMDeploy shipped v0.18.0 on 2026-09-28: input logprobs, expanded SM90 quantized GEMMs with unified TurboMind linear paths, Ascend GLM-5.2, Qwen3.5 dflash, and MoE shared-expert/FFN sharding. PyTorch path prototypes TurboMind W4A16 (AWQ), piecewise CUDA Graph prefill, XTuner TileLang sparse MLA, checkpoint-engine weight updates, and request-only KV cache metrics.

Zoom

Unsloth AI released version v0.1.815-beta alongside official prebuilt Linux x86_64 CUDA 13 wheels for Flash-Attention 2.8.4, Causal-Conv1D 1.7.0, and Mamba-SSM 2.3.2.post1 targeting PyTorch 2.13/2.14 on Python 3.13. The release enables fast local deployment of Alibaba's Qwen-Image-2.1 across both consumer and datacenter GPUs using Fast FP8 and GGUF quantization (defaulting to Q4_K_M) for combined image generation and precise image editing. Crucially, the Unsloth workspace introduces custom Agent Skills compatible with Claude Code and .agents formats, featuring @skill tagging in chat and 2x faster reasoning blocks (boosting UI rendering from 30 FPS to 60 FPS).

Zoom