AI Coding Live News — Real-Time 𝕏/Twitter Updates & Breakthroughs

Google Cloud@googlecloud·Collected 14d ago

👀64

🔥97

Google Cloud Open-Sources TorchTPU: Native PyTorch on Google TPU Hardware with Day-Zero vLLM and SGLang Support

Google Cloud officially open-sourced TorchTPU, enabling native, zero-compilation-barrier PyTorch execution directly on Google Cloud TPU hardware (TPU v5p and TPU v6e). Overcoming the legacy friction of PyTorch/XLA's rigid graph compilation and dynamic-shape stalls, TorchTPU supports native PyTorch eager execution and custom Triton/C++ kernels. Leading high-throughput serving engines vLLM and SGLang announced immediate Day-0 support. Concurrently, Google Cloud introduced OpenAI models and Codex into its Vertex Model Garden, becoming the first hyperscaler to offer all three top frontier closed models.

Key Takeaways

Google Cloud open-sources TorchTPU, delivering native, compilation-free PyTorch execution on TPU v5p and TPU v6e hardware
Eliminates PyTorch/XLA graph-compilation overhead, natively supporting PyTorch eager mode and custom Triton/C++ kernels
vLLM and SGLang announce immediate Day-0 TorchTPU releases, unlocking substantial throughput boosts on TPU infrastructure
OpenAI frontier models and Codex join Google Cloud's Model Garden, solidifying Google Cloud as the premier multi-model hyperscaler

Read full story

Linked Models & Agents:

⚡ Codex⚡ Gemini 4 Argon 🤖 vLLM 🤖 SGLang 🌐 Google Cloud 🌐 OpenAI

#TorchTPU#PyTorch#TPU#Google Cloud#vLLM#SGLang#Inference

EdgeKernelBench Team@huggingface·Collected 15d ago

👀64

🔥95

EdgeKernelBench: Unified Benchmark and LEKA Agent for LLM-Driven Kernel Generation on Heterogeneous Edge Backends

Researchers introduced EdgeKernelBench and the LEKA agent architecture in arXiv:2610.24648, establishing the first unified framework for evaluating and generating low-level high-performance kernels across heterogeneous edge backends. While LLM-driven kernel generation has demonstrated success on datacenter GPUs, edge deployments remain plagued by hardware diversity, fragmented compiler toolchains (CUDA, OpenCL, TileLang, Triton), and strict memory-compute constraints. EdgeKernelBench bridges this gap through a unified intermediate representation (IR) suite spanning 331 canonical workloads. The companion Layered Edge Kernel Agent (LEKA) hierarchically decomposes optimization decisions, unlocking efficient automated kernel synthesis on edge silicon.

Key Takeaways

EdgeKernelBench released in arXiv:2610.24648, establishing the premier unified evaluation harness for edge kernel generation
Spans CPU/LLVM, GPU/CUDA, and GPU/OpenCL stacks with 331 canonical compute patterns across 650 concrete benchmark workloads
Introduces LEKA (Layered Edge Kernel Agent), hierarchically decoupling compilation and schedule decisions across IR abstraction tiers
Solves single-shot generation failures (where models achieved only 25.5% [email protected]), paving the way for automated edge NPU acceleration

Read full story

Linked Models & Agents:

🤖 LEKA

#EdgeKernelBench#LEKA#Kernel Generation#CUDA#OpenCL#Edge AI#Compiler

Hugging Face Daily Papers@HuggingFace·Collected 38d ago

👀79

🔥92

PISA: Block Sparse Attention with O(N log N) Complexity and Hardware-Aware Triton Kernels

Researchers from SJTU and GAIR introduced PISA, a block-sparse attention mechanism that eliminates the quadratic bottleneck of conventional block selection in long-context LLMs. By adopting a coarse-to-fine pyramid Top-K selection hierarchy across O(log N) key levels, PISA reduces complexity to O(N log N). Accompanied by hardware-aware fused Triton kernels that avoid materializing query-key matrices, it matches dense baselines while substantially boosting long-context retrieval throughput.

Key Takeaways

Overcoming Quadratic Block Selection: Conventional block-sparse attention methods still suffer from O(N²) scoring overhead when selecting retained blocks; PISA achieves true O(N log N) complexity via pyramid Top-K selection.
Coarse-to-Fine Key Hierarchy: Pools key vectors into an O(log N)-level hierarchy, applying LogSumExp scoring across bounded candidate pools to progressively narrow down candidate blocks.
Hardware-Aware Triton Kernels: Custom-built fused Triton kernels for training and inference eliminate the need to materialize query-key score matrices in HBM, slashing memory footprint and memory bandwidth bottlenecks.

Read full story

Linked Models & Agents:

#Attention Mechanism#Long Context#Triton#Sparse Attention#LLM Acceleration

Hugging Face@huggingface·Collected 40d ago

👀77

🔥95

Hugging Face TRL 1.14.0: chunked log-probs for DPO/KTO/GRPO, DDP all-reduce fix; up to 31% lower peak VRAM on H100

TRL v1.14.0 (2026-09-25) removes trl.losses fused-loss forks: DPO/KTO/GRPO stream via _ChunkedLogProbFunction into each trainer's own loss, fixing formula drift and plain-DDP missing all-reduce. Ships an in-tree Triton logprob+entropy kernel (~14× vs selective_log_softmax+entropy_from_logits). On 1×H100, Qwen3-0.6B, bs=4, seq 512: chunked+precompute_ref_log_probs median step 0.1971s (−12.1% vs v1.13 fused) and 4.83GB peak (−31%). Removes six low-use experimental trainers; vLLM window moves to ≤0.30.0.

Key Takeaways

Shipped: GitHub v1.14.0 = PyPI trl 1.14.0; docs huggingface.co/docs/trl
Architecture: remove trl.losses; DPO/KTO/GRPO via _ChunkedLogProbFunction into existing losses; use_liger_kernel still enables Liger model kernels
Perf (official, 1×H100 / Qwen3-0.6B / bs=4 / seq512): 0.1971s (−12.1%) and 4.83GB (−31%) with chunked+precompute_ref_log_probs vs v1.13 fused
Kernel: in-tree Triton logprob+entropy ~14× (0.89ms vs 12.8ms) on the stated H100 shape; default on CUDA/ROCm/XPU
Breaking: six experimental trainers removed; plain-DDP KTO all-reduce fixed; vLLM window >=0.20.0,<=0.30.0

Read full story

Linked Models & Agents:

⚡ qwen3-0.6b🌐 Hugging Face 🌐 vLLM

#TRL#DPO#KTO#GRPO#chunked log-probs#Liger#DDP#vLLM#Post training#RLHF