AI Coding Live News — Real-Time 𝕏/Twitter Updates & Breakthroughs

👀81

🔥98

Mistral AI and NVIDIA Forge Industrial Engineering AI Alliance, Previewing 1T Mistral Large 4

At GTC Berlin, Mistral AI and NVIDIA announced a strategic partnership dedicated to advancing AI for industrial engineering, computational physics, and electronic design automation (EDA). The alliance couples Mistral frontier architectures with NVIDIA CUDA-X libraries, Warp physics simulation frameworks, and Omniverse digital twins. Mistral simultaneously revealed architectural specifics for its forthcoming flagship, Mistral Large 4: spanning roughly 1 trillion total parameters with 49 billion active parameters (MoE), trained on 3,800 Grace Blackwell GPUs, with open weights slated for official release at the end of October 2026.

Key Takeaways

Mistral AI and NVIDIA launched a strategic alliance applying frontier foundation models to CAE, physics AI, and EDA semiconductor design
Integrates NVIDIA Warp, CUDA-X simulation libraries, and Omniverse to establish open physics datasets and industrial agent benchmarks
Disclosed architectural parameters for Mistral Large 4: ~1T total parameters with 49B active sparse MoE, trained on 3,800 Grace Blackwell GPUs
Weights for Mistral Large 4 are scheduled for open-source release at the end of October 2026, joining the Nemotron Coalition

Read full story

Linked Models & Agents:

⚡ Mistral Large 4 🌐 Mistral AI 🌐 NVIDIA

#Mistral AI#NVIDIA#Mistral Large 4#Industrial AI#CAE#EDA#Grace Blackwell#GTC Berlin

EdgeKernelBench Team@huggingface·Collected 15d ago

👀64

🔥95

EdgeKernelBench: Unified Benchmark and LEKA Agent for LLM-Driven Kernel Generation on Heterogeneous Edge Backends

Researchers introduced EdgeKernelBench and the LEKA agent architecture in arXiv:2610.24648, establishing the first unified framework for evaluating and generating low-level high-performance kernels across heterogeneous edge backends. While LLM-driven kernel generation has demonstrated success on datacenter GPUs, edge deployments remain plagued by hardware diversity, fragmented compiler toolchains (CUDA, OpenCL, TileLang, Triton), and strict memory-compute constraints. EdgeKernelBench bridges this gap through a unified intermediate representation (IR) suite spanning 331 canonical workloads. The companion Layered Edge Kernel Agent (LEKA) hierarchically decomposes optimization decisions, unlocking efficient automated kernel synthesis on edge silicon.

Key Takeaways

EdgeKernelBench released in arXiv:2610.24648, establishing the premier unified evaluation harness for edge kernel generation
Spans CPU/LLVM, GPU/CUDA, and GPU/OpenCL stacks with 331 canonical compute patterns across 650 concrete benchmark workloads
Introduces LEKA (Layered Edge Kernel Agent), hierarchically decoupling compilation and schedule decisions across IR abstraction tiers
Solves single-shot generation failures (where models achieved only 25.5% [email protected]), paving the way for automated edge NPU acceleration

Read full story

Linked Models & Agents:

🤖 LEKA

#EdgeKernelBench#LEKA#Kernel Generation#CUDA#OpenCL#Edge AI#Compiler

GradiumGradiumAI·Collected 15d ago

👀91

🔥81

View on 𝕏 ↗

Gradium open-sources Phonon on-device TTS: a 31M-parameter, 64 MB model that starts speaking in 7 ms on a laptop CPU, with the English version under Apache 2.0

On Oct 20, French voice AI company Gradium open-sourced Phonon, a family of on-device text-to-speech models: the English Phonon 31m (64 MB q8 weights, Apache 2.0) and Phonon 40m (single-language English, French, German, Spanish and Portuguese variants plus a five-language model, CC BY-NC non-commercial). Built on Kyutai's Pocket TTS and the Mimi codec, it ships its own runtime for CPU, Metal, CUDA, Vulkan, WebGPU, Core ML and Snapdragon, running offline on phones, browsers and a Raspberry Pi. Gradium reports 7 ms to first audio on a MacBook Air M5 CPU versus 517 ms for Kokoro.

Key Takeaways

Check the license: only the English Phonon 31m is Apache 2.0 for commercial use; every Phonon 40m variant, including multilingual, is CC BY-NC 4.0 and needs a deal with Gradium for commercial use
Size (official): 64 MB q8 weights for 31m and 74.6 MB for 40m; 37 built-in English voices, 102 in the multilingual model, plus voices cloned or designed with Gradium
Speed (vendor-measured): 6.9 ms to first audio and 29.9x real time on a MacBook Air M5 CPU with 4 threads, versus 517 ms for Kokoro (which does not stream); 38 ms and 3.7x real time on a Raspberry Pi 5; 7.5 ms on Snapdragon 8 Elite Gen 5
Accuracy (vendor-built hard-sentence set, 500 per language, public on HF): English WER of 1.70% for 31m and 2.22% for 40m versus 3.50% for Kokoro and 8.71% for NeuTTS Nano, with text normalization for numbers, dates and version strings
Get started with uvx ptts or pip install ptts, the phonon-tts npm package for browsers, or the gradium-ai/xn-ptts runtime on GitHub

Read full story

Linked Models & Agents:

⚡ Phonon 31m⚡ Phonon 40m⚡ Phonon 40m multilingual⚡ Pocket TTS⚡ Kokoro🤖 Pipecat 🌐 Gradium 🌐 Kyutai 🌐 Arm 🌐 Qualcomm 🌐 Hugging Face

#Phonon#TTS#Apache 2.0#Voice Agent

PyTorchPyTorch·Collected 16d ago

👀88

🔥87

View on 𝕏 ↗

Google and Meta open-source TorchTPU: native PyTorch on TPUs, where porting is as simple as .to("cuda") to .to("tpu"), with native TPU backends for vLLM and SGLang

At PyTorch Conference 2026 on Oct 20, Google open-sourced TorchTPU on GitHub, co-developed with Meta's PyTorch team, making TPUs a native PyTorch device alongside GPUs and CPUs: eager mode and torch.compile, about 450 ATen ops, and torch.distributed collectives. Google also released vLLM-TorchTPU and SGLang-TorchTPU engines, integrations with TorchTitan, Ray, VeRL and Hugging Face Transformers v5 (device_map="tpu"), and a preview TPU agent suite for coding assistants such as Claude Code and Antigravity. AWS shipped the comparable TorchNeuron a day earlier.

Key Takeaways

Porting (official PyTorch blog): existing code runs by changing model.to("cuda") to model.to("tpu") and the distributed backend to tpu_dist; the team notes peak TPU performance still needs TPU-aware shapes and sharding
Coverage (vendor-reported): 98% of about 4,917 popular Hugging Face models (4,821) run without crashing, and 97% of 657 architecture representatives (638) match GPU numerics
Training scale (vendor-reported): Qwen3 MoE 235B on TorchTitan scales from 1,024 to 8,192 TPU v7 chips with MFU roughly flat at 27-29%; on 64 TPU v7 chips it reaches 1,828 TPS versus 1,937 TPS for JAX/MaxText
Inference cost (Google citing SemiAnalysis InferenceX): TPU v7 with vLLM-TorchTPU hits 2.01 cents per million tokens on 8K/1K, down from 3.6 cents at its September debut
Four repos open-sourced: torch_tpu, vllm-torchtpu, sglang-torchtpu and a TPU TorchTitan; the TPU agent suite is in preview for TPU v6e and TPU7x

Read full story

Linked Models & Agents:

⚡ Qwen3 MoE 235B⚡ Llama 3 405B⚡ Muse Glimmer 30B⚡ Qwen 3.5🤖 TPU agent suite 🤖 Claude Code 🤖 Google Antigravity 🌐 Google 🌐 Google Cloud 🌐 Meta 🌐 Hugging Face 🌐 Inferact 🌐 RadixArk 🌐 Anyscale

#TorchTPU#PyTorch#TPU#vLLM#SGLang#TorchTitan#PyTorch Conference

vLLMvllm_project·Collected 25d ago

👀91

🔥88

View on 𝕏 ↗

vLLM adds NVIDIA Vera Rubin NVL72 support: up to 7.84x per-GPU throughput over GB200 on AgentX with MiniMax M3

On Oct 9 the vLLM team, with Inferact, Red Hat and NVIDIA, shipped early Vera Rubin NVL72 support: CUDA 13.4 nightly images already serve DeepSeek, Kimi, GLM and MiniMax. Vendor-reported results: up to 7.84x per-GPU throughput vs GB200 on SemiAnalysis AgentX (MiniMax M3, matched interactivity) and up to 3.7x vs GB300 NVL72 on the MLPerf Inference v6.1 VLM benchmark (Qwen3-VL-235B-A22B).

Key Takeaways

Versus GB200 NVL72: 5x NVFP4 FLOPS, ~2.4x HBM4 bandwidth, 1.7x bidirectional NVLink, 2–4x faster softmax exponentials (vLLM blog)
AgentX with MiniMax M3: up to 7.84x per-GPU throughput at matched interactivity, 5.18x under a 150 TPS cap (vendor-reported, early)
MLPerf Inference v6.1: Qwen3-VL-235B-A22B on vLLM + Dynamo, up to 3.7x vs GB300 NVL72
CUDA 13.4 locality domains split MoE weights for ~1.2x average MoE-layer speedup in small-token decode
Image: vllm/vllm-openai:cu134-nightly (CUDA 13.4 + PyTorch 2.15)

Read full story

Linked Models & Agents:

⚡ MiniMax-M3⚡ Qwen3-VL-235B-A22B⚡ DeepSeek-V4.1-Flash ⚡ Kimi K3 🌐 vLLM 🌐 NVIDIA 🌐 Inferact 🌐 Red Hat

#vLLM#Vera Rubin#MoE#GPU#MLPerf

NVIDIA@nvidia·Collected 32d ago

👀140

🔥96

NVIDIA DGX Spark 64GB: OEM sale from Oct 23 at $4,999; dual-node up to ~1.7×, local models to ~100B

On 2 Oct 2026 NVIDIA blogged DGX Spark 64GB: on sale Friday 23 Oct via Acer/ASUS/Dell/Gigabyte/HP/MSI from $4,999. Same GB10 Grace Blackwell, DGX OS, and full NVIDIA AI stack; 64GB unified memory for local agents up to ~100B params. Two units cluster over ConnectX-7+QSFP to 128GB (~200B); Qwen 3.8 27B test showed up to ~1.7× vs one node.

Key Takeaways

Official blog by Allen Bourgoyne dated 2026-10-02
OEM availability Friday 2026-10-23 from Acer/ASUS/Dell/Gigabyte/HP/MSI; starting at $4,999; 64GB OEM-only
Same GB10 + DGX OS + CUDA AI stack; Agent Toolkit, Ollama, vLLM, PyTorch, Nemotron OOTB
Dual-node via ConnectX-7+QSFP pools 128GB / ~200B params; Qwen 3.8 27B up to ~1.7× vs single
Later this month: Sync Model Launcher for Qwen3.8 27B + OpenCode local wiring

Read full story

Linked Models & Agents:

⚡ Qwen3.8-27B⚡ Nemotron🤖 NVIDIA Agent Toolkit 🤖 OpenCode 🤖 Ollama 🌐 NVIDIA 🌐 Acer 🌐 ASUS 🌐 Dell 🌐 Gigabyte 🌐 HP 🌐 MSI

#Nvidia#Dgx spark#Gb10#Local AI#Coding agents#Ollama#Vllm#Opencode#Grace blackwell

InternLM / Shanghai AI Lab@InternLM·Collected 37d ago

👀87

🔥91

LMDeploy 0.18.0: Qwen3.5 dflash, SM90 quantized GEMMs, Ascend GLM-5.2; TurboMind W4A16 + piecewise CUDA Graph

Shanghai AI Lab’s LMDeploy shipped v0.18.0 on 2026-09-28: input logprobs, expanded SM90 quantized GEMMs with unified TurboMind linear paths, Ascend GLM-5.2, Qwen3.5 dflash, and MoE shared-expert/FFN sharding. PyTorch path prototypes TurboMind W4A16 (AWQ), piecewise CUDA Graph prefill, XTuner TileLang sparse MLA, checkpoint-engine weight updates, and request-only KV cache metrics.

Key Takeaways

Shipped v0.18.0 — Qwen3.5 dflash, Ascend GLM-5.2, SM90 quantized GEMMs
TurboMind W4A16 AWQ prototype + piecewise CUDA Graph prefill + TileLang sparse MLA
checkpoint-engine weight updates + request-only KV cache metrics + input logprobs
Upgrade: pip install -U "lmdeploy>=0.18.0"

Read full story

Linked Models & Agents:

⚡ Qwen3.5⚡ GLM-5.2 🌐 InternLM 🌐 Shanghai-AI-Lab

#LMDeploy#TurboMind#Qwen3.5#Dflash#AWQ#CUDA Graph#Ascend#Inference

Unsloth AI@UnslothAI·Collected 39d ago

👀99

🔥93

Unsloth Ships v0.1.815 & Prebuilt CUDA 13 Wheels: Run Qwen-Image-2.1 Locally with Claude-Compatible Agent Skills

Unsloth AI released version v0.1.815-beta alongside official prebuilt Linux x86_64 CUDA 13 wheels for Flash-Attention 2.8.4, Causal-Conv1D 1.7.0, and Mamba-SSM 2.3.2.post1 targeting PyTorch 2.13/2.14 on Python 3.13. The release enables fast local deployment of Alibaba's Qwen-Image-2.1 across both consumer and datacenter GPUs using Fast FP8 and GGUF quantization (defaulting to Q4_K_M) for combined image generation and precise image editing. Crucially, the Unsloth workspace introduces custom Agent Skills compatible with Claude Code and .agents formats, featuring @skill tagging in chat and 2x faster reasoning blocks (boosting UI rendering from 30 FPS to 60 FPS).

Key Takeaways

Official CUDA 13 prebuilt wheels: instant binaries for Flash-Attention 2.8.4 and Mamba-SSM on PyTorch 2.13/2.14, eliminating compilation errors
Local Qwen-Image-2.1 execution: Fast FP8 and GGUF Q4_K_M support for both high-fidelity image generation and interactive image editing
Claude Code skill interoperability: native support for importing and executing custom skills from .agents and Claude Code folders with @skill
60 FPS reasoning blocks: doubled chain-of-thought rendering performance from 30 FPS to 60 FPS with reworked UX
Expanded platform support: in-app Debian upgrades and a dedicated installer for Ubuntu 24.04+ on ARM64 architectures

Read full story

Linked Models & Agents:

⚡ Qwen-Image-2.1⚡ Qwen-Image🤖 Claude Code 🤖 Unsloth Agent 🌐 unslothai 🌐 Alibaba

#Unsloth#Qwen-Image#Agent Skills#CUDA 13#Flash-Attention#Local LLM#Fine-Tuning

ggml-org@ggerganov·Collected 41d ago

👀100

🔥95

llama.cpp v0.5.0: CUDA conv2d implicit GEMM, Metal MoE/SSM fusion, MiMo-V2.6 & HunyuanOCR & Mimir 1B

llama.cpp v0.5.0 (2026-09-23) accelerates CUDA conv2d via implicit GEMM, adds Metal MoE and SSM_CONV fusion, lets the server bind multi-address/--host lists and UNIX sockets, and adds HRM-Text (DFM Mimir 1B), MiMo-V2.6, and HunyuanOCR (DFlash) conversion plus ggml 0.25.0 backends.

Key Takeaways

Shipped v0.5.0 on 2026-09-23 — major step past the prior v0.4.1 line
Perf: CUDA conv2d implicit GEMM; Metal MoE + SSM_CONV fusion; CUDA graphs for MTP drafting
Models: HRM-Text/Mimir 1B, MiMo-V2.6 convert, HunyuanOCR DFlash, Qwen4Exp sparse FA, Nemotron MTP/H
Server: multi-address/--host lists + UNIX sockets; input_image on function-call outputs; router race fixes
API: LoRA from FILE*; sampler env vars; llama-bench --version

Read full story

Linked Models & Agents:

⚡ qwen⚡ deepseek⚡ nemotron🌐 ggml-org

#llama.cpp#Ggml#CUDA#Metal#local LLM#MiMo#HunyuanOCR#Inference

vLLM@vllm_project·Collected 43d ago

👀100

🔥93

View on 𝕏 ↗

vLLM v0.30.0 ships 762 commits with hot paths for Kimi K3, DeepSeek-V4.1-Flash, Qwen3.8-Flash-Next

The vLLM project (@vllm_project) released v0.30.0 with 762 commits from 315 contributors. Highlights include hybrid-attention hot paths for Kimi K3 (KDA/AttnRes/MLA), DeepSeek-V4.1-Flash (MXFP8 KV and async Engram), and Qwen3.8-Flash-Next (QSA/PLE, lower sparse-GQA overhead); plus HiSparse host-tier decode, Model Runner V2 EAGLE3 pipeline parallelism with adaptive verification, dual-key Gumbel-max watermarking, Fast Start CUDA IPC weight cache, and new models such as GLM-5.3-Flash, K2-Horizon, Cohere Compass, and Bailing V3 VL.

Key Takeaways

Hybrid-attention hot paths cover Kimi K3, DeepSeek-V4.1-Flash, and Qwen3.8-Flash-Next.
Model Runner V2 brings EAGLE3-style drafts to pipeline parallelism with online acceptance estimation.
Fast Start maps post-quantized TP-sharded weights over CUDA IPC for faster restarts.

Read full story

Linked Models & Agents:

⚡ Kimi K3 ⚡ DeepSeek-V4-Flash ⚡ Qwen3.8-Max 🌐 vllm

#vLLM#v0.30.0#Kimi K3#DeepSeek-V4.1-Flash#Qwen3.8-Flash-Next#speculative decoding

Sayak Paul@RisingSayak·Collected 47d ago

👀40

🔥90

View on 𝕏 ↗

Hugging Face Kernels: swap optimized layer kernels without rewriting the model

Hugging Face engineer @RisingSayak highlighted 🤗 Kernels (huggingface.co/docs/kernels): when you find a faster kernel, replace a supported layer’s forward() instead of rewriting the model. You can map different kernels for inference vs training and per device type, register choices globally or locally, then keep what benchmarks help. Hub dynamic loading ships via the kernels Python package (torch≥2.5 + CUDA), with community kernels in kernels-community.

Key Takeaways

Swap optimized kernels by replacing a layer’s forward()—no model rewrite.
Map different kernels for inference vs training and per device type, globally or locally.
Hub loading via the kernels package (torch≥2.5/CUDA); docs and kernels-community available.

Read full story

Linked Models & Agents:

🌐 huggingface

#Hugging Face#Kernels#CUDA#PyTorch#Hub#Inference#Training