AI Coding Live News

NVIDIA open-sources Nemotron 3.5 Bolt E4B 4GB, a Gemma 4 E4B-based small model for local assistants, characters and robots with tool use

On 2026-10-23 (15:41 UTC) the official @NVIDIAAI account announced Nemotron 3.5 Bolt E4B 4GB, a small open model for local assistants, interactive characters and robots that can hold a conversation and use tools. The official Hugging Face model card says it is post-trained from Gemma-4-E4B-it, has 8B total parameters (4B effective), ships in Q4_K_M quantization, supports up to 64K input and 64K output tokens, and has a switchable thinking mode, plus a DFlash speculative-decoding drafter and a BF16 version. Weights, data and recipes are open under the OpenMDW-1.1 license, with commercial use allowed. On the model card the 4GB build scores 55.60 on MultiChallenge (Gemma4 E4B-it at the same quantization 39.36, Qwen3.5 4B 52.23), but trails Qwen3.5 4B on Tau2-Bench (52.32) and BFCL-v3 (67.31).

NVIDIA open-sources Nemotron 3.5 Bolt E4B 4GB, a Gemma 4 E4B-based small model for local assistants, characters and robots with tool use

Key Takeaways

  • NVIDIA officially released Nemotron 3.5 Bolt E4B 4GB for local assistants, interactive characters and robots
  • Post-trained from Gemma-4-E4B-it: 8B total parameters (4B effective), Q4_K_M quantization, 64K context, switchable thinking mode
  • Open weights, data and recipes under the OpenMDW-1.1 license with commercial use allowed, plus a DFlash speculative-decoding drafter
  • Model card results: 55.60 on MultiChallenge leads its size class, but it trails Qwen3.5 4B on Tau2-Bench and BFCL-v3
Read full story
Linked Models & Agents:
⚡ Nemotron 3.5 Bolt E4B⚡ Gemma 4 E4B🌐 NVIDIA🌐 Google

Krafton AI Open-Sources Raon-1 Mini: A 30B MoE with 4B Active Parameters Matching Muse Glimmer on Coding Autonomy

Krafton AI, the research division of global gaming giant Krafton, officially announced the open-weights release of Raon-1 mini under Apache 2.0. Pretrained from scratch, Raon-1 mini features a sparse Mixture-of-Experts (MoE) architecture with 30 billion total parameters and only 4 billion active parameters per token. The release is accompanied by a dedicated DFlash speculative drafting model and a live interactive playground. Internal evaluations reveal that Raon-1 mini matches Meta's Muse Glimmer across agentic tool-use and coding benchmarks while requiring substantially lower VRAM and sustaining lower generation latency.

Krafton AI Open-Sources Raon-1 Mini: A 30B MoE with 4B Active Parameters Matching Muse Glimmer on Coding Autonomy

Key Takeaways

  • Krafton AI released Raon-1 mini under Apache 2.0, architecting a 30B total / 4B active parameter sparse MoE model
  • Trained entirely from scratch, matching Meta's Muse Glimmer across agentic coding and multi-turn tool orchestration
  • Bundled with a dedicated DFlash draft model for high-throughput speculative decoding on consumer-grade workstations
  • Accompanied by a live public evaluation playground, providing a viable open backbone for local gaming and coding copilots
Read full story
Linked Models & Agents:
⚡ Raon-1 mini⚡ Muse Glimmer🌐 Krafton AI

LMDeploy 0.18.0: Qwen3.5 dflash, SM90 quantized GEMMs, Ascend GLM-5.2; TurboMind W4A16 + piecewise CUDA Graph

Shanghai AI Lab’s LMDeploy shipped v0.18.0 on 2026-09-28: input logprobs, expanded SM90 quantized GEMMs with unified TurboMind linear paths, Ascend GLM-5.2, Qwen3.5 dflash, and MoE shared-expert/FFN sharding. PyTorch path prototypes TurboMind W4A16 (AWQ), piecewise CUDA Graph prefill, XTuner TileLang sparse MLA, checkpoint-engine weight updates, and request-only KV cache metrics.

LMDeploy 0.18.0: Qwen3.5 dflash, SM90 quantized GEMMs, Ascend GLM-5.2; TurboMind W4A16 + piecewise CUDA Graph

Key Takeaways

  • Shipped v0.18.0 — Qwen3.5 dflash, Ascend GLM-5.2, SM90 quantized GEMMs
  • TurboMind W4A16 AWQ prototype + piecewise CUDA Graph prefill + TileLang sparse MLA
  • checkpoint-engine weight updates + request-only KV cache metrics + input logprobs
  • Upgrade: pip install -U "lmdeploy>=0.18.0"
Read full story
Linked Models & Agents:

llama.cpp v0.5.0: CUDA conv2d implicit GEMM, Metal MoE/SSM fusion, MiMo-V2.6 & HunyuanOCR & Mimir 1B

llama.cpp v0.5.0 (2026-09-23) accelerates CUDA conv2d via implicit GEMM, adds Metal MoE and SSM_CONV fusion, lets the server bind multi-address/--host lists and UNIX sockets, and adds HRM-Text (DFM Mimir 1B), MiMo-V2.6, and HunyuanOCR (DFlash) conversion plus ggml 0.25.0 backends.

llama.cpp v0.5.0: CUDA conv2d implicit GEMM, Metal MoE/SSM fusion, MiMo-V2.6 & HunyuanOCR & Mimir 1B

Key Takeaways

  • Shipped v0.5.0 on 2026-09-23 — major step past the prior v0.4.1 line
  • Perf: CUDA conv2d implicit GEMM; Metal MoE + SSM_CONV fusion; CUDA graphs for MTP drafting
  • Models: HRM-Text/Mimir 1B, MiMo-V2.6 convert, HunyuanOCR DFlash, Qwen4Exp sparse FA, Nemotron MTP/H
  • Server: multi-address/--host lists + UNIX sockets; input_image on function-call outputs; router race fixes
  • API: LoRA from FILE*; sampler env vars; llama-bench --version
Read full story
Linked Models & Agents:
⚡ qwen⚡ deepseek⚡ nemotron🌐 ggml-org
ADSponsored

LightSeek open-sources Kimi K3 draft trio: EAGLE-3, DFlash2, DSpark via TorchSpec + vLLM

LightSeek Foundation released a Kimi K3 draft-model collection (EAGLE-3, DFlash2, DSpark) trained with TorchSpec on 40 NVIDIA GB200 GPUs, with open data recipes. Their blog reports acceptance length across 10 benchmarks and SPEED-Bench end-to-end throughput; DFlash2 averages ~4.02 accepted tokens and leads EAGLE-3 at lower concurrency.

Key Takeaways

  • Three draft architectures for Kimi K3 speculative decoding: EAGLE-3, DFlash2, DSpark.
  • Trained on 40×GB200 with TorchSpec; data recipes plus Docker/CI shared.
  • On HumanEval, DFlash2 acceptance length ~5.08 vs EAGLE-3 ~3.29.
Read full story
Linked Models & Agents:

vLLM 0.28.0 Released: Full-Stack Kimi-K3, DeepSeek-V4 Sparse MLA, and Tiered Disk KV Cache

vLLM 0.28.0 landed with 584 commits, bringing stack-wide Kimi-K3 support, end-to-end DeepSeek-V4 sparse MLA with DFlash2/DSpark speculative decoding, Model Runner V2 disaggregation, and tiered disk KV cache.

Key Takeaways

  • Adaptive speculative token budgets slash end-to-end TTFT by 55–65% with 1.5–3x faster sequence-parallel kernels;
  • End-to-end DeepSeek-V4 sparse MLA and Kimi-K3 shared-expert sharding save ~17 GiB VRAM per GPU;
  • Tiered KV cache introduces a disk tier, doubles max batched tokens to 16,384, and supports diverse hardware backends.
Read full story

Meta AI Releases Muse Glimmer: 30B Open-Weight Always-On Local Agent Model Under 20GB VRAM

Meta AI released Muse Glimmer, an Apache 2.0 licensed 30B open-weights model engineered for always-on local desktop agents under 20GB VRAM with DFlash speculative acceleration.

Key Takeaways

  • Released under permissive Apache 2.0 license with public weights and developer guides on Hugging Face;
  • Tuned specifically for agent loops to autonomously discover devices, query APIs, and build local web apps;
  • DFlash speculative decoding maintains high-throughput interactivity on consumer-grade Macs and PCs.
Read full story
Linked Models & Agents: