llama.cpp v0.5.0 (2026-09-23) accelerates CUDA conv2d via implicit GEMM, adds Metal MoE and SSM_CONV fusion, lets the server bind multi-address/--host lists and UNIX sockets, and adds HRM-Text (DFM Mimir 1B), MiMo-V2.6, and HunyuanOCR (DFlash) conversion plus ggml 0.25.0 backends.
Key Takeaways
- ✓Shipped v0.5.0 on 2026-09-23 — major step past the prior v0.4.1 line
- ✓Perf: CUDA conv2d implicit GEMM; Metal MoE + SSM_CONV fusion; CUDA graphs for MTP drafting
- ✓Models: HRM-Text/Mimir 1B, MiMo-V2.6 convert, HunyuanOCR DFlash, Qwen4Exp sparse FA, Nemotron MTP/H
- ✓Server: multi-address/--host lists + UNIX sockets; input_image on function-call outputs; router race fixes
- ✓API: LoRA from FILE*; sampler env vars; llama-bench --version
Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Core Background & Industry Pain Points
Local/edge stacks need CUDA and Apple Silicon plus new vision/MoE arches. Pain: slow conv2d/MoE kernels, single --host binding, lagging converters. llama.cpp v0.5.0 (2026-09-23) advances backends, model coverage, and server/router robustness with ggml 0.25.0.
Architecture Highlights & Internals
CUDA conv2d via implicit GEMM; Metal MoE + SSM_CONV fusion; CUDA graphs for MTP drafting. Server --host accepts comma-separated TCP addresses and UNIX sockets; function-call outputs may include input_image. New convert/arch paths: HRM-Text/Mimir 1B, MiMo-V2.6, HunyuanOCR DFlash, Qwen4Exp sparse FA, Nemotron MTP/H. API: LoRA from FILE*; sampler env vars.
Authoritative Benchmarks & Measured Scores
No official tok/s table in the notes — re-run llama-bench --version on your hardware. Multimodal correctness fixes include SigLIP bucket overruns.
Developer Hands-on Guide
Checkout v0.5.0, rebuild, and test multi-bind --host. See github.com/ggml-org/llama.cpp.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.