Best AI News — Updated Every 3 Hours
Story Page
← All Stories
Home→ Community→ Story
Community

A llama.cpp PR makes Q2_0 3.0–3.6x faster on x86 CPUs, 8B decode goes 2.39 → 8.20 tok/s

Via r/LocalLlama
Friday, Aug 7, 2026 · 12:27PM
Summary

I was going through the current llama.cpp CPU PRs and #26348 stood out because this isn't the usual +5% kernel optimization. It adds an x86 VNNI implementation for the Q2_0 × Q8_0 dot product, and the author's controlled CPU-only benchmarks show roughly 3–3.6x higher throughput across Bonsai models

Continue reading the full article
Read at r/LocalLlama
www.reddit.com
Back to all stories