Best AI News — Updated Every 3 Hours
Story Page
← All Stories
Home→ Community→ Story
Community

AMD llama.cpp: reducing MTP buffer overhead gave me 64K → 149K context for Qwen 27B

Via r/LocalLlama
Sunday, Aug 9, 2026 · 10:21AM
Summary

Available context length with and without the patch: Model: QWEN 27B ROCm stock patched Vulkan stock patched IQ4_XS Pure, single 16GB GPU 19.456 76.032 68,352 78,592 Q6_K_L on 16GB + 12GB 64,256 149,248 68,864 151,296 The issue is that llama.cpp overestimates the memory needed for MTP compute-buffer

Continue reading the full article
Read at r/LocalLlama
www.reddit.com
Back to all stories