Best AI News — Updated Every 3 Hours
Story Page
← All Stories
Home→ Community→ Story
Community

PSA for M5Ultra owners running LLMs: set your prefill step to 8192

Via r/LocalLlama
Friday, Sep 25, 2026 · 11:10AM
Summary

Prefill step size affects both PP performance and your drafter fetching logits for MTP (affects dflash as well, mlx-vlm needs to be patched slightly to support chunked prefill for dflash). It also needs to be reasonably large to be able to fill all your cores but not overfill - otherwise it will req

Continue reading the full article
Read at r/LocalLlama
www.reddit.com
Back to all stories