Show HN: A layer on top llama.cpp enabling big MoE streaming on phone and laptop
By HelldezNoBan · 2026-10-20 · 1 points · 0 comments
https://github.com/Helldez/BigMoeOnEdge
Adaptive Streaming Optimizer: Run any Big MoE model on any device, any OS, any HW, local OpenAI-compatible server. Android, Windows, Mac, IOS, Linux. CPU, GPU, NPU (35B on phone 12gb RAM 10 tok/s decode 100-300+ prefill, Windows, Mac even more: 25+ tok/s 16 gb RAM)
Open the full discussion on BetterNews