hyperionaidev/smolmaxxing: Running LLMs on minimal hardware devices

The smolmaxxing repository details techniques and custom kernels designed to run various large language models on extremely resource-constrained hardware, including a low-end Xiaomi TV box and a 2013 iPad Air. By implementing custom CPU kernels, flash weight streaming, expert pagers for mixture-of-experts models, and memory optimizations, the project successfully runs models ranging from 0.8B to 16B parameters at practical generation speeds. Open article →