ShinMegamiBoson/lookatmyquant: A quantization compiler with trellis codes and Apple Silicon runtimes
LookAtMyQuant is a quantization compiler that measures how stored model parts affect output to compress models within a size budget with minimal damage, such as reducing Gemma 4 E2B to achieve significantly lower KL divergence than Q4_K_M at the same file size. The compiler utilizes post-training quantization with trellis codes, Fisher information row allocation, and distillation. The software package also includes custom Apple Silicon Metal runtimes and speculative decoding support via a DSpine drafter. Open article →