Study benchmarks 12 open-weight LLM architectures

This project examines the architectures of twelve open-weight large language models ranging from 1.7B to 35B parameters to understand how different design approaches affect performance, memory costs, and computational efficiency. The study analyzes various strategies addressing memory and compute limitations, including grouped-query attention, mixture of experts, hybrid recurrent state space models, quantization, and ternary weights. Open article →