Quark: An Open-Source 165M Parameter Language Model Built End-to-End

Researchers developed and released Quark, a 165M-parameter language model trained end-to-end for about $4,200 in compute to test every stage of the model stack. The open-source model supports up to 128K tokens, achieves strong zero-shot benchmark scores for its size, and includes a checkpoint trained using reinforcement learning alone without supervised fine-tuning. Open article →