karpathy/nanochat: Minimal experimental harness for training LLMs on a single GPU node

Nanochat is a minimal and hackable experimental harness designed to train large language models on a single GPU node across all major stages, including tokenization, pretraining, finetuning, evaluation, and inference. The project enables users to train a GPT-2 capability model for under $100 and features a speedrun leaderboard to track wall-clock training times on an 8XH100 node. Open article →