Arun Prabhakaran measures Rank-8 LoRA training on Qwen3-4B

This article provides a first-principles breakdown of neural network learning mechanisms—covering backpropagation, AdamW optimization, residual connections, and Rank-8 LoRA fine-tuning—measured empirically using live GPU telemetry on the Qwen3-4B model. The author demonstrates why standard mixed-precision training requires 68.87 GB of VRAM and how Rank-8 LoRA reduces training state to 198.18 MB while completing updates in 21.9 seconds. By initializing adapter matrix B to zero and A randomly, LoRA preserves the pretrained base weights while enabling efficient domain adaptation. Open article →