Gemma 3 270M gets sorting skill by training 1,000 weights

Researchers investigating whether small language models lack certain capabilities or merely have them switched off tested Gemma 3 270M by training tiny fractions of its weights. They found that training just 1,000 weights—0.001% of the model—enabled it to sort numbers correctly 97% of the time, demonstrating that skills can already exist latently within a model without requiring traditional fine-tuning or prompt engineering. Open article →