LLMs can learn (some) multi-step reasoning on filler tokens

Researchers demonstrated that large language models can learn multi-step reasoning using filler tokens rather than open chain-of-thought text. By modifying training methods and utilizing LoRAs on models such as Qwen3.6-35B-A3B and gpt-oss-20b, the authors successfully trained models to perform multi-step reasoning tasks on filler tokens up to certain theoretical and empirical limits. Open article →