Reasoning models explained as token pathfinding

The author examines how reasoning models function through the lens of pathfinding in token space, arguing that reinforcement learning maximizes success probability across multiple paths rather than single-path convergence. This perspective clarifies why individual chains of thought are not guaranteed to be faithful and how reward hacking stems from both verifier misspecification and imperfect sampling. Furthermore, the analysis shows that the benefits commonly attributed to thinking longer during inference actually originate from search compute expended during training. Open article →