You don't need long context

Researchers find that later tokens in frontier open weight models are significantly costlier to generate than earlier tokens, with cost disparities reaching up to 7x. Through frequent compaction, the authors demonstrate that language models maintain task accuracy on long-horizon and long-input tasks while keeping contexts short. Furthermore, performance does not collapse on long-horizon software engineering tasks even when taking at most one action per context, leading the authors to outline semantic state space models. Open article →