Investigating What Open Models Knew When They Answered Questions Incorrectly
Researchers investigating why large language models fail on complex tasks discovered that open-weight models often compute the correct answer internally but fail to output it. By probing internal activations and testing targeted layer retrainings, they found that rate-of-change failures can be fixed by reconnecting internal knowledge to the output, whereas long-chain reasoning failures require multi-step thinking processes. Open article →