The shelf life of LLM findings

Researchers revisiting 50 widely cited claims about large language models across 20 different models found that within 15 months of publication, most findings either no longer held or showed substantially diminished effects. Nearly half of the tested claims expired, with capability failures like citation fabrication largely disappearing in newer models, while preferences and biases tended to persist or strengthen. Open article →