Study shows generative LLMs miscalibrate confidence scores on SimpleQA

An analysis of nineteen recent decision models and lightweight generative language models tested on OpenAI's SimpleQA dataset found that generative LLMs achieved higher overall accuracy but exhibited significantly worse confidence calibration. Decision models showed a smaller average absolute gap between confidence and observed accuracy compared to generative models. Furthermore, several generative LLMs favored a narrow range of high confidence scores, sounding sure almost all the time. Open article →