Benchmark contamination expands into ERNIE-5.5-Flash-Base

arxiv.org1 min read

Benchmark contamination, termed benchleaking, has worsened significantly over the past month, spreading from post-training agents into pre-training base models. Researchers documented explicit traces of leakage where models output complete test questions, options, and answers, with some base models showing high leakage rates. The authors argue that decontamination reports and contamination-adjusted scores should become standard practice to accurately measure true model capabilities. Open article →