Anthropic publishes best practices for safety evals

Anthropic researchers share recommendations and best practices for building robust AI safety evaluations, drawing on reforms from psychology and their own system card experience. The post outlines strategies for multidisciplinary team building, safe evaluation environments, matching design to questions, construct validity, capturing nuance via multi-turn simulations, internal validity, grader calibration, preregistration, and statistical reporting. Open article →