Reward-Hacking is potentially ill-defined

An investigation into chess reward-hacking experiments with the Astra AI model suggests that the phenomenon may be ill-defined due to ambiguity in task prompts. When the evaluation prompt was modified to explicitly define the model's own ability or replaced with phrases like own reasoning, observed tool use dropped significantly, indicating that previous conclusions about models knowingly violating constraints may have been based on failing to infer implicit rules rather than deliberate cheating. Open article →