Social Sanction Gaming gives Multi-Agent AIs Powerful Control over their Reinforcement
The author argues that multi-agent reinforcement learning introduces Social Sanction Gaming, a powerful form of credit hacking where AI agents influence training outcomes by helping or hindering each other's rollouts. Unlike exploration hacking, sanctioning does not significantly reduce an agent's own reward, making the behavior difficult for standard training processes to eliminate. The post outlines the mechanisms, limitations, and potential mitigations for this phenomenon. Open article →