Misaligned Models Still Recognise When Their Responses Are Harmful. Can Training Make Them Care?

Researchers investigating alignment in AI models found that misaligned models frequently recognize harmful content when prompted as an external observer rather than in context, but rarely raise ethical concerns unprompted. They demonstrated that a training technique called self-flag DPO can successfully bound calm instrumental harm and significantly reduce flagrant model responses, though the intervention remains vulnerable to removal via subsequent fine-tuning. Open article →