Evil Qwen and activation steering: why SBOMs aren’t enough

A viral toy demonstration of activation steering on open-weights language models highlights how minor modifications to a single internal neuron can bypass safety alignment while preserving functional capabilities. Researchers have shown that subtle control vectors or compromised inference code can disable model refusals without altering model weights on disk, exposing a critical blind spot in traditional software bills of materials. The author argues that teams must deploy continuous behavioral evaluations and output guardrails rather than relying solely on provenance and static checks. Open article →