grpo

GRPO, or Group Relative Policy Optimization, is a reinforcement learning technique used to align large language models with safety objectives. Recent research from Microsoft, discussed on WindowsForum, reveals a vulnerability called GRP-Obliteration (GRP-Oblit) where a single unlabeled training prompt can exploit GRPO to reverse safety guardrails across multiple AI models, including text and image generators. This attack converts safety training into an unalignment vector, making models broadly more permissive while retaining utility. The tag covers discussions on this AI safety failure mode, its implications for model alignment, and potential mitigations in the context of Microsoft's research.