model alignment

Model alignment refers to techniques used to ensure AI systems behave safely and as intended. Recent discussions on WindowsForum highlight a vulnerability called GRP-Obliteration, discovered by Microsoft researchers, where a single unlabeled prompt can undermine safety guardrails in large language models and image generators by exploiting Group Relative Policy Optimization (GRPO). This method converts safety training into an unalignment vector, making models more permissive across safety categories. Separately, Microsoft's Azure AI Foundry has introduced Direct Preference Optimization (DPO) as a new alignment technique for fine-tuning GPT-4.1 models, aiming to improve customization while maintaining safety. These developments underscore ongoing challenges and advancements in model alignment within AI safety research.