attention analysis

Attention analysis is a technique used to detect backdoors in large language models (LLMs), as demonstrated in recent Microsoft research. The approach identifies three observable signatures of potential model poisoning: an attention "double triangle" pattern, memorized leakage of poisoning data, and fuzzy trigger activation. A lightweight scanner performs forward-pass-only detection to reconstruct likely triggers, helping security teams and model consumers reduce the risk of deploying compromised models in production. This tag covers discussions of attention-based detection methods for LLM security, particularly in enterprise and AI deployment contexts.