Microsoft Entra ID Risk Scores: Why AI Still Needs Evidence to Resolve Alerts

The core argument: a score is not a verdict​

Pettit's point is simple. Impossible-travel and anomalous-login alerts arrive from Okta, Microsoft Entra ID and Google Workspace. A better model can push the nastiest ones to the top of the queue, but an analyst still has to decide whether the login was malicious.

He lays out the division of labor across the usual stack:

• Identity providers flag behavior outside the norm, such as an unfamiliar location, an unexpected IP or travel that is physically impossible. What they return is a hedged risk label.
• A SIEM pulls events from many sources together, but a human still interprets them.
• SOAR can challenge a user or revoke a session. That settles easy cases but doesn't show what the evidence says.

His example is an impossible-travel alert sitting at the top of the queue. Someone still has to check for travel or a VPN, whether the device is known, what endpoint telemetry shows, and above all what the account did after signing in. An account that logs in from an odd place and then behaves normally is a different case from one that starts touching applications it never uses.

The numbers, and how far they stretch​

The commentary leans on three statistics. They need careful scoping.

• SpyCloud's 2026 Identity Exposure Report. Sign-in risk detections represent the probability that a given authentication request isn't from the authorized owner of the account. Microsoft's documentation also says some detections, such as leaked credentials and verified threat actor IPs, are always high risk because there is proof of the condition. Others, such as unfamiliar sign-in properties, can fire at high, medium or low depending on confidence.

Scores can change after the fact. Real-time detections surface in reports within about 5 to 10 minutes. Offline detections can take up to 48 hours, so a sign-in's risk level can change after the fact. That supports Pettit's point that the first label is a starting signal.

Policy can intervene automatically. ID Protection calculates some risks in real time or offline after authentication, and detecting risk at sign-in lets users self-remediate and admins investigate quickly. Risk-based Conditional Access can require MFA, require reauthentication or block access. For user risk, it can require a password change or "Require risk remediation." A user who completes the required step clears the risk, and Microsoft says this cuts the investigation burden on administrators. Risk-based access policies require Entra ID P2.

Microsoft's score is no longer a single-sign-in view. Unified risk signals correlate identity signals across products in the same time window. Sources include Defender for Identity, Defender for Endpoint, Defender for Cloud Apps and Purview. Per Microsoft's documentation, several medium or low-confidence signals together can now raise a user to high risk. Defender for Identity must be configured for compounding to work. One of Microsoft's example scenarios is session cookie theft, where an anomalous token detection plus a suspicious OAuth app are scored together.

So the fair reading is narrower than the headline. Entra's score is a signal that feeds policy and investigation. It is not, on its own, an auditable account of everything the identity and its devices did afterward.

What Microsoft's own investigation guidance asks analysts to do​

Microsoft's guidance reads like a human checklist, which supports the "someone still investigates" thesis:

1. Review the ID Protection dashboard and risk reports for risky users, sign-ins and detections.
2. Review sign-in logs for similar activity with the same characteristics, which could indicate more compromised accounts.
3. Review which resources might be compromised, including potential data downloads or administrative modifications.
4. Check whether the app is commonly used, whether the device is registered or compliant, and whether the location fits the user.
5. Where available, check Microsoft Sentinel or Defender XDR for related alerts. In the Defender portal, the Product name filter for AAD Identity Protection surfaces Entra alerts.
6. Contact the user, with a caveat from Microsoft that email or Teams might itself be compromised.
7. Mark the sign-in confirmed compromised, confirmed safe, or dismissed.

For atypical-travel detections, Microsoft's guidance is concrete. Confirm as safe if the user really traveled or uses that IP in their duties. If the range belongs to a sanctioned VPN, confirm as safe and add the range to named locations. If the activity wasn't the user, mark it compromised and reset the password. Block the user if the attacker can reset the password or perform MFA. Revoke tokens as appropriate.

Microsoft also suggests ways to reduce false positives: add corporate VPN and IP ranges to named locations, and consider a known-traveler list.

Entra's own "verdict" buttons​

Risk reports expose the human decision explicitly. In the Risky sign-ins report, administrators can confirm a sign-in compromised (a true positive), confirm it safe (a false positive), or dismiss the risk (a benign true positive, such as a known penetration test). That feedback improves detection. The reports hold filterable data for up to 30 days for risky sign-ins and 90 days for risk detections, and can be exported or streamed to a SIEM.

Microsoft also notes that its system can dismiss a risk on its own with the detail "AI confirmed sign-in safe." That happens when a risk was a false positive or was remediated by policy, for example by completing MFA or a secure password change. That is automation resolving some cases, but only in the narrow, policy-driven sense.

If you have Security Copilot, the risky users report can also provide a natural-language summary of why a user's risk was elevated and how to respond.

A checklist for judging "AI SOC" tools​

The strongest part of the commentary is its buying guidance. Treat these as questions to ask a vendor, not as proven facts about any product:

• Reach. Can it investigate across the systems a given alert needs, such as identity, endpoint, cloud, email and SIEM?
• Determination. Does it state what it concluded, or just hand back a number?
• Evidence. Can another analyst see every query run and the reasoning, and reproduce it?
• Honesty. Can it return "inconclusive" when evidence is thin, instead of forcing a verdict?

Pettit names Prophet Security as a tool that does this for IdP alerts, including anomalous logins, MFA fatigue and session hijacking. He says it correlates identity data with endpoint, SIEM and email context and attaches evidence. That is the vendor's capability claim. Pettit's description of it as a "leading" platform is promotional, and nothing here independently validates its accuracy in real environments.

The Microsoft-shop takeaway​

• Don't judge your Entra deployment by whether alerts have scores. Judge it by whether risk drives enforced policy and whether analysts can reconstruct what happened afterward.
• Check your licensing. Risk-based Conditional Access needs Entra ID P2, and unified risk signals depend on Defender for Identity being configured.
• Microsoft says legacy risk policies configured directly in ID Protection retire on October 1, 2026, and recommends moving them to Conditional Access. If you still have them, check that your equivalent policies exist.
• Treat location anomalies as questions. Pair the risk report with sign-in logs, device state and post-login activity.
• Be skeptical of any vendor claim to "resolve" alerts unless it shows its work.

Bottom line​

The commentary's thesis holds up: a risk score prioritizes, and resolution takes evidence, interpretation and a defensible conclusion. Microsoft's documentation adds nuance. Entra can enforce, self-remediate and correlate across Defender products, but its own guidance still hands the final true-positive call to a person. The practical test for any identity tool, Microsoft's or a third party's, is whether the verdict arrives with the evidence behind it.