Admins and makers should treat this as a preview with real limits. Here's what it does, how to set it up, and what to keep in mind.
What's new: feedback joins the usage numbers
Until now, Insights Agent dealt with usage metrics: how many people use an agent and whether that number is going up. Microsoft's own documentation describes the existing agent as reporting aggregate metrics only. It doesn't identify individual users, and it doesn't capture verbatim feedback. That documentation notes that "a separate capability" handles feedback summarization. The V2 Frontier preview is that capability.
Microsoft's blog post makes a simple case. Usage tells you whether an agent is catching on. It doesn't tell you what people like or what they want changed. A thumbs-down gives you a signal, and a written comment tells you why. The V2 preview puts both in one chat, so an owner can go from "people are using it" to "people think the answers are unclear" without exporting a spreadsheet.
The Message Center notice behind the rollout, MC1520901, has been republished by Microsoft 365 tracking site mwpro. It adds detail that the blog post leaves out. According to that notice, if you enable feedback sharing, agent owners can use Insights Agent to review user ratings, comments, and feedback themes for agents they own. Two more points will matter to privacy-minded admins: shared feedback is limited to your tenant, and submitter identities are not shown to agent owners.
Section summary: V2 adds feedback (ratings, comments and AI-generated theme summaries) to the existing usage analytics. Owners only see feedback for agents they own, and the person who left each comment isn't shown.
The admin switch: off by default
None of this works until an admin opts in. The MC1520901 notice says no action is required by default. Feedback sharing is off ("No agents"). Tenants that do nothing won't suddenly start showing employee comments to agent builders.
Microsoft Learn says the setting controls whether developers receive feedback about an agent, including thumbs-up or thumbs-down ratings and comments. Just as important is what the setting leaves alone: this setting doesn't affect whether people can rate agents or what data an agent can access. If you turn sharing off, users can still click thumbs-down. Their feedback just doesn't go to the agent's developers.
How to enable agent feedback sharing
- Sign in to the Microsoft 365 admin center.
- Go to Agents > Settings > Agent feedback sharing.
- Choose one of three options:
- No agents: developers get no feedback for any agent (the default).
- All agents: developers get feedback for every agent.
- Selected agents: developers get feedback only for the agents you pick.
- If you chose Selected agents, pick the agents whose feedback you want to share.
- Select Save.
Selected agents is probably the sensible place to start for most organizations. Try it on a few agents with active owners before turning it on for everything, including the experiment someone in Finance built at 11 p.m. and never touched again.
The Frontier prerequisite
Feedback sharing is only half of the setup. The Insights Agent feedback feature is a Frontier preview, which brings its own gate. Microsoft's Frontier documentation describes the program as tenant-managed, opt-in early access to features before general availability. It lists these requirements:
- The admin enabling Frontier needs one of these roles: AI Administrator, Security Administrator or Office Apps Administrator.
- Frontier access is set under Copilot > Settings > View all. Search for "Frontier" and select Copilot Frontier. The options are No access (the default), All users or Specific users and groups.
- To see Copilot-related Frontier features, users need a Microsoft Copilot license or another license that includes Copilot.
- Microsoft says it can take up to three hours for Frontier features and agents to show up after you enable them.
- Under Agents > Settings > Agent and plugin access, Allow agents and plugins built by Microsoft must be checked for Frontier agents to work.
- Since March 29, 2026, newly published Frontier agents are available only to Frontier-enrolled users.
The same documentation lists the usual preview caveats. Frontier experiences may be changed, suspended or discontinued. They may not be covered by standard support commitments or SLAs. HIPAA Business Associate Agreement coverage doesn't extend to Frontier experiences. Microsoft states these caveats for the program as a whole, not specifically for Insights Agent, but admins in regulated industries should read them before rolling anything out.
Section summary: Getting this working takes two separate admin decisions: enrolling users in Frontier, and turning on feedback sharing for specific agents. Both are off by default.
For makers: getting started
If you're enrolled in Frontier and your admin has turned on sharing for an agent you own, Microsoft's steps are:
- Open Microsoft 365 Copilot Chat.
- Search the agent picker for Insights Agent V2 (Frontier) and select Add.
- Start asking questions. Replace [Agent Name] with the name of your agent.
Microsoft's suggested prompts:
| Goal | Example prompt |
|---|---|
| Read raw comments | Show me recent verbatim comments for [Agent Name]. |
| Count ratings | How many thumbs up and thumbs down did [Agent Name] get this week? |
| Find patterns | Summarize the feedback themes for [Agent Name]. |
According to Microsoft, ratings answers include the number of responses and the time window, so a "90% positive" result based on ten votes over two days doesn't look as impressive as it might.
Don't over-trust the theme summary
Microsoft is unusually direct about this. Its blog calls the theme summary an AI-generated synthesis and says it doesn't replace reading the actual feedback. Microsoft recommends reading the original comments next to the summary before changing anything.
The source material gives no accuracy benchmark for theme summaries, no minimum sample size and no guarantee that every comment makes it into a theme. Think of the summary as a starting point for reading the comments. Microsoft gives an example of the intended loop: if several comments say an answer is unclear, check the agent's instructions or knowledge sources, make one change, test it, and then watch the next round of feedback.
Section summary: Use the summary to find patterns, then read the original comments before you change anything.
Usage analytics: the other half
The V2 experience also includes the usage questions that Insights Agent already handled. Microsoft's examples include:
- Which agents have the highest usage this week?
- What's the weekly active user trend for [Agent Name]?
- Compare adoption between [Agent A] and [Agent B].
Supported metrics include daily, weekly and monthly active users, unique-user trends, adoption growth, agent-to-agent comparisons and "stickiness," which is daily active users divided by monthly active users. Microsoft's sample-prompt documentation also describes a "rolling retention" ratio that compares unique users in back-to-back time windows. Microsoft explicitly says this isn't user-level cohort retention.
Some limits from Microsoft's troubleshooting guidance are worth knowing:
- The Insights Agent reports on agents you have access to. If you don't see the agent in your normal Copilot agent surface, you won't get data from the Insights Agent either.
- Install and uninstall counts aren't supported. Microsoft's advice is that, for now, you use activity metrics - DAU, WAU, MAU, and unique users over time - as a proxy for adoption.
- Questions about individual users, such as who used an agent or whether specific people came back, get a refusal message instead of a made-up number.
Analysis: useful, but know its limits
This section is analysis based on general industry experience, not Microsoft's claims.
This feature closes a real gap. Many organizations have dozens or hundreds of Agent Builder agents, and most of them get built, shared and then left alone. Their owners rarely hear how they're doing. Routing feedback to owners, while hiding who submitted it and keeping it inside the tenant, is a reasonable compromise between useful signal and employee privacy.
There are still some limits:
- Selection bias. People who click thumbs-down are usually the frustrated ones, and most satisfied users don't rate anything. Microsoft acknowledges this when it says feedback reflects only what users chose to share.
- Usage isn't value. Microsoft's blog warns against treating usage alone as a measure of quality or business value. A heavily used agent might just be the only one people know about.
- Anonymity isn't automatic in small groups. The submitter's identity is hidden, but a pointed comment about a niche agent used by five people might not be anonymous in practice. Admins should mention this when they announce the feature.
- Preview changes. It's a Frontier preview, so prompts, metrics and access rules may change before general availability, and nobody has announced a general availability date.
Checklist
IT admins:
- Read the Frontier program terms, especially the SLA and HIPAA caveats.
- Decide on a feedback-sharing policy before anyone asks, and start with Selected agents.
- Tell users that their ratings may now reach agent owners, without their names attached.
Makers and agent owners:
- Check your Frontier enrollment and agent-level sharing with your admin.
- Add Insights Agent V2 (Frontier) in Copilot Chat and start by reading the verbatim comments.
- Change one thing at a time, test it, and use later feedback to see whether it worked.
Agent owners finally get to hear from the people using their agents. Admins decide who gets that feedback, and owners should treat the AI summaries as a starting point, not a verdict.
References
- Explore user feedback with Microsoft 365 Insights Agent Microsoft Copilot Community · 2026-10-22T16:00:00+00:00
- MC1520901: Microsoft 365 Insights Agent v2 mwpro.co.uk
- Agent settings in Microsoft 365 admin center - Microsoft 365 admin learn.microsoft.com