Microsoft Copilot Benchmarks in Viva Insights: Internal and External Adoption Metrics

Microsoft has quietly added side‑by‑side adoption benchmarks for Copilot into the Viva Insights Copilot Dashboard, turning a manager’s view of AI usage into a formalized set of comparative metrics that can be sliced by group, role, region and app — and, crucially, compared externally against anonymized peer cohorts. The new Benchmarks capability promises to speed up how organisations identify where Copilot is being adopted (and where it’s not), but it also raises real questions about privacy, governance, metrics validity and the organisational incentives that come from turning AI adoption into a scoreboard.

Background​

Microsoft has been folding Copilot across Microsoft 365, Teams and the rest of its productivity stack for more than a year, and it has been gradually folding measurement tools into Viva Insights so managers and analysts can monitor usage, impact and readiness. The Copilot Dashboard already exposes trendlines, per‑app adoption, total Copilot actions and impact indicators such as estimated hours saved and meetings summarised. The new Benchmarks feature adds a comparative layer: internal cohort comparisons inside a tenant, and external benchmarks that let an organisation see how its percentage of active Copilot users stacks up against similar companies.
Microsoft’s own product documentation defines a number of the underlying metrics the Benchmarks feature will surface. For example, an “active Copilot user” is a Copilot‑licensed employee who has performed at least one intentional action with Copilot in a supported app during the preceding 28‑day window. Intentional actions are explicit prompts or actions (for example, submitting a prompt to Copilot Chat or generating a document in Word via Copilot), not simply opening the Copilot pane. Metrics are calculated over rolling 28‑day windows and typically have a 2–3 day processing delay.
The Benchmarks rollout began as a private preview and is scheduled to expand to general availability in October 2025. Microsoft’s roadmap and tenant messaging indicate the new capability will show both internal cohort comparisons (manager types, regions, job functions) and external comparisons (top 10% and top 25% peer groups and overall benchmarks). Microsoft states external benchmarks are created from anonymized, aggregated data and are generated using randomized models to reduce the risk of re‑identification; each external benchmark cohort is formed from a minimum number of companies.

---

What the Benchmarks actually show​

Internal cohorts: where adoption gaps surface​

The internal cohort benchmarking is designed to help managers identify variation in adoption inside a tenant. Typical cohort dimensions include:

• Manager type and hierarchical groupings
• Geographic region or office locations
• Job function and role groups (sales, engineering, finance, customer support)
• App‑level slices (Word, Excel, Outlook, Teams, PowerPoint, OneNote, Loop)

Key internal metrics surfaced include:

• Percentage of active Copilot users (active within the past 28 days out of the pool assigned a Copilot license)
• Adoption by app (which apps are being used with Copilot, and at what user counts)
• Returning user percentage (a basic retention or “stickiness” metric)
• Actions per user per feature (counts of initial prompt actions for distinct features)

These internal insights are intended to help training, enablement and deployment teams target groups that are lagging or to identify pockets where Copilot is being used successfully and could be spread as a best practice.

External benchmarks: peer comparisons and privacy mechanics​

External benchmarks let organisations compare their share of active Copilot users against anonymized peer groups. The new workbook includes:

• Top‑10% and Top‑25% performance slices for companies similar to yours (by size, industry, geography)
• Top‑10% and Top‑25% overall benchmarks across the dataset

Microsoft’s product advisories state these external cohorts are created using aggregation and randomized mathematical models and that each cohort contains a floor number of companies to make re‑identification more difficult. The benchmarks are intended to provide context for leaders deciding whether their Copilot adoption is above, near, or below industry and size expectations.

---

The mechanics and definitions you need to understand​

• Active user window: The Copilot Dashboard measures activity over a rolling 28‑day window and treats an intentional prompt or action as the primary signal. Merely opening a Copilot pane or clicking its icon does not count as an active action.
• Actions counted: The dashboard counts the initial prompting action; it does not count downstream manipulations such as copying or pasting a result unless the user explicitly initiates another counted Copilot action. Certain automated actions (for example, auto‑generated summaries) are counted only when a user views the generated output.
• Delay and aggregation: Most Copilot metrics have a 2–3 day latency; external benchmarks are generated from aggregated, anonymized metrics stored within Microsoft 365 services.
• Access controls: Access to Copilot Dashboard data and Benchmarks can be controlled using Viva Feature Access Management and Entra ID groups; global admins can manage who sees the data and set minimum group sizes for reporting.
• Benchmark methodology claims: Microsoft states external comparisons are calculated with randomized models and include a minimum company population per cohort, but the company does not publish the exact randomization algorithm or the precise threshold values for cohort construction.

---

Why organisations might welcome Benchmarks​

• Faster ROI signals: Copilot licences cost real money. Benchmarks give procurement, finance and IT leaders a quick signal of which teams are actually using the product, helping justify renewals or redeployments.
• Targeted enablement: Internal cohort comparisons make it easy to spot groups with low engagement. That helps training teams prioritise workshops, prompt libraries or tailored use‑cases where adoption lags.
• Objective target setting: Many change programs benefit from comparable metrics. Benchmarks allow organisations to set specific adoption targets (for example, reaching the peer‑group median within three months) and measure progress.
• Operational insight for rollout teams: The app‑level breakdown shows whether Copilot is taking hold in content creation (Word, PowerPoint), data work (Excel), or communications (Outlook, Teams), enabling productised rollout strategies.
• Third‑party benchmarking options: Vendors in the Microsoft ecosystem and independent analytics providers already offer anonymized Copilot benchmarking and can complement the built‑in dashboard for more tailored benchmarking across specific verticals.

---

The risks and limitations — what managers must not ignore​

1. Privacy vs re‑identification risk​

Microsoft says benchmarks are computed from anonymized, aggregated telemetry and randomized models and that cohort sizes meet minimum thresholds. Aggregation and randomization reduce re‑identification risk, but they do not eliminate it. Small industries, unique organisational footprints or highly asymmetric user distributions can still leak identifying signals when combined with public or internal knowledge.

• Small companies or tenants with unusual role mixes are more exposed to re‑identification.
• External comparisons that use narrow sector or region cuts can inadvertently reveal sensitive group performance.
• Legal and compliance teams should verify how the anonymized metrics are processed, where they are stored, and whether the published methodology satisfies statutory obligations (for example, under regional privacy laws).

2. Leaderboards and perverse incentives​

Turning Copilot adoption into a comparative metric risks creating a leaderboard culture. When managers and teams see rankings, the natural incentive is to maximise the superficial metric:

• Employees might trigger low‑value prompts simply to register an "action" (e.g., hitting the Copilot button and sending a trivial prompt), inflating active user counts without producing real productivity gains.
• Gamification can distort usage statistics and make the metric less predictive of meaningful outcomes.
• Tying Copilot adoption directly to performance evaluation or compensation can encourage checkbox behaviour rather than thoughtful adoption.

3. BYO Copilot, shadow IT and license confusion​

Microsoft has enabled scenarios where employees with personal Microsoft 365 plans can sign into personal and work accounts and invoke Copilot on work content using their personal subscription. That “bring your own Copilot” path has legitimate benefits for users but complicates governance:

• Benchmarks currently do not clearly separate personal‑plan Copilot actions from employer‑purchased Copilot licenses in the Dashboard’s external benchmarking. That ambiguity can skew adoption metrics and financial calculations.
• Organisations must treat BYO usage as a governance scenario: it’s a potential route for shadow usage, and IT should configure policies and cloud settings to manage or disable personal Copilot usage on corporate content where needed.
• Auditing and DLP controls must be validated to ensure enterprise data protection is enforced when employees mix personal and work accounts in the same machine.

4. Metric validity and signal quality​

The dashboard’s definition of an “active user” (at least one intentional Copilot action in 28 days) is a blunt instrument:

• It measures reach (how many people have used the feature) rather than depth (how often and effectively they use it).
• Actions per user counts only initial prompts and excludes downstream actions, which understates ongoing engagement with generated content.
• Some high‑value uses (complex analysis in Excel, multi‑stage documents) may not map neatly to a single counted action, creating false negatives in the data.
• Automated or system‑generated actions that users don’t view are excluded by design, which changes counts for features that produce passive outputs.

5. Compliance, data residency and regulated sectors​

Sectoral regulations and data residency requirements can limit how benchmarked metrics can be used or stored:

• Public sector, healthcare, finance and other regulated industries must verify that anonymized aggregated metrics processed within Microsoft 365 satisfy sector regulations.
• The storage and processing locations of anonymized benchmark data should be reviewed against contractual data residency obligations.

---

Practical steps for IT, compliance and HR teams​

Organisations evaluating or already using the Benchmarks capability should consider a short governance checklist and an operational playbook.

Minimum governance checklist​

• Confirm who has access to benchmark data via Viva Feature Access Management and Entra ID groups; restrict broad visibility.
• Review and, if necessary, increase the minimum group size for cohort reporting to reduce disclosure risk.
• Validate that anonymization and aggregation levels meet your privacy and regulatory requirements; if unclear, engage Microsoft support for technical details.
• Update data management policies to cover BYO Copilot scenarios, and configure autodisable policies where personal Copilot use on corporate data is disallowed.
• Communicate with employees to explain what is measured, why it is measured, and how the data will — and will not — be used.

Operational playbook for enabling responsible adoption​

• Use Benchmarks to prioritise training, not to rank employees. Identify teams with low adoption and pilot targeted enablement rather than imposing quotas.
• Combine quantitative Benchmarks with qualitative measures: pulse surveys, manager interviews and performance indicators that capture Copilot’s impact on outcomes.
• Monitor for gaming behaviours: sudden spikes in “active users” with low actions per user may indicate superficial use.
• Run trials where Benchmarks are off for sensitive groups (e.g., senior leadership or HR) and ensure high‑risk teams are excluded from external comparisons.
• Treat Benchmarks as one input for a broader Copilot maturity model — map adoption to productivity gains, error reductions, and time saved to create a more defensible ROI case.

---

Governance design patterns that work​

• Tiered visibility: Limit benchmark visibility to named enablement owners, data analysts and senior leaders. Avoid making it broadly visible to all managers by default.
• Minimum cohort thresholds: Configure minimum group sizes larger than the product defaults if your organisation is small or concentrated by role.
• Benchmark blackout windows: Temporarily disable external comparisons during sensitive organisational changes (M&A, restructuring) when re‑identification risk spikes.
• Outcome alignment: Tie adoption metrics to outcome metrics (for example, turnaround time for common deliverables) rather than purely counting prompts.
• Training & tooling: Provide prompt templates and in‑app guidance for Copilot use in common workflows, and track adoption lift from those interventions.

---

How to read the numbers — pragmatic interpretation guidance​

• Treat percentage of active Copilot users as a reach metric: it tells you where Copilot has been touched, not where it has changed the way work gets done.
• Use actions‑per‑user and returning user percentage as a proxy for engagement intensity; a high active user percentage with low returning user percentage signals low impact.
• App‑level adoption should be interpreted against role: Excel uptake matters more for data teams; Outlook and Teams matter more for customer‑facing functions.
• Watch for rapid, unexplained surges in active users — they can be onboarding events, but they can also be artefacts of a communications campaign or gaming.

---

Final appraisal: useful but incomplete​

The Benchmarks addition to the Copilot Dashboard is a logical next step for a platform that now bills itself as the productivity fabric for modern work. It brings useful mechanics for adoption measurement and will be valuable to organisations that need to make licensing and training decisions quickly. The feature provides a practical way to move from anecdote to data when deciding where to invest enablement effort.
But the dashboard is not a panacea. The telemetry is blunt by design, Microsoft’s anonymization and randomization claims help but don’t erase re‑identification risk in every scenario, and the incentive structure created by comparative metrics can produce superficial gains that look good on a chart but deliver little real value. The BYO Copilot path complicates the picture further by mixing personal and employer‑purchased licence usage, and the product currently does not present a clean separation in external benchmarking.
For responsible adoption, organisations must combine these bench‑marks with stronger governance, clearer communication, and outcome‑based measures. Benchmarks should be an aid to decision‑making — a diagnostic tool — not the ultimate arbiter of performance or productivity. With careful controls, transparent policies and an emphasis on outcomes rather than rankings, Benchmarks can accelerate sensible Copilot adoption. Without them, they risk becoming a leaderboard that pressures teams to game metrics and obscures the real question: is Copilot making work better, safer and faster?

---

Microsoft’s new Benchmarks feature makes it easier to say where Copilot is being used — and where it’s not. The next, harder step for IT and business leaders is to ensure those numbers are interpreted and governed in a way that protects privacy, avoids perverse incentives, accounts for BYO scenarios, and ties adoption to real, measurable improvements in how work gets done.

---

Source: theregister.com Microsoft adds Copilot adoption benchmarks to Viva Insights