A sweeping industry report released this year warns that Microsoft’s Copilot is touching orders of magnitude more sensitive business data than many IT teams realise — with Concentric AI reporting that Copilot accessed nearly three million confidential records per organisation in the first half of 2025 — raising hard questions about governance, data residency, and attack surface as generative and agentic AI become embedded in everyday workflows.
Corporate use of generative AI assistants has surged: organisations are embedding Copilot and similar tools into email triage, document drafting, incident response, and even regulated workflows in healthcare, finance and government. That rapid adoption is colliding with long-standing enterprise problems — oversharing, permission sprawl, duplicate and stale records — to create a new class of AI-mediated data risk. Concentric AI’s 2025 Data Risk Report aggregates telemetry across customer environments and highlights how those legacy issues compound when an AI assistant is added to the mix.
At the same time, Microsoft maintains that Copilot “complies with our existing privacy, security, and compliance commitments,” pointing to GDPR support and an EU Data Boundary for Microsoft 365 Copilot. Microsoft’s documentation also describes how Copilot uses Microsoft Graph content (emails, chats, documents) and how data related to interactions with Copilot can be managed by administrators and deleted by end users. Those platforms-level protections are real, but they do not erase the operational gaps that the Concentric report surfaces.
Organisations cannot outsource responsibility to platform providers alone. Microsoft’s compliance and data residency commitments are important and real, but they do not absolve customers from implementing least-privilege access controls, DSPM/DLP strategies, and AI-aware monitoring. The defensive playbook is familiar — inventory, classification, least privilege, monitoring — but it must be applied with urgency and adapted to the new patterns of AI behaviour.
The Concentric report should be read as a wake-up call: Copilot already touches enormous volumes of sensitive content in real, production environments. That presents an opportunity — and an obligation — for security teams to harden controls before the next major incident turns an AI assistant from a productivity booster into a systemic breach vector.
Conclusion
Generative AI will continue to reshape enterprise workflows, and Copilot is at the centre of that shift. The headlines about millions of records being accessed are alarming but useful: they quantify the risk and show where security teams must act. The next phase — securing AI — requires better data hygiene, new policy guardrails, tighter defaults from vendors, and AI-aware tooling. Organisations that move quickly to close the governance gap will protect customers, reduce regulatory exposure, and still reap the productivity benefits of Copilot and other assistants; those that don’t face amplified risk, from targeted fraud to large-scale data loss.
Source: Club386 Survey shows Copilot has a treasure trove of sensitive business data | Club386
Background
Corporate use of generative AI assistants has surged: organisations are embedding Copilot and similar tools into email triage, document drafting, incident response, and even regulated workflows in healthcare, finance and government. That rapid adoption is colliding with long-standing enterprise problems — oversharing, permission sprawl, duplicate and stale records — to create a new class of AI-mediated data risk. Concentric AI’s 2025 Data Risk Report aggregates telemetry across customer environments and highlights how those legacy issues compound when an AI assistant is added to the mix. At the same time, Microsoft maintains that Copilot “complies with our existing privacy, security, and compliance commitments,” pointing to GDPR support and an EU Data Boundary for Microsoft 365 Copilot. Microsoft’s documentation also describes how Copilot uses Microsoft Graph content (emails, chats, documents) and how data related to interactions with Copilot can be managed by administrators and deleted by end users. Those platforms-level protections are real, but they do not erase the operational gaps that the Concentric report surfaces.
What Concentric’s findings actually say — and what they don’t
Key claims from the Data Risk Report
- Concentric reports that Copilot accessed almost three million confidential records per organisation during the first half of 2025. That figure is derived from Concentric’s analysis of production environments monitored by its DSPM (data security posture management) product.
- The report also found organisations averaged roughly 3,000 interactions with Copilot, during which sensitive business information could be read, modified, or surfaced in outputs.
- Concentric highlights pervasive oversharing and data sprawl: organisations reportedly hold millions of duplicates and large volumes of very old records, while large numbers of critical files are shared externally or to personal accounts. These structural data hygiene problems amplify the risk from AI tools.
How to interpret these numbers
- These figures come from Concentric’s customer telemetry and sampled environments. They are compelling indicators of pattern and scale but are not a neutral population survey of all companies worldwide. As with any vendor-sourced report, the sample may skew toward organisations already concerned about data risk — and to industries where Concentric has deployments (tech, healthcare, finance, government). Treat the numbers as high-confidence signals of systemic risk, not as a universal per-company constant.
- The headline “three million” metric is useful as a red-flag: it shows that, in real deployments, Copilot workflows are touching huge volumes of sensitive records. But whether that translates to actual exfiltration or regulatory breach events depends on the organisation’s governance, retention, and audit controls.
How Copilot handles data in practice — Microsoft’s position
Data flows and protections Microsoft publishes
Microsoft documents explain Copilot’s core architecture: it orchestrates large language models (LLMs) with content the user can access from Microsoft Graph and productivity apps. For Microsoft 365 Copilot, the company asserts compliance with GDPR and describes an EU Data Boundary that routes EU traffic within the EU when possible. Microsoft also exposes administrative controls — Microsoft Purview, content search, retention policies — and says users can delete their Copilot activity history via their account portal.Practical gaps that remain
- The platform-level guarantees (data residency, GDPR adherence) are necessary but not sufficient to limit risk. Even when data stays inside the Microsoft trust boundary, internal misuse, misconfiguration, or poorly governed Copilot prompts can surface confidential material in outputs or create new derivative sensitive content.
- Microsoft notes that Copilot-generated artifacts (like documents produced via Copilot Chat) may be stored in OneDrive or otherwise be persisted. Administrators can set retention and search policies, but these protections require deliberate configuration and ongoing enforcement.
- Crucially, prompts and AI outputs are records of interaction. That metadata — the exact prompt text and the AI’s responses — may become a repository of sensitive context (for example, a user’s prompt that contains PII, credentials, or contract clauses). Those records are a new class of forensic data that organisations must govern.
Why Copilot can act as a new attack surface
Oversharing meets agentic assistants
Generative assistants like Copilot are explicitly designed to fetch, summarise, and remix enterprise content. When a user with broad access privileges asks Copilot to summarise “recent contracts mentioning supplier X,” Copilot will draw on whatever content the user can access. If permissions are overbroad, the assistant will surface confidential details — and those outputs may then be saved, forwarded, or pasted into external channels. This amplifies accidental exposure. Concentric’s report ties high interaction counts and oversharing to elevated risk across healthcare, finance and government sectors.Attack vectors to consider
- Malicious insider: a user intentionally leverages Copilot to extract proprietary IP or regulated data and push it to external accounts or personal email.
- Compromised credentials: an attacker who compromises a user account can query Copilot and harvest sensitive content available to that account.
- Misconfigured APIs or third-party plug-ins: Copilot integrations that call out to other services (public web searches, external connectors, plug-ins) can route fragments of corporate data outside the tenant’s control if not carefully controlled.
- Data-poisoning or output leakage: AI outputs may inadvertently contain snippets of confidential data that can be scraped or exfiltrated if outputs are shared widely.
Cases that show the stakes
Recent large-scale incidents make the risk concrete. Retailers and manufacturers have been taken offline or materially disrupted by cyber incidents that affected operations and customer data.- Marks & Spencer confirmed a cybersecurity incident that forced operational changes and limited services while external investigators and regulators were notified. That kind of disruption shows how quickly customer-facing services can be impacted following a breach.
- Jaguar Land Rover experienced a production-halting cyberattack that required emergency credit and major recovery operations, underscoring how supply chains and manufacturing can be crippled by successful intrusions. When critical corporate data or operational controls are implicated, the financial and reputational impact can be catastrophic.
Technical and governance mitigations: what works right now
Many of the controls organisations already use for cloud and SaaS security are applicable to Copilot but need to be adapted for AI-specific behaviours.Immediate tactical steps (0–30 days)
- Audit and map Copilot usage across the tenant: identify top users, frequency, and what types of content Copilot is interacting with. Concentric recommends monitoring Copilot requests to surface abnormal patterns.
- Apply least privilege: tighten file and folder permissions so that Copilot’s effective access footprint is reduced to the bare minimum for the user to do their job.
- Enforce multi-factor authentication and conditional access for all accounts that can call Copilot, and apply session controls to restrict automated flows.
- Configure retention and Purview policies for Copilot interaction logs; ensure admins can search and export prompts and responses if required by an investigation.
Short-to-medium term (1–3 months)
- Deploy Data Loss Prevention (DLP) policies specifically tied to content used in AI workflows. Use content inspection rules that block or quarantine Copilot outputs that contain PII, PHI, or financial data.
- Integrate a DSPM or semantic classification layer that understands context — not just keywords — so that Copilot requests are matched against dynamic sensitivity labels before data is returned. Concentric and other vendors now offer AI-aware DSPM that can factor in context and permissions when Copilot queries data.
- Educate users with clear guidelines and enforce prompt hygiene: ban copying of credentials or explicit PII into prompts, require redaction of certain fields before submitting to Copilot, and create approved templates for common queries.
Long-term architectural changes (3–12 months)
- Adopt model governance and private LLM options where highly sensitive data must be used for automation. Keep training and inference for regulated workloads within tenant-controlled environments where possible.
- Implement continuous detection: alert on anomalous Copilot query patterns (large-scale exports, repeated complex queries across sensitive data sets) and link those alerts to automated containment workflows.
- Negotiate contractual and technical SLAs with platform providers that include audit logs, on-demand forensics, and defined data residency and processing commitments.
Practical controls administrators should enforce today
- Restrict Copilot scope by role: enable Copilot access selectively; do not roll out wide open.
- Data labeling and classification: establish mandatory labels for PHI/PII/IP that drive automated enforcement.
- Block public plug-ins by default: disallow or tightly control connectors that route queries to third-party public LLMs or services.
- Log everything: capture prompts, outputs, timestamp, and the content sources Copilot referenced; store logs in a tamper-evident archive for compliance audits. Microsoft exposes Purview and export APIs to support this, but they must be configured.
Legal and compliance considerations
- GDPR and other data protection regimes treat unauthorised processing and unauthorised disclosure seriously. Even when a vendor claims compliance or an EU Data Boundary, organisations are still the data controllers in many contexts and remain liable for failing to protect personal data processed by third-party services. Microsoft’s documentation legitimises its compliance posture, but it places responsibility on customers to configure and govern Copilot correctly.
- Data residency promises (EU Data Boundary) are useful but not absolute. Microsoft’s own docs note that during high utilization traffic may be routed outside the EU for model processing; organisations with strict legal obligations must scrutinise contractual terms and operational behaviour.
- Incident response contracts and cyber insurance policies should be reviewed to ensure they account for AI-specific leak scenarios (prompts containing PII, AI-driven automated exports, derivative content leaks).
Threat modelling: realistic worst-case scenarios
- Scenario A — credential compromise: an attacker obtains a privileged user’s credentials and instructs Copilot to compile reports that include personally identifiable information and financial records. Automated scripting of repeated queries yields bulk data exfiltration in hours rather than weeks.
- Scenario B — output proliferation: a well-meaning employee pastes a Copilot-generated report containing sensitive snippets into an external shared folder or sends it to a personal email address; that single action creates a persistent leakage vector.
- Scenario C — model-assisted social engineering: Copilot is used to assemble target-specific profiles (emails, addresses, roles) that enable high-quality phishing or fraud campaigns against employees or customers. The immediacy of AI-generated context increases attack success rates.
Vendor responsibilities and product-level fixes to demand
Enterprises should press platform providers for:- Finer-grained access controls: block inference over selected repositories (e.g., HR, R&D) while leaving Copilot active for general productivity content.
- Context-aware redaction: native features that automatically redact or mask PII when a Copilot prompt references regulated fields.
- Transparent routing and residency logs: continuous, observable logs that show where prompts were processed and which datacenter served the LLM request.
- Stronger defaults: “deny by default” for external sharing of Copilot outputs, and default opt-in for all AI features that capture user content (instead of opt-out). Microsoft has adjusted some defaults after Recall backlash, but more fine-grained defaults are needed at the enterprise level.
Why “Recall” controversy matters in this debate
Microsoft’s Recall feature — a proposed desktop timeline that captures periodic screenshots — was delayed and reworked after privacy researchers and vendors raised alarms about its potential to capture sensitive content and be abused. The Recall episode is instructive: even when data collection is local and encrypted, people and third-party software can create emergent exposures. Security-focused vendors blocked Recall in their browsers and Microsoft pushed the feature into extended preview cycles to add protections. That pattern demonstrates how product design choices for AI features can become national news and regulatory flashpoints, and why enterprises must demand stronger defaults and proof of secure-by-design architecture.A practical checklist for security teams (executive summary)
- Inventory: identify where Copilot and other AI assistants are enabled.
- Classify: tag sensitive repositories and prevent Copilot access to the highest-risk data by default.
- Harden: enforce MFA, conditional access, and device health checks for Copilot access.
- Monitor: log prompts and outputs; alert on anomalous patterns.
- Educate: roll out mandatory AI usage policies and prompt hygiene training.
- Remediate: run data hygiene projects to remove duplicates, orphaned accounts, and stale shares.
- Contract: update supplier contracts to include AI-specific processing terms and audits.
Strengths and limitations of the Concentric findings
- Strengths: Concentric’s report is powerful because it is empirical — based on production telemetry — and because it quantifies interaction volumes at scale. It brings needed attention to the intersection of historical data hygiene problems and modern AI assistants.
- Limitations and caution: the report’s sample is drawn from Concentric’s customers and therefore may not be statistically representative of every industry or geographic region. The headline figures should be read as indicative of systemic risk rather than deterministic proof that every organisation exposed three million records. The report’s methodological details (sample sizes, sector breakdowns, normalisation methods) should be reviewed by security teams before wholesale extrapolation. Concentric’s own materials make clear the data comes from their deployments and DSPM scans.
Final analysis: balancing innovation and risk
Generative assistants like Copilot will remain highly valuable productivity tools. The critical takeaway from Concentric’s report — and from the Recall controversy and recent high-profile breaches — is that AI changes the speed and scale of consequences. Where a misconfigured share used to leak a document or two, an AI-powered workflow can synthesise and replicate sensitive content across many artifacts in minutes.Organisations cannot outsource responsibility to platform providers alone. Microsoft’s compliance and data residency commitments are important and real, but they do not absolve customers from implementing least-privilege access controls, DSPM/DLP strategies, and AI-aware monitoring. The defensive playbook is familiar — inventory, classification, least privilege, monitoring — but it must be applied with urgency and adapted to the new patterns of AI behaviour.
The Concentric report should be read as a wake-up call: Copilot already touches enormous volumes of sensitive content in real, production environments. That presents an opportunity — and an obligation — for security teams to harden controls before the next major incident turns an AI assistant from a productivity booster into a systemic breach vector.
Conclusion
Generative AI will continue to reshape enterprise workflows, and Copilot is at the centre of that shift. The headlines about millions of records being accessed are alarming but useful: they quantify the risk and show where security teams must act. The next phase — securing AI — requires better data hygiene, new policy guardrails, tighter defaults from vendors, and AI-aware tooling. Organisations that move quickly to close the governance gap will protect customers, reduce regulatory exposure, and still reap the productivity benefits of Copilot and other assistants; those that don’t face amplified risk, from targeted fraud to large-scale data loss.
Source: Club386 Survey shows Copilot has a treasure trove of sensitive business data | Club386
