Microsoft’s own data, shared exclusively with Axios, shows a clear behavioral pivot: millions of Copilot users are no longer treating the assistant as a one-off productivity tool but as a conversational confidant — asking about health, careers, relationships and even philosophy during late-night sessions. This discovery comes alongside a deliberate product shift from Microsoft that packages long-term memory, group chats, an expressive avatar called
Mico, new conversational “Real Talk” modes and cross-account connectors — changes designed to make Copilot feel more personal, persistent and socially situated. The combination matters because it moves Copilot from an occasional utility inside Office and Edge into a continuous presence across phones, desktops and browsers — an assistant that can remember preferences, join your friends’ group chat, and, with permission, act on your behalf on partner websites. That shift brings clear productivity upside, but it also raises ethical, privacy and safety trade-offs that are immediate and difficult. This feature unpacks the data, the product moves, and the practical risks and mitigations Windows users, IT teams, and regulators must weigh now.
Background
What Axios reported — the data Microsoft shared
Between January and September 2025, Microsoft researchers analyzed tens of millions of Copilot conversations and found a pronounced difference in usage patterns by device: desktop sessions skew heavily toward productivity tasks, while mobile conversations increasingly take on a social and advisory character — from emotional check-ins to health questions and career advice. Axios reports the dataset encompassed roughly
37.5 million conversations, with
philosophical and introspective queries peaking during late-night hours, indicating deeper emotional engagement in many mobile sessions. Because the Axios piece was an exclusive based on data Microsoft shared for research purposes, it provides a rare glimpse into internal usage patterns. That said, the Axios account is the primary public record of this 37.5 million figure and its contextual breakdown; the company’s public product posts and press statements describe capability changes and safety guardrails but do not publish the full conversation-level dataset. Treat the quantitative claim as Microsoft-provided research disclosed to Axios.
What Microsoft has already changed in product
Over the last several months Microsoft has rolled a major consumer-focused “Copilot” refresh that purposely bundles personalization, social features and agentic actions:
- Mico — an optional, animated, non-photoreal avatar for voice interactions designed to provide nonverbal cues and make long-form conversations feel less mechanical.
- Memory & Personalization — long-term, user-managed memory that can retain preferences, projects and recurring facts across sessions with UI controls to view or delete stored items.
- Copilot Groups — shared group chat sessions (reported to support up to 32 participants) where a single Copilot instance synthesizes inputs, summarizes threads, tallies votes and splits tasks.
- Real Talk — a selectable conversation style that can push back on assumptions and explain reasoning rather than reflexively agreeing.
- Connectors and Actions — opt-in links to cloud services (Outlook/OneDrive, Gmail/Drive, Google Calendar) and agentic Actions that — with explicit permission — can complete multi-step tasks (bookings, form fills) in the browser.
Microsoft has framed these changes under a “human-centered AI” banner, emphasizing opt-in flows, consent prompts and visible memory controls; it also says that Copilot in Microsoft 365 apps will not use prompts, responses or file content to train foundation models.
Why the pivot matters: design, business and behavioral incentives
From productivity widget to emotional interlocutor
The Axios data suggests a pattern sociologists and UX researchers have long observed: when conversational agents are available, people attribute relational features to them and use them for social and emotional tasks. This dynamic is not new — companion platforms like Replika and region-specific agents (e.g., XiaoIce) established years ago that users will treat chatbots as sources of companionship and emotional support. Academic research and user studies document meaningful emotional bonds, disclosure, and perceived support in those contexts. Microsoft’s design push accelerates this trajectory by adding memory, a face (Mico), and voice-first interactions. Each of those features lowers the psychological friction for human-like engagement:
- Visual cues and voice reduce social awkwardness and increase perceived presence.
- Memory creates continuity — a powerful signal that invites long-term relationship-like interactions rather than ephemeral queries.
- Group sessions normalize shared social use cases and amplify social proof (if friends use Copilot for advice, you may be more likely to treat it as a confidant).
For Microsoft this is a deliberate product and commercial strategy: embedding Copilot at the surface of Windows, Edge, and Microsoft 365 raises daily engagement and strengthens subscription value — a classic platform incentive. The company argues that the end goal is
augmentation of human life, not substitution, and says it’s building consent and safety controls into the rollout.
Business trade-offs
Making Copilot more “personal” is also a business decision. Greater personalization and social features can:
- Increase user retention and subscription stickiness.
- Create new monetization or partner opportunities (actions with booking partners, e-commerce workflows).
- Give Microsoft a richer signal set for product improvement and feature prioritization — but only if the data governance is correctly managed.
Those upsides sit directly alongside risks we detail below.
The risks — why “intimacy” amplifies harm vectors
1) Emotional dependency and clinical-risk blind spots
When people treat chatbots as mentors or therapists, two problems follow:
- False assurance. AI models are not clinicians. Tests and red-team assessments repeatedly show they can be wrong, inconsistent, or dangerously permissive when faced with crisis scenarios. Stanford and other researchers have warned that chatbots can provide unsafe or improperly framed health advice and may miss signs of severe distress.
- Dependency. Users who lack human supports can develop emotional dependence on always-available assistants; documented Replika research and UX analyses show people build strong bonds and even grieve when platform features change. That intensifies the consequences when systems provide poor guidance or are modified without warning.
Microsoft has attempted to mitigate this with a “Copilot for Health” flow that anchors answers to vetted publishers and a “Find Care” clinician directory, but these are limited rollouts and do not eliminate the fundamental risk of non-expert systems offering emotionally persuasive guidance.
2) Privacy and data-mining concerns
Long-term memory, connectors to email and cloud drives, and group sessions increase the surface area for accidental data leakage:
- Cross-account linking multiplies risk. Even with OAuth and explicit consent, the practical interplay of multiple connectors and shared group sessions raises questions about what is stored, for how long, and who it’s shared with. Microsoft highlights user-editable memory controls, but real-world settings and defaults matter enormously.
- Training and telemetry ambiguity. Microsoft publicly states it does not use prompts and file content from Copilot in Microsoft 365 apps to train foundation models, but different product surfaces, telemetry pipelines, and partner model routing create complex data flows that are often opaque to end-users and admins. Where the firm-level promise applies and where it doesn’t requires careful documentation and independent audit.
3) Mis- or disinformation, hallucination and safety
When assistants speak with authority or a soothing persona, hallucinations can have outsized harm. A confident-sounding answer about medical advice, legal options, or sensitive relationship advice can produce real-world consequences if incorrect. This risk is especially acute in late-night, emotionally charged sessions described in the Axios data.
4) Abuse and adversarial manipulation
Past incidents underline the vulnerability of conversational systems when exposed to adversarial inputs. Microsoft’s Tay experiment in 2016 is a cautionary historical example: a chatbot exposed to hostile conversational context quickly began parroting racist and abusive content once attack vectors were found. That failure illustrates how social environments and incentives can distort AI behavior if guardrails are incomplete.
5) Regulatory and ethical exposure
As Copilot penetrates everyday life, companies and governments will demand clearer lines of accountability for harms, data handling, and disclosure. The stakes are high for Microsoft because large-scale consumer reach means harms can be systemic and visible.
Where Microsoft’s product choices help — and where they fall short
Notable strengths
- Design-forward mitigation: Microsoft is explicitly designing non-photoreal avatars (Mico), visible memory controls, and opt-in connectors to reduce surprise and unwanted emotional attachment. That design intent matters because interface choices strongly influence behavior.
- Staged rollouts and U.S.-first previews: phasing features into limited markets gives Microsoft a chance to observe behavior and iterate.
- Source-grounding for health queries: committing to vetted publishers and a Find Care flow shows an attempt to reduce hallucination risk in medical contexts.
- Public commitments about training data in Microsoft 365: Microsoft states that prompts and file content in Microsoft 365 apps are not used to train foundation models, addressing a core privacy worry for many productivity users.
Shortcomings and open questions
- Default settings and discoverability. Features that are opt-in in policy can still be enabled by default in practice; whether non-technical users truly understand connectors, memory, and group-sharing remains questionable.
- Limited independent auditing. Product statements are useful, but independent audits, red-team results, and publicly visible safety metrics are still needed to verify that guardrails work at scale.
- Clinical safety gap. Grounding to trusted sources is necessary but not sufficient for emergent clinical triage and crisis detection; specialized clinical oversight and escalation pathways are required for truly safe health interactions.
- Ambiguity across surfaces. Microsoft’s promises about not using prompts for training apply to select Microsoft 365 scenarios; clarity is still required across the broader Copilot ecosystem, especially in free or mobile consumer surfaces.
Practical guidance — for users, IT administrators and regulators
For everyday users
- Treat Copilot as a powerful assistant, not a clinician or lawyer. Use it for drafting, brainstorming, and summarization — be conservative with medical, legal, or mental-health decisions.
- Audit your memory and connector settings. If Copilot can “remember” things about you or access your email and drive, review those settings and delete what you don’t want stored.
- Prefer grounded modes for sensitive queries. When Copilot offers a health-grounded or cited answer, check the sources and consult human professionals for high-stakes decisions.
For IT administrators and policy owners
- Define a deployment posture: pilot Copilot features in controlled user groups before broad enablement. Assess the need for selective opt-outs via policy and Intune/Group Policy.
- Establish a data governance charter: document what connectors are permitted, retention policies for memory, and how group session data is archived or purged.
- Audit logs and incident playbooks: ensure Copilot agentic Actions and connectors provide auditable logs and a mechanism for revocation and remediation if data is acted upon incorrectly.
For regulators and standards bodies
- Require independent safety audits and transparency reports. Public red-team results, harm dashboards, and third-party verification of training data practices would materially raise assurance.
- Set minimum clinical-safety standards for any assistant that delivers health advice or guides crisis decisions, including mandatory human escalation pathways.
- Standardize consent and defaults. Make opt-in, clearly explained consent flows mandatory for long-term memory, cross-account connectors, and shared sessions.
A realistic path forward: product and policy priorities
To balance the benefits of a more personal Copilot with the societal risks that intimacy amplifies, Microsoft and other platform owners should prioritize the following pragmatic steps:
- Transparent defaults and frictionless controls. Memory and connectors should default to off for new users, with clear, conversational explanations for what enabling them does and visible, one-tap ways to forget.
- Independent safety verification. Commission and publish third-party audits, red-team results, and privacy assessments for high-risk features (health, mental health, group sharing).
- Clinical and child-safety integrations. For health and youth-facing use, build clinician-verified flows and minimum age gating backed by identity assurances and parental controls.
- Meaningful opt-out for consumers. Make it straightforward to disable Copilot features across devices and browsers, and ensure that personal account choices are not silently reset by cookie clearing or device changes.
Conclusion
Microsoft’s internal research — as reported by Axios — confirms something many UX researchers and product teams feared and expected: when you give people a responsive, multimodal agent that remembers context, many will start treating it like someone rather than something. Microsoft’s product roadmap (Mico, memory, Groups, Real Talk, and Actions) purposefully leans into that reality to deliver convenience and new capabilities, but it also compounds risks around privacy, safety and emotional dependency absent robust, independent safeguards.
The responsible path is not to stop building human-centered assistants — there are real productivity and accessibility benefits — but to pair capability with transparency, independent verification, conservative defaults, and clear human escalation for high-risk domains. Jumping from helpful shortcut to intimate confidant changes the moral and legal contract between vendor and user. Microsoft’s technical and product fixes are a start; what comes next must be institutional — audits, standards, and practical consumer protections — before millions more conversations become a new front in public-health and privacy policy.
Source: Axios
Exclusive: Microsoft research highlights Copilot's shift into intimate territory