Copilot Actions: Real Web Automation in the Cloud—Promises vs. Limits

I asked Microsoft’s Copilot to book a dinner reservation for me so it could prove the central promise of modern AI agents: act on my behalf, not just answer questions—and it mostly worked, but with enough caveats to make clear this technology is still in its experimental phase.

Background​

Microsoft’s Copilot Actions is the consumer-facing implementation of what the industry calls an AI agent: a system that navigates the web, fills forms, and completes multi-step tasks for a user. Microsoft publicly described Actions as a capability that can “book event tickets, grab dinner reservations or send a thoughtful gift,” and it lists launch partners including OpenTable, Booking.com, Expedia and others. (blogs.microsoft.com) (theverge.com)
The core technical model that powers Actions is important to understand. When you ask Copilot to “do” something, Microsoft provisions a cloud-based virtual machine (VM) running a browser, and Copilot drives that browser programmatically. The system uses screenshots and DOM analysis of that remote browser to decide where to click and what to type. Sessions are sandboxed and disposable, and the UI shows a split view with the remote browser pane and a Copilot chat sidebar. Early hands-on reporting confirms this approach. (windowsforum.com)

How Copilot Actions works — a technical sketch​

The cloud browser model​

• Copilot creates an isolated browser session in the cloud (a VM).
• The VM navigates the target website and renders pages as a normal browser would.
• Copilot “sees” the page by capturing screenshots and analyzing them to locate inputs and buttons.
• The assistant executes clicks and form entries in that remote browser on your behalf.

This architecture has two immediate consequences: it reduces local CPU/GPU load (the heavy work happens in Microsoft’s cloud), and it isolates the agent from your local device—so Copilot cannot directly read local files or local cookies unless specifically permitted. Multiple independent reports corroborate this design. (windowsforum.com)

What Actions can and cannot do today​

• Can: find listings, open site pages, pre-fill reservation or checkout forms, navigate multi-step flows.
• Cannot (reliably): bypass CAPTCHAs, complete flows that require sign-in and multi-factor authentication without human input, or fully automate payments where sites require fresh credential entry or verification steps.

Those limits are not bugs so much as deliberate guard rails: site-level defenses and authentication flows are designed to prevent automated actors, and Copilot respects them. In practice that means many tasks will proceed automatically up to the point a human verification is required.

---

Hands-on: what the dinner-reservation test shows​

In a recent review, Copilot Actions was asked to “Make a dinner reservation for two for 8 p.m. at a good Japanese restaurant nearby using OpenTable.” Copilot launched a cloud browser, found a suitable OpenTable page (using Bing when no site was specified), navigated the reservation widget, completed the selection process and prepared the reservation—then stopped and asked the user to enter a phone number and a verification code that arrived by SMS. That manual step was necessary because the website required phone verification and Copilot cannot receive SMS on behalf of a user.
A second test—buying a book from Barnes & Noble—demonstrated a similar pattern. Copilot prompted for genre clarification, navigated search results, selected a title, and prepared the purchase flow. When the process required personal payment details, the automation paused for user input.
These experiments reveal two practical truths:

• Copilot Actions is effective for discovery and form-filling on heterogeneous sites.
• Real-world friction points (CAPTCHAs, SMS MFA, payment entry) still require human intervention.

---

Usability: promising, but often slower than doing it yourself​

A surprising but recurring observation across tests is that using Copilot Actions can be slower than manually completing the same task in your local browser. Several factors contribute:

• VM spin-up and remote page rendering add latency.
• Screenshot capture and analysis introduce processing delays.
• The system intentionally paces interactions to avoid mistakes.

That latency is acceptable if the agent removes the mental load of repetitive multi-step tasks or runs asynchronously (e.g., assembling options and returning results later). But when you’re already comfortable with a website, the overhead currently outweighs the convenience. For now, Actions excels at reducing tedium for complex searches and initial form completion rather than maximizing speed.

---

Privacy and security: sandboxed but not risk-free​

What the cloud model protects​

Because Actions runs in a cloud VM, Copilot cannot directly access local files, local cookies, or device state unless explicit integrations are built. This isolation prevents the assistant from harvesting local data by default, which is a meaningful privacy advantage over locally-running agents.

What the cloud model exposes​

• Copilot takes screenshots of the pages it visits in the remote browser to analyze the page layout and determine where to interact. Those screenshots are processed in Microsoft’s cloud to enable the agent’s actions. That behavior raises reasonable questions about screenshot retention, telemetry, and downstream access. Early documentation and Microsoft’s public comments emphasize sandboxing and ephemeral sessions, but finer-grained telemetry policies and retention windows remain points to clarify. (blogs.microsoft.com)

Authentication and credentials​

Actions will currently require a user to step in for MFA (SMS codes, authenticator prompts) and credential entry on most sites—by design. If Microsoft expands Actions to store credentials or integrate with a secure vault (so the agent can authenticate for you), that will materially change the privacy calculus and requires strong security guarantees, transparent retention policies, and user controls. Until such integrations appear, users retain a degree of control simply because they must complete verification steps.

Practical privacy guidance​

• Use Actions only on reputable sites you trust.
• Avoid entering highly sensitive information into third-party sites via the tool until Microsoft publishes clear retention and telemetry guarantees.
• For now, treat Copilot Actions as an assistant for neutral, non-sensitive tasks.

---

Regulatory and regional availability​

Copilot Actions is not universally available. Microsoft has explicitly limited Actions in some regions, including the EU, because of regulatory and privacy compliance considerations. That regional roll-out strategy means users in affected jurisdictions will not see Actions until Microsoft has resolved local legal requirements. Always confirm availability in your region before planning to rely on the feature for critical tasks. (learn.microsoft.com)

---

The business model and practical limits​

Microsoft bundles Copilot into multiple tiers: a free consumer offering, a paid Copilot Pro subscription with expanded capabilities, and enterprise Microsoft 365 Copilot products aimed at businesses with governance tools. The company has not publicly documented hard session limits for free vs paid accounts, and independent testing shows free accounts may be rate-limited—reviewers were cut off after a small number of sessions in some trials. That specific quota is not consistently disclosed and should be considered unverified until Microsoft publishes official limits.

---

Strengths: why Actions matters now​

• Real web automation on heterogeneous sites. Copilot Actions demonstrates that an agent can navigate arbitrary web layouts using screenshots and programmatic interaction—an important technical milestone.
• Low local resource usage. Because sessions run in Microsoft’s cloud, local CPU/GPU impact is minimal, making Actions usable on low-powered machines and mobile devices.
• Unified conversational control. Users can combine search, clarification, and action in a single chat-driven workflow—a precursor to how people may interact with the web in the future. (blogs.microsoft.com)

---

Risks and limitations: why you shouldn’t hand over the keys yet​

• Site-level defenses block autonomy. CAPTCHAs, SMS MFA, and sign-ins are deliberately designed to stop automation. Copilot must stop for these, limiting hands-off completion.
• Latency and reliability. Current speed and occasional missteps mean Copilot can feel more like a lab demo than a production-ready assistant.
• Privacy ambiguity around screenshots and telemetry. Ephemeral sessions are a good start, but retention policies, telemetry collection, and whether screenshots are used in model training need clearer disclosure. Flag any claims that lack explicit Microsoft documentation as unverified.
• Regulatory exposure. In regions with stricter privacy rules, Actions may be disabled or modified to comply, limiting universality.

---

Practical checklist: when to use Copilot Actions today​

• Use it for low-risk, repetitive tasks (e.g., initial search and pre-fill of forms).
• Avoid using Actions for financial transactions that require saving credentials until enterprise-grade vaulting and governance are confirmed.
• Expect to intervene for OTPs, CAPTCHAs, and any sign-in flows.
• Confirm regional availability (notably, Actions may be blocked or restricted in some jurisdictions).

---

Recommendations for Microsoft (what needs to happen)​

• Publish an explicit, easy-to-read data retention and telemetry policy for Actions screenshots and session logs.
• Offer an optional, secure credential vault with transparent auditing and customer-controllable retention for users who want full automation.
• Improve latency (faster VM warm-up and smarter incremental rendering) and add a background/asynchronous mode so Actions can run and notify you when tasks complete.
• Work with major sites to create an authenticated agent standard—an industry API that allows trusted agents to perform verifiable, auditable actions without defeating security controls.

These steps would move Copilot Actions from a promising demo to a dependable assistant capable of safely handling account-level tasks for users who opt in.

---

Cross-check: what independent coverage confirms​

• Microsoft’s official blog introduced Actions and framed it as a major Copilot expansion, listing launch partners and describing agentic capabilities. (blogs.microsoft.com)
• Independent technology press coverage confirms the cloud VM approach and the early limitations around MFA, CAPTCHAs, and regional availability. That reporting highlights both the promise and the experimental status of Actions. (theverge.com) (windowsforum.com)

Where reporting diverges—such as exact session limits for free accounts—those specifics remain unverified and should be treated cautiously until Microsoft publishes concrete numbers.

---

A realistic timeline and outlook​

This space moves quickly. The agent model—demonstrated by Copilot Actions, Google’s Project Mariner, and other vendor efforts—is advancing, but several hard problems remain: secure credential handling, reliable handling of anti-bot measures, privacy clarity, and regulatory compliance.
Expect incremental improvements over the next 6–18 months:

• Better vaulting and enterprise governance for credentials.
• Agreements with large platforms for safe agent access patterns.
• Performance optimizations that reduce the time penalty of remote sessions.

Full, trustworthy, and unattended agents that can complete purchases, authenticate across services, and act proactively on your calendar will require not only engineering work but also policy improvements and user trust-building—so mainstream users should prepare for a gradual transition rather than a sudden hand-off.

---

Conclusion​

Copilot Actions is a credible, functioning step toward agentic browsing: it can find listings, traverse site flows, and pre-fill forms across diverse websites by running a sandboxed cloud browser on your behalf. For many users, that capability will translate into genuine productivity gains—but the feature is not yet ready to “run your life.” Practical barriers—site defenses, MFA, privacy questions around screenshots, and speed—mean users will still be required to intervene for critical steps.
Treat Actions today as a smart assistant for research and form-filling, not as a replacement for your hands, eyes, and security judgments. As Microsoft and other vendors iterate, this agentic approach will almost certainly become a common part of how people interact with the web—but responsible adoption depends on stronger standards for security, clearer privacy guarantees, and careful regulatory alignment. (blogs.microsoft.com)

---

Recommended short-term user actions:

• Try Actions for low-risk tasks to build familiarity.
• Keep MFA and personal verifications enabled; do not store all credentials with an agent until secure vaulting is explicit.
• Monitor Microsoft’s documentation and announcements for changes in availability, pricing, and retention policy—especially if you live in regions with stricter privacy rules.

Copilot Actions is an early but important milestone: not quite a life manager yet, but a clear preview of a future in which AI does far more of the web’s repetitive work—if we can solve the security, privacy, and trust problems along the way.

---

Source: PCMag Australia I Let Microsoft’s Copilot Book My Dinner Reservation. It Works, But It's Not Ready to Run My Life Yet