Microsoft has recently expanded the capabilities of its Copilot Vision AI, enabling it to scan the entire desktop environment on Windows systems. This enhancement allows users to select any application, browser, or the full desktop for AI analysis and guidance, significantly broadening the tool's utility beyond its previous limitation to two or three applications.
The updated Copilot Vision functions similarly to a screen-sharing application. Users can activate it by clicking the glasses icon within the Copilot app and choosing the specific area of the screen they wish to share. Once enabled, the AI can analyze the visible content, offering real-time insights, contextual tips, and interactive guidance across various tasks, including creative projects, resume enhancements, and gameplay strategies.
This feature is distinct from Microsoft's Recall, which automatically records screen activity in the background. In contrast, Copilot Vision requires manual activation, providing users with greater control over their privacy and the AI's access to on-screen content.
Initially tested with Microsoft Edge in 2023, Copilot Vision has now been integrated more broadly into Windows 11. It extends its functionality to mobile devices, allowing users to interact with real-world objects or text captured by their smartphone cameras. This advancement aligns with Microsoft's vision of embedding AI deeply into user experiences, offering assistance that is both intuitive and contextually relevant. (windowscentral.com)
The rollout of this feature is currently available to Windows Insiders, with plans for a wider release in the near future. Users interested in exploring Copilot Vision's capabilities can access it through the Copilot app on their Windows devices.
Source: News9live Microsoft Copilot Vision AI now scans entire desktop on Windows
