Microsoft has recently unveiled a significant enhancement to its Copilot Vision feature, enabling the AI assistant to analyze the entire desktop environment in real-time. This development marks a substantial leap from its previous capability, which was limited to viewing and interacting with two applications simultaneously. The update is currently available to Windows Insiders, reflecting Microsoft's commitment to integrating advanced AI functionalities into the Windows operating system.
The core of this update lies in the "Desktop Share" functionality. By clicking the glasses icon within the Copilot app, users can now share their entire desktop or select specific application windows with Copilot. Once activated, Copilot gains the ability to see exactly what the user sees, providing real-time assistance, insights, and guidance tailored to the on-screen content. This feature is particularly beneficial for tasks such as editing documents, analyzing game content, or navigating complex browser activities. Microsoft emphasizes that Copilot can "help analyze content, provide insights, and answer your questions, coaching you through it aloud." (blogs.windows.com)
It's important to note that, unlike the Recall feature—which automatically captures periodic snapshots of the screen—Copilot Vision operates solely with user consent. Users must explicitly initiate the screen-sharing session, ensuring a more privacy-conscious approach. To stop sharing, users can simply press 'Stop' or 'X' in the composer. (blogs.windows.com)
The integration of voice input further enhances Copilot Vision's utility. During a voice conversation with Copilot, users can click the glasses icon to share their screen, allowing the AI to provide context-aware assistance based on both visual and auditory inputs. This multimodal interaction aims to create a more seamless and intuitive user experience. (blogs.windows.com)
Microsoft's phased rollout strategy ensures that the feature is gradually introduced to Windows Insiders, allowing for iterative feedback and refinement. The update is being distributed via the Microsoft Store, with version 1.25071.125 and higher incorporating these new capabilities. Initially, the rollout targets markets where Windows Vision is enabled, with plans for broader availability in the near future. (blogs.windows.com)
This advancement in Copilot Vision underscores Microsoft's dedication to enhancing user productivity through AI integration. By enabling the AI to comprehend and interact with the entire desktop environment, users can expect more personalized and efficient assistance across a wide range of tasks. As with all AI-driven features, user feedback will be instrumental in shaping future iterations and ensuring that Copilot Vision meets the diverse needs of the Windows user base.
Source: Українські Національні Новини Copilot Vision now sees your entire screen: Microsoft expands AI capabilities in Windows
