Microsoft has recently expanded the capabilities of its AI assistant, Copilot Vision, to provide comprehensive screen analysis across the entire desktop environment. This enhancement allows users to share their entire desktop or specific application windows with Copilot, enabling the AI to offer real-time insights, guidance, and assistance tailored to the content displayed on the screen.
Previously, Copilot Vision was limited to analyzing content within two applications simultaneously. With the latest update, it can now interpret and interact with the entire desktop or any selected browser or application window. This functionality is particularly beneficial for tasks such as refining creative projects, improving resumes, or navigating new software applications. Users can activate this feature by clicking the glasses icon within the Copilot app and selecting the desired desktop or window to share. To discontinue sharing, users can press 'Stop' or 'X' in the composer interface.
The development of Copilot Vision has been progressive. Initially, it was introduced as a feature capable of analyzing content within the Microsoft Edge browser. Over time, its functionality expanded to include the ability to interpret images captured through mobile phone cameras, providing contextual information and assistance based on visual inputs.
This advancement underscores Microsoft's commitment to integrating AI more deeply into user workflows, offering a more interactive and supportive computing experience. By enabling Copilot Vision to access and analyze the entire desktop environment, users can receive more comprehensive assistance, enhancing productivity and user engagement.
It's important to note that this feature is currently available to Windows Insiders, with plans for broader rollout in the near future. Users interested in exploring this functionality can participate in the Windows Insider Program to gain early access to Copilot Vision's expanded capabilities.
Source: The Verge Now Microsoft’s Copilot Vision AI can scan everything on your screen