Microsoft has initiated the rollout of Copilot Vision on Windows to Windows Insiders, marking a significant advancement in integrating AI capabilities directly into the operating system. This feature allows users to share their entire desktop with the AI assistant, enabling real-time analysis, insights, and interactive guidance.
Unlike the Recall feature, which continuously monitors screen activity, Copilot Vision requires manual activation. Users can invoke it by clicking the glasses icon within the Copilot app, initiating a floating toolbar with voice and vision controls. To terminate the session, users can click the 'Stop' or 'X' button. Additionally, voice interaction is supported, allowing users to communicate with the AI using spoken commands.
Copilot Vision is designed to function across various applications, though it currently supports up to two apps simultaneously. It assists with tasks such as enhancing creative projects, refining resumes, or providing guidance in new software environments. The feature is available to Windows Insiders running Copilot app version 1.25071.125 and higher.
Privacy is a key consideration in Copilot Vision's design. Microsoft states that only the AI's responses are logged to monitor for unsafe interactions, while user inputs, images, and page content are neither logged nor stored. All data is deleted upon ending the voice session.
In parallel, Microsoft is testing a new 'Click To Do' action called 'Describe Image,' which enables the AI to describe the contents of a photo upon user request.
These developments reflect Microsoft's ongoing commitment to integrating AI into the Windows ecosystem, aiming to enhance user productivity and interaction through advanced, privacy-conscious features.
Source: gHacks Technology News Microsoft begins testing Copilot Vision on Windows - gHacks Tech News