Here’s a summary of what “Copilot Vision” is and what it means for Windows users, based on the NewsBytes article: What is Copilot Vision?
Copilot Vision is a new feature for Microsoft Copilot, the AI assistant in Windows.
It allows users to share their screen or app windows with Copilot.
Once shared, Copilot can guide users step-by-step through features in complex programs (such as Adobe Photoshop), analyze photos, webpages, and more—in real time.
How does it work?
You activate Copilot Vision within the Copilot app via a glasses icon.
You then select a browser or app to share. This is similar to sharing your screen in Microsoft Teams, but is opt-in (it only analyzes what you choose to share, unlike Recall, which does periodic full-screen snapshots).
Copilot can then highlight portions of your screen to provide instructions, answer questions, or guide you through tasks.
Key features:
User Control: Copilot only sees what you actively share. No other content on your screen is analyzed without your explicit action.
Highlights Tool: If you’re stuck (e.g., in a game or app), you can ask Copilot how to do something and it will visually guide you.
Multi-app Analysis: You can share two apps/files simultaneously; Copilot Vision can connect information between them (e.g., matching events from a webpage to open times in your calendar).
No Extra Cost (for now): It’s available for free to all Windows 10 and 11 users in the US.
Practical Examples:
Ask for tips getting past a game level.
Receive visual instructions in design software.
Drag and compare information between two apps (e.g., schedule events).
Privacy and Security:
Only the content you select is shared.
No background, continuous monitoring.
Availability:
The rollout is currently for US users of Windows 10/11.