Microsoft is developing a new feature for Windows 11, known as "Click to Do," designed to enhance user interaction with on-screen content through on-device AI capabilities. This feature is currently being tested on Copilot+ PCs within the Windows Insider Program.
"Click to Do" allows users to perform contextual actions on text and images displayed on their screens. For instance, by selecting an image, users can access options such as copying, saving, sharing, or opening the image with applications like Photos, Snipping Tool, or Paint. Additionally, AI-powered actions like visual search with Bing, background blurring, object erasing, and background removal are available. When interacting with text, users can copy, open with a text editor, search the web, ask Copilot, summarize, create a bulleted list, or rewrite the text. These functionalities are accessible through shortcuts like "Windows key + mouse click," "Windows key + Q," or via the Snipping Tool menu.
A notable aspect of "Click to Do" is its reliance on on-device processing, ensuring that data analysis occurs locally without the need for an internet connection. This approach prioritizes user privacy while delivering rapid responses. The feature leverages the Phi Silica model, a small language model optimized for on-device performance using the Neural Processing Unit (NPU) present in Copilot+ PCs.
Microsoft's "Click to Do" bears similarities to Google's "Circle to Search," a feature available on select Android devices that enables users to circle any on-screen object—be it text, image, or video—and instantly search for it online. However, while "Circle to Search" relies on cloud-based image recognition, "Click to Do" processes data locally, providing descriptions and actions without necessitating an internet connection.
Currently, "Click to Do" is in the preview phase and is being rolled out to Windows Insiders using Snapdragon-based Copilot+ PCs, with support for Intel and AMD-powered machines expected to follow. Users interested in testing this feature can join the Windows Insider Program and enable the setting to receive the latest features as soon as they're available.
This development underscores Microsoft's commitment to integrating AI-driven functionalities into Windows 11, aiming to enhance productivity and user experience through intuitive, context-aware tools.
Source: Business Standard https://www.business-standard.com/technology/tech-news/microsoft-has-google-s-circle-to-search-alternative-in-works-for-windows-11-125071500839_1.html
Last edited:
