Published on September 18, 2024, by Mauro Huculak at Windows Central
Windows 11's latest feature update, the 2024 Update (also known as version 24H2), brings a significant upgrade in artificial intelligence capabilities. The rollout introduces a host of new features, including Windows Copilot Runtime, Recall AI, Live Captions, Auto Super Resolution, and several others designed to enhance productivity and accessibility.
## What AI Changes are Coming to Windows 11 2024 Update?
The 24H2 Update is more than just cosmetic changes; it serves as a launchpad for nearly a dozen AI-powered features, most of which can operate without an internet connection. Central to these innovations is the Windows Copilot Runtime, a pivotal integration into the operating system that allows various AI features to function locally.
### Windows Copilot Runtime
Think of the Windows Copilot Runtime as the beating heart of AI functionality in Windows 11. It integrates over 40 AI models capable of performing tasks right from your device. Some of these models include:
- Phi Silica: A Small Language Model (SLM) that performs tasks similar to larger language models.
- Screen Region Detector, Optical Character Recognizer, and Image Encoder are just a few of the other models that enhance user interactivity and functionality.
With the introduction of the Windows Copilot Library, developers can now easily embed these AI capabilities into their applications. It passively runs on your computer, ready for use when you need it.
### Windows Recall
Possibly the most talked-about feature is Windows Recall. This controversial tool automatically takes snapshots of your activity and uses AI to organize and analyze them. Data collected is stored locally in a new system called the Windows Semantic Index, designed to make information retrieval effortless.
To search through your captured data, you can use natural language queries or navigate using a timeline slider. Setting limits on what gets recorded is crucial for privacy. The Recall feature requires storage space—typically defaulting to 25GB for those with a 256GB hard drive. Users can also set filters to prevent specific websites or applications from being tracked.
Despite its utility, Recall raises significant privacy concerns. Although Microsoft claims stringent security measures—including device encryption and the Microsoft Pluton security processor—the feature's ability to capture intimate details of your work has many users feeling uneasy. Critically, initial unencrypted data meant that sensitive information was vulnerable; Microsoft now addresses this with changes to encrypt stored data.
### Windows Studio Effects
Studio Effects integrates advanced webcam features that leverage the Neural Processing Unit (NPU) to add professional-grade effects to video streams and audio inputs. Users can access various enhancements, such as:
- Background Blur
- Eye Contact: Adjusts the viewer's perspective to appear as if you are looking directly at the camera.
- Voice Focus: Removes background noise for clearer audio.
These effects can be toggled easily from the Quick Settings menu or through the dedicated camera settings in Windows.
### Live Captions and Voice Clarity
The Live Captions feature is now AI-powered, enabling automatic transcriptions and translations from any audio source. This invaluable tool enhances accessibility for users who may face language barriers.
Moreover, the Voice Clarity feature employs AI to eliminate background noise during direct communications. Initial availability was limited to Surface devices, but it now extends to all compatible hardware with the 2024 Update, showing Microsoft's intent to democratize these tools across their user base.
### Auto Super Resolution
Auto Super Resolution (Auto SR) is a noteworthy addition for gamers. This feature utilizes on-device AI to enhance game graphics, similar to technologies like NVIDIA's DLSS or AMD's FidelityFX. Auto SR will automatically upscale games for better resolution and performance without requiring developers to make changes.
This innovation promises a more immersive gaming experience while maintaining energy efficiency, channeling the workload to the NPU instead of the GPU. The feature can be enabled globally or per application through the Graphics settings page.
### Imaging Features: Restyle Image for Photos and Cocreator for Paint
With the update, the Photos app introduces the Restyle Image feature, employing AI to modify photographs using various artistic techniques. Text prompts can guide the AI in altering images to fit user-directed styles.
Meanwhile, the Paint app receives the Cocreator feature, which scans drawings and assists users in refining their art with generative techniques, making art creation more accessible.
### Copilot App and Suggestions
Significant changes are coming to the Copilot feature too. It now operates through an app interface rather than the previous right-side panel. This new design aligns more closely with chat interfaces that many users are familiar with, such as ChatGPT.
Despite the changes to Copilot's functionality—it's no longer able to alter system settings—it will still offer Copilot suggestions in applications. These suggestions can guide users in problem-solving, enhancing workflow efficiency without directly interfacing with system-level controls.
## Impact and Implications for Windows Users
The Windows 11 2024 Update represents a major shift towards integrating AI into everyday computing tasks. AI's potential to streamline workflows and elevate user experience cannot be overstated. However, the balance between convenience and privacy remains an ongoing concern, particularly with features like Recall that inherently collect user data.
This update aims not only to improve how users interact with their devices but also to make productivity tools accessible to a broader audience. As AI capabilities become increasingly commonplace, users must remain aware of how these technologies are deployed and maintained.
### Expert Commentary
This update places Microsoft in a competitive stance within the tech industry. By embedding AI functionality directly into the core operating system, they enhance usability while proactively addressing common user challenges. Recognizing the sensitivity surrounding data security is crucial. Microsoft has made strides towards data protection; however, hesitance from users regarding features like Recall indicates that trust-building must accompany technological advancement.
## Conclusion
The Windows 11 2024 Update is a testament to Microsoft's commitment to infusing AI into user interactions fundamentally. While the array of features offers exciting prospects for productivity and accessibility, users are urged to remain vigilant regarding privacy implications. Thus, the conversation around AI integration is likely to evolve, reflecting both the opportunities and challenges that come with this powerful technology.
In summary, the advanced AI capabilities ushered in by the 2024 Update stand to reshape the user experience on Windows, inviting both excitement and caution among its users.
Source: Windows Central What's new with AI on Windows 11 version 24H2 (2024 Update)
Windows 11's latest feature update, the 2024 Update (also known as version 24H2), brings a significant upgrade in artificial intelligence capabilities. The rollout introduces a host of new features, including Windows Copilot Runtime, Recall AI, Live Captions, Auto Super Resolution, and several others designed to enhance productivity and accessibility.
## What AI Changes are Coming to Windows 11 2024 Update?
The 24H2 Update is more than just cosmetic changes; it serves as a launchpad for nearly a dozen AI-powered features, most of which can operate without an internet connection. Central to these innovations is the Windows Copilot Runtime, a pivotal integration into the operating system that allows various AI features to function locally.
### Windows Copilot Runtime
Think of the Windows Copilot Runtime as the beating heart of AI functionality in Windows 11. It integrates over 40 AI models capable of performing tasks right from your device. Some of these models include:
- Phi Silica: A Small Language Model (SLM) that performs tasks similar to larger language models.
- Screen Region Detector, Optical Character Recognizer, and Image Encoder are just a few of the other models that enhance user interactivity and functionality.
With the introduction of the Windows Copilot Library, developers can now easily embed these AI capabilities into their applications. It passively runs on your computer, ready for use when you need it.
### Windows Recall
Possibly the most talked-about feature is Windows Recall. This controversial tool automatically takes snapshots of your activity and uses AI to organize and analyze them. Data collected is stored locally in a new system called the Windows Semantic Index, designed to make information retrieval effortless.
To search through your captured data, you can use natural language queries or navigate using a timeline slider. Setting limits on what gets recorded is crucial for privacy. The Recall feature requires storage space—typically defaulting to 25GB for those with a 256GB hard drive. Users can also set filters to prevent specific websites or applications from being tracked.
Despite its utility, Recall raises significant privacy concerns. Although Microsoft claims stringent security measures—including device encryption and the Microsoft Pluton security processor—the feature's ability to capture intimate details of your work has many users feeling uneasy. Critically, initial unencrypted data meant that sensitive information was vulnerable; Microsoft now addresses this with changes to encrypt stored data.
### Windows Studio Effects
Studio Effects integrates advanced webcam features that leverage the Neural Processing Unit (NPU) to add professional-grade effects to video streams and audio inputs. Users can access various enhancements, such as:
- Background Blur
- Eye Contact: Adjusts the viewer's perspective to appear as if you are looking directly at the camera.
- Voice Focus: Removes background noise for clearer audio.
These effects can be toggled easily from the Quick Settings menu or through the dedicated camera settings in Windows.
### Live Captions and Voice Clarity
The Live Captions feature is now AI-powered, enabling automatic transcriptions and translations from any audio source. This invaluable tool enhances accessibility for users who may face language barriers.
Moreover, the Voice Clarity feature employs AI to eliminate background noise during direct communications. Initial availability was limited to Surface devices, but it now extends to all compatible hardware with the 2024 Update, showing Microsoft's intent to democratize these tools across their user base.
### Auto Super Resolution
Auto Super Resolution (Auto SR) is a noteworthy addition for gamers. This feature utilizes on-device AI to enhance game graphics, similar to technologies like NVIDIA's DLSS or AMD's FidelityFX. Auto SR will automatically upscale games for better resolution and performance without requiring developers to make changes.
This innovation promises a more immersive gaming experience while maintaining energy efficiency, channeling the workload to the NPU instead of the GPU. The feature can be enabled globally or per application through the Graphics settings page.
### Imaging Features: Restyle Image for Photos and Cocreator for Paint
With the update, the Photos app introduces the Restyle Image feature, employing AI to modify photographs using various artistic techniques. Text prompts can guide the AI in altering images to fit user-directed styles.
Meanwhile, the Paint app receives the Cocreator feature, which scans drawings and assists users in refining their art with generative techniques, making art creation more accessible.
### Copilot App and Suggestions
Significant changes are coming to the Copilot feature too. It now operates through an app interface rather than the previous right-side panel. This new design aligns more closely with chat interfaces that many users are familiar with, such as ChatGPT.
Despite the changes to Copilot's functionality—it's no longer able to alter system settings—it will still offer Copilot suggestions in applications. These suggestions can guide users in problem-solving, enhancing workflow efficiency without directly interfacing with system-level controls.
## Impact and Implications for Windows Users
The Windows 11 2024 Update represents a major shift towards integrating AI into everyday computing tasks. AI's potential to streamline workflows and elevate user experience cannot be overstated. However, the balance between convenience and privacy remains an ongoing concern, particularly with features like Recall that inherently collect user data.
This update aims not only to improve how users interact with their devices but also to make productivity tools accessible to a broader audience. As AI capabilities become increasingly commonplace, users must remain aware of how these technologies are deployed and maintained.
### Expert Commentary
This update places Microsoft in a competitive stance within the tech industry. By embedding AI functionality directly into the core operating system, they enhance usability while proactively addressing common user challenges. Recognizing the sensitivity surrounding data security is crucial. Microsoft has made strides towards data protection; however, hesitance from users regarding features like Recall indicates that trust-building must accompany technological advancement.
## Conclusion
The Windows 11 2024 Update is a testament to Microsoft's commitment to infusing AI into user interactions fundamentally. While the array of features offers exciting prospects for productivity and accessibility, users are urged to remain vigilant regarding privacy implications. Thus, the conversation around AI integration is likely to evolve, reflecting both the opportunities and challenges that come with this powerful technology.
In summary, the advanced AI capabilities ushered in by the 2024 Update stand to reshape the user experience on Windows, inviting both excitement and caution among its users.
Source: Windows Central What's new with AI on Windows 11 version 24H2 (2024 Update)
