The PC is on the verge of a significant transformation that diverges from its traditional role. Microsoft has been working towards this shift, emphasizing that the fundamental component of this evolution will be AI , particularly Copilot . This new AI-driven approach aims to enhance user interaction, making the experience more fluid and intuitive.
The integration of Copilot into PCs and Windows 11 has been gradual, yet Microsoft intends to accelerate this change. They believe this transition will revolutionize how we interact with our PCs and redefine our work habits—significantly reducing the amount of time we spend manipulating devices directly.
“Hey, Copilot”: The Voice as a Catalyst for Change
The mouse and keyboard have been fundamental tools for interacting with computers for decades. However, Microsoft recognizes that this paradigm is shifting. Instead of relying solely on these traditional input devices, voice interaction is becoming increasingly prevalent. According to Microsoft, PCs must evolve to achieve three key objectives:
- Enable natural interaction through both text and voice, with an ability to comprehend user input.
- Allow the PC to perceive what the user sees, providing guided assistance based on real-time feedback.
- Empower the system to execute actions and complete tasks autonomously, benefiting the user.
To support this initiative, Microsoft has introduced the feature that allows users to start a conversation with their PC simply by saying “Hey, Copilot.” This voice command, when activated in the Copilot settings, presents an on-screen microphone icon and a sound alert, making it easy for users to engage with their PCs.
The second component of this interaction comes from Copilot Vision , a feature that enables the AI to view the desktop and applications as the user does. Announced in previous updates, this capability is set to launch globally wherever Copilot is available. Copilot Vision is designed to assist users by answering queries based on the visible screen content, implementing a concept similar to what OpenAI’s Operator and Anthropic’s Computer Use have proposed.
Moreover, users can utilize the Highlights feature, prompting Copilot with requests such as “teach me how to do this in this app.” This functionality extends to popular applications like Word, Excel, and PowerPoint, allowing Copilot to enhance user experience by writing or analyzing tasks.
While voice input remains central, Microsoft plans to include a chat window option for users who prefer typing their commands. This flexibility is indicative of Microsoft’s ambition to establish Copilot as the core of the user experience within Windows 11. Evidence of this ambition is also present in the new “Ask Copilot” button on the taskbar, envisioned as a dynamic hub for achieving more with minimal effort. This feature requires proactive activation in the Windows settings.
Copilot Actions: The AI That Works For You
Another promising feature on the horizon is Copilot Actions , where the AI can take control of browser activities and manage applications on the computer to fulfill user tasks. Microsoft describes Copilot Actions as an AI agent capable of executing tasks by interacting with various applications and files, mimicking human behavior. Users can expect this capability to become fully operational across all desktop applications, shifting Copilot from a passive assistant to a proactive aide. Tasks that Copilot will handle include updating documents, organizing files, sending emails, or even booking flights.
To utilize Copilot Actions, users need to grant permission for the AI agent to access their applications and data. This requirement does raise concerns regarding data security, particularly regarding potential mistakes or improper data handling by the AI. To mitigate these concerns, Microsoft employs rigorous security measures. Each agent operates within a contained workspace , ensuring limited access to sensitive information. Additionally, agents start their activities with restricted permissions and can only access extra resources upon user approval.
In the preview version of Copilot Actions, the AI can access specific directories like Documents, Downloads, Desktop, or Pictures, thereby enhancing user collaboration while ensuring privacy. Furthermore, agents must be “signed” by trusted sources, similar to how applications are distributed through renowned app stores. Microsoft’s aspirational goals are commendable; however, users in Spain and the European Union may currently find many Copilot features unavailable. Efforts are underway to clarify these limitations with Microsoft representatives.