Microsoft is set to unveil Copilot Vision for Windows today, enhancing its AI assistant with the capability to share screens or applications. This innovation enables the assistant to guide users through functionalities in software such as Adobe Photoshop, as well as to analyze the images and web pages in users’ view.
According to the Copilot team, “When you choose to enable it, Copilot Vision can see what you see and talk to you about it in real time. It acts as your second set of eyes, able to analyze content, help when you’re lost, provide insights, and answer your questions as you go.”
Initially tested earlier this year, Microsoft is now rolling out the Copilot Vision feature for Windows 10 and Windows 11 users in the United States at no cost. Users will not need a Copilot Pro subscription to access this feature, although it remains part of Microsoft’s experimental Copilot Labs AI initiatives.
Users can activate Copilot Vision through a glasses icon within the Copilot app. This feature allows them to select a browser window or app to share with the AI, similar to sharing during a Microsoft Teams meeting. The use of Copilot Vision is opt-in, meaning users must actively share their applications for it to function, unlike Recall, which continuously captures screen content.
While unable to fully explore Copilot Vision due to its limited release in the US, I experienced a brief demonstration at Microsoft’s 50th anniversary celebration in April. Following the sharing of an application, Copilot can highlight various sections of the screen to provide guidance and answer queries. In addition, Copilot Vision will be available on both iOS and Android platforms.
Update, June 12th: This article has been updated to clarify that Copilot Vision does not require a Copilot Pro subscription.
