On Thursday, OpenAI introduced support for a variety of note-taking and coding applications within the ChatGPT app for macOS. The company characterized this feature as agentic, enabling the chatbot to interpret on-screen information from the supported applications and assist users with their queries. Currently, this functionality is accessible only to paid subscribers, who can also utilize the Advanced Voice Mode while engaging with these applications. OpenAI has announced that Windows users and free tier subscribers will gain access to this feature next year.
ChatGPT Now Works With Some Third-Party Apps
The launch of this feature was revealed on the 11th day of OpenAI’s 12-day shipping initiative. The new agentic ability allows ChatGPT to read inputs from various note-taking and coding apps, a detail highlighted in a post on X (formerly Twitter) by the company.
Kevin Weil, Chief Product Officer at OpenAI, disclosed that ChatGPT is compatible with approximately 30 macOS applications. The comprehensive list includes popular programs such as Apple Notes, BBEdit, MatLab, Nova, Script Editor, TextMate, VSCode Insiders, VSCodium, Cursor, WindSurf, Android Studio, AppCode, CLion, DataGrip, GoLand, IntelliJ IDEA, PHPStorm, PyCharm, RubyMine, RustRover, WebStorm, Prompts, Warp, Notion, and Quip.
ChatGPT for macOS working with Notion
Photo Credit: OpenAI
With this feature enabled, users can launch a supported app and navigate to any desired screen or content. By using the shortcut Option + Space, a small text-field view of the app will open, allowing the chatbot to “see” what is displayed on the user’s screen. Users can ask questions using text prompts or the Advanced Voice Mode, and ChatGPT will provide responses. Additionally, users can highlight specific sections of text or code and issue natural commands like “how to make this better,” and the AI will grasp the context of the inquiry.
Although OpenAI refers to this functionality as agentic, it fundamentally operates as a vision-based capability. The agentic aspect pertains primarily to obtaining permission for app access. Despite this, ChatGPT is unable to interact with these apps in a way that involves generating text, writing code, or altering existing information.
For user privacy, individuals will have the ability to manage which apps can be accessed by ChatGPT and under what circumstances. OpenAI has emphasized that users have the same level of control over this data as they do over their conversation history.
This new feature is available to subscribers of ChatGPT Plus, Pro, Teams, Enterprise, and Edu plans. Free users and those using the Windows app will need to wait until next year to gain access to these capabilities.
