Google is in the process of launching its two key features for the Gemini platform—live video and screen-sharing. Announced at Google I/O 2024, these functionalities have been developed by Google DeepMind under the Project Astra initiative. These innovations are designed to enable real-time multimodal data processing, allowing the AI chatbot to respond to user queries about their devices and environments instantly. The company had initially indicated that these features would be available by March. Presently, they are accessible exclusively to Gemini Advanced subscribers using mobile applications.
Google Initiates Rollout of New Gemini Features
The features were first highlighted by 9to5Google, where Reddit user Kien_PS shared a screenshot on the Bard subreddit, showcasing the “Share-screen with live” capability. This user later provided a demo video on Sunday, illustrating the functionality of the feature.
In a separate statement, Google spokesperson Alex Joseph informed Technology News that these AI enhancements are being rolled out to Gemini Live. Besides the screen-sharing option, Gemini will now also utilize the user’s device camera to provide answers based on what the user observes in real-time.
The real-time data processing ability enables users to ask Gemini for outfit recommendations by displaying their wardrobe or to identify landmarks or stores while outside. The screen-sharing functionality builds upon the existing “Talk about the screen” feature, allowing Gemini to assist users as they navigate through various screens on their mobile devices.
Both features are part of Gemini Live, which was introduced to users last year and is capable of conducting two-way live voice conversations. Google has expressed its intention to enhance Gemini’s usefulness in real-time situations.
Additionally, the Gemini Live video feature bears similarities to OpenAI’s Advanced Voice Mode with Vision for ChatGPT, as well as the real-time video functionality found in Ray-Ban Meta Smart Glasses. With advancements in AI and the supporting infrastructure, tech companies are now capable of delivering faster inference for real-time applications.
At the moment, the new Gemini features are exclusive to Gemini Advanced subscribers. Google has not yet disclosed details about the potential rollout to free users. A Gemini Advanced subscription can be acquired through the Google One AI Premium plan at a price of Rs. 1,950.
