Google is enhancing its search-focused AI Mode chatbot with new multimodal features that allow it to interpret and respond to questions about images. This upgrade will be rolled out to “millions more” users, expanding the availability of AI Mode.
The latest update integrates a tailored version of Gemini AI alongside Google’s Lens technology for image recognition. Users will now have the ability to take or upload a photo and receive detailed responses, complete with relevant links, regarding the image’s content. This multimodal functionality is accessible starting today through the Google app on both Android and iOS devices.
“AI Mode builds on our extensive work in visual search and takes it to the next level,” said Robby Stein, Vice President of product for Google Search. “With Gemini’s multimodal features, AI Mode is capable of comprehending the entire scene within an image, taking into account how objects are related, as well as their specific materials, colors, shapes, and arrangements.”
According to Google, this update utilizes a “fan-out technique” that generates multiple queries regarding the image and its contents, which leads to responses that are both nuanced and contextually pertinent. This capability includes features such as the identification of books seen within an image, recommendations for other titles with high ratings, and answers to queries that help refine suggestions.
AI Mode for Search is Google’s competitive offering against platforms like Perplexity and ChatGPT Search, delivering chatbot-like interactions that provide AI-generated summaries derived from Google’s expansive search index.
Initially, AI Mode was made available exclusively to Google One AI Premium subscribers last month, limited to Labs. However, the company has announced that it has now begun to extend AI Mode to millions of additional users within Labs in the United States, broadening access beyond just paying subscribers.
