Google is inserting more of its Gemini ai into every product it has, and the next target in its sights is Photos. At its I/O developer conference today, the company's CEO Sundar Pichai announced a feature called Ask Photos, which is designed to help you find specific images in the gallery of him talking to Gemini.
Ask Photos will appear as a new tab at the bottom of the Google Photos app. It will begin rolling out to One subscribers first, starting in US English in the coming months. When you tap that panel, you'll see the Gemini star icon and a welcome message above a bar asking you to “search or ask about Photos.”
According to Google, you can ask things like “show me the best photo of every national park I've visited,” which not only relies on GPS information but also requires the ai to exercise some judgment to determine what's “best.” The company's vice president of photos, Shimrit Ben-Yair, told Engadget that he will be able to provide feedback to the ai and let it know which images it prefers. “Learning is key,” Ben-Yair said.
You can also ask Photos to find your top photos from a recent vacation and generate a caption to describe them so you can share them more quickly on social networks. Again, if you didn't like what Gemini suggested, you can also make adjustments later.
For now, you will have to write your question in Ask Photos; voice input is not yet supported. And as the feature rolls out, those who choose to use it will see their existing search feature “upgraded” to Ask. However, Google said that “you won't miss key search features, like quick access to your face groups or the map view.”
The company explained that the Ask Photos process consists of three parts: “understand your question,” “craft an answer,” and “ensure security and remember corrections.” Although safety is only mentioned in the final stage, it must be baked throughout. The company acknowledged that “the information in your photos can be deeply personal and we take our responsibility to protect it very seriously.”
To do this, queries are not stored anywhere, although they are processed in the cloud (not on the device). People will not review conversations or personal data on Ask Photos except “in exceptional cases to address abuse or harm.” Google also said it does not train “any generative ai products outside of Google Photos with this personal data, including other Gemini models and products.”
Your media continues to be protected by the same security and privacy measures that cover your use of Google Photos. That's a good thing, since one of the potentially most useful ways to use Ask Photos could be to get information like passport or license expiration dates from photos you might have taken years ago. It also uses Gemini's multimodal capabilities to read text in images and find answers.
Of course, ai is not new to Google Photos. You've always been able to search the app for things like “credit card” or a specific friend, using the company's facial and object recognition algorithms. But Gemini ai offers generative processing, so Photos can do much more than just display images with certain people or items in them.
For example, you can also get Photos that tell you what themes you might have used at the last birthday parties you threw for your partner or child. Gemini ai is working here to study your images and find out what themes you've already adopted.
There are many promising use cases for Ask Photos, which is an experimental feature at the moment and will “begin rolling out soon.” Like other Photos tools, it could start as a premium feature for One subscribers and Pixel owners before rolling out to everyone who uses the free app. However, there is no official information yet on when or if that could happen.
Catch up on all the news from Google I/O 2024 here!
