Google AI: New Ways To Search In 2024 

For more than two decades, we’ve continuously redefined what a search engine can do — always guided by our mission to organize the world’s information and make it universally accessible and useful. This has gone hand in hand with our ongoing advancements in AI, which help us better understand information in its many forms — whether it’s text, audio, images or videos.

As part of this evolution, we’ve made it easier to express what you’re looking for in ways that are more natural and intuitive. For instance, you can search with your voice, or you can search with your camera using Lens. And recently, we’ve been testing how generative AI’s ability to understand natural language makes it possible to ask questions on Search in a more natural way.

Ultimately, we envision a future where you can search any way, anywhere you want. Now, as we enter 2024, we’re introducing two major updates that bring this vision closer to reality: Circle to Search and an AI-powered multisearch experience. Let’s take a look at what’s ahead.

Circle (or highlight or scribble) to Search

Today we unveiled Circle to Search, a new way to search anything on your Android phone screen without switching apps. With a simple gesture, you can select images, text or videos in whatever way comes naturally to you — like circling, highlighting, scribbling or tapping — and find the information you need right where you are.

When something grabs your interest (like these adorable dog goggles!), it can be disruptive to stop what you’re doing and use another app or browser to start searching for information. But now, whether you’re texting friends, browsing social media or watching a video, you can search what’s on your screen right when your curiosity strikes.

Circle to Search is launching globally on select premium Android smartphones on January 31, starting with the Pixel 8, the Pixel 8 Pro and the new Galaxy S24 Series. And as we’ve shared, Search and Shopping ads will continue to appear in dedicated ad slots throughout the results page.

Point your camera, ask a question, get help from AI

In 2022, we pioneered multisearch in Lens as a new way to search multimodally, with both images and text. Since it launched, multisearch has been best for refining visual queries — like searching for a photo of red sneakers with Lens and adding the word “blue” to find them in your preferred color. But now, thanks to our recent breakthroughs in generative AI, multisearch makes exploring the world easier than ever before.

Starting today, when you point your camera (or upload a photo or screenshot) and ask a question using the Google app, the new multisearch experience will show results with AI-powered insights that go beyond just visual matches. This gives you the ability to ask more complex or nuanced questions about what you see, and quickly find and understand key information.

For example, imagine you’re at a yard sale and you come across a strange-looking board game. There’s no box or instructions, so immediately some questions spring to mind: What is this game and how is it played? This is where the new multisearch experience can help.

Just take a picture of the game, add your question (“How do you play this?”), and you’ll get an AI-powered overview that brings together the most relevant information from across the web. This way, you can quickly find out what the game is called and how to win. And with the AI-powered overview, it’s easy to dig deeper with supporting links and get all the details.

AI-powered overviews on multisearch results are launching this week in English in the U.S. for everyone — no enrollment in Search Labs required. To get started, just look for the Lens camera icon in the Google app for Android or iOS. If you’re outside the U.S. and opted into SGE, you can preview this new experience in the Google app. You’ll also be able to access AI-powered overviews on multisearch results within Circle to Search.

Continuing to boldly experiment with generative AI in Search

This week’s launch of AI-powered insights for multisearch is the result of testing we began last year to see how gen AI can make Search radically more helpful, with our Search Generative Experience (SGE) in Search Labs. We’ve gotten lots of useful feedback from people who’ve

chosen to join this experiment, and we’ll continue to offer SGE in Labs as a testbed for bold new ideas.

But our goal is to make AI helpful for everyone, not just early adopters. So moving forward, as we continue to experiment and uncover which applications of gen AI are most helpful, we’ll introduce them into Search more broadly, like we’re doing now with multisearch results.

Today’s updates will make Search even more natural and intuitive, but we’ve only just scratched the surface of what’s possible. To try out the latest capabilities we’re testing, enroll in Search Labs (where available) and opt into the SGE experiment. We hope you’ll join Labs as we continue reimagining the future of Search.

By Elizabeth Reid, Vice President, Search.