At its Search On event today, Google unveiled several new features that together are its strongest attempts yet to get people to do more than just type a few words into a search box. By leveraging its new multitasking unified model (MUM) machine learning technology in small ways, the company hopes to start a virtuous cycle: it will provide more details and context-rich answers, and in return, expect users to ask more detailed and in-context questions. . -Good questions. The company hopes that the end result will be a richer and deeper search experience.
Google Senior Vice President Prabhakar Raghavan oversees searches alongside Assistant, ads, and other products. He likes to say – and repeated in an interview last Sunday – that “search is not a solved problem.” That may be true, but the problems he and his team are trying to solve now have less to do with arguing on the web and more to do with adding context to what they find there.
For its part, Google will begin to flex its ability to recognize constellations of related topics using machine learning and present them to you in an organized way. An upcoming Google search redesign will start showing “Things to know” boxes that will direct you to different subtopics. When there is a section of a video that is relevant to the overall topic, even when the video as a whole is not, it will send it there. The shopping results will begin to show available inventory at nearby stores and even clothing in different styles associated with your search.
For its part, Google is offering, although perhaps “asking” is a better term, new ways to search that go beyond the text box. It is making an aggressive effort to take its Google Lens image recognition software to more places. It will be integrated into the Google app on iOS and also into the Chrome web browser on desktop computers. And with MUM, Google expects users to do more than just identify flowers or landmarks, but use Lens directly to ask questions and shop.
“It’s a cycle that I think will continue to increase,” says Raghavan. “More technology leads to greater accessibility for the user, it leads to better expressiveness for the user and, technically, it will demand more of us.”
Those two sides of the search equation are meant to kick off the next stage of Google search, one in which its machine learning algorithms become more prominent in the process by directly organizing and presenting information. In this, Google’s efforts will be greatly aided by recent advancements in AI language processing. Thanks to systems known as large language models (MUM being one of these), machine learning has greatly improved at mapping the connections between words and topics. It is these skills that the company is leveraging to make the search not only more precise, but also more exploratory and, hopefully, more useful.
One of the Google examples is instructive. You may not have the first idea what the parts of your bike are called, but if something breaks, you will have to find out. Google Lens can visually identify the derailleur (the gearshift part that hangs near the rear wheel) and instead of just giving you the discreet information, it will allow you to ask questions about how to fix that thing directly, leading you to the information (in this case, the excellent Berm Peak Youtube channel).
The push for more users to open Google Lens more often is fascinating on its own merits, but the big picture (so to speak) is about Google’s attempt to gather more context about their queries. More complicated multimodal searches that combine text and images require “a completely different level of contextualization than we vendors have to have, so it helps us tremendously to have as much context as we can,” says Raghavan.
We are a long way from the so-called “ten blue links” of the search results that Google provides. It has been displaying information boxes, image results, and direct answers for a long time. Today’s ads are another step, one in which the information Google provides is not just a classification of relevant information, but a distillation of what its machines understand when crawling the web.
In some cases, like shopping, that distillation means you’ll likely send more page views to Google. As with Lens, it’s important to keep an eye on that trend – Google searches increasingly push you to Google’s own products. But here there is also a greater danger. The fact that Google is telling you more things directly adds to a burden it has always had: speak with less prejudice.
By that, I mean bias in two different senses. The first is technical: the machine learning models Google wants to use to improve search have well-documented problems with racial and gender biases. They train by reading large swaths of the web, and as a result, they tend to learn unpleasant ways of speaking. Google’s issues with its AI ethics team are also well documented at this time: fired two principal investigators after they published an article on this very topic. As Google Vice President of Search Pandu Nayak said The edge‘s James Vincent in his article on today’s MUM announcements.Google knows that all language models have biases, but the company believes it can avoid “publishing it for people to consume directly.”
Be that as it may (and, to be clear, it may not be), it sidesteps another consequential issue and another kind of bias. As Google begins to tell you more about its own information syntheses directly, what is the point of view from which you are speaking? As journalists, we often talk about how the so-called “view from nowhere” is an inappropriate way of presenting our reports. What is Google’s point of view? This is a problem that the company has faced in the past, sometimes known as the “one true answer” problem. When Google tries to give people short, definitive answers using automated systems, it often ends spread bad information.
To that question, Raghavan responds by pointing out the complexity of modern linguistic models. “Almost all language models, if you look at them, are inlays in a large space. There are certain parts of these spaces that tend to have more authority, certain parts that have less authority. We can mechanically evaluate these things quite easily, ”he explains. Raghavan says the challenge is how to present some of that complexity to the user without overwhelming them.
But I have a feeling that the real answer is that, for now at least, Google is doing what it can to avoid facing the issue of its search engine’s point of view by avoiding domains that it could be accused of, as it says. Raghavan. , “Excessive editorialization”. Often when they talk to Google executives about these bias and trust issues, they focus on the easier-to-define parts of these large spaces as the “authority.”
For example, the new “Things to know” boxes from Google will not appear when someone searches for things that Google has identified as “particularly harmful / sensitive”, although a spokesperson says that Google is “not allowing or rejecting specific selected categories, but our systems are able to understand in a scalable way the topics for which these types of functions should or should not be activated “.
Google search, its inputs, outputs, algorithms, and language models have become almost unimaginably complex. When Google tells us that it is now able to understand the content of the videos, we assume that it has the computer skills to achieve it, but the reality is that even indexing such a massive corpus is a monumental task that overshadows the original mission of indexing the video. early web. (Google is only indexing audio transcripts from a subset of YouTube, for the record, although with MUM it intends to do visual indexing and other video platforms in the future.)
Often when talking to computer scientists, the peddler problem get on. It’s a famous conundrum where you try to calculate the shortest possible route between a certain number of cities, but it’s also a rich metaphor for thinking about how computers perform their machinations.
“If they gave me all the machines in the world, I could solve some pretty big cases,” says Raghavan. But for the search, he says it is unsolved and maybe it can’t be solved just by throwing more computers at it. Instead, Google needs to come up with new approaches, like MUM, that make better use of resources that Google can realistically create. “If you gave me all the machines there were, I’m still limited by human curiosity and cognition.”
Google’s new ways of understanding information are impressive, but the challenge is what to do with the information and how to present it. The funny thing about the salesman problem is that nobody seems to stop and ask what exactly is going on, what are you showing all your customers when you go door to door?