Google is in the midst of what it described as the biggest change to its ranking algorithm for at least five years, and one of the biggest ever, as it pushes artificial intelligence deeper into its search engine.
The change, which involves a new language analysis technique to try to understand users' queries better, is set to affect its response to one in 10 searches, according to Pandu Nayak, vice-president of search. Google began to update the software in its data centres in recent days to make the change for English language searches, and will eventually apply it to other languages as well, he said.
The update marks the first application of a piece of research on natural language processing from last year that has drawn considerable attention in AI circles. Understanding language is one of the most difficult problems in AI, given the fluidity of language depending on the context and the person using it.
Until now, Google's algorithm has tried to single out the most important words in any search query, ignoring the many smaller or common words that seem less significant. This enables it to zero in on the main subject matter of a query, but often results in it misunderstanding a user's precise request.
The new technique, known as BERT, relies on a huge, general-purpose language model that has been built up from analysis of vast amounts of text online. Rather than read the string of words in a query sequentially, it analyses them all at the same time - including smaller words that would have been ignored before.
The search company said that the BERT technique would return more useful responses to many queries, though the change would be too subtle for most people to notice. One example it gave of the types of question it could now handle - "How old was Taylor Swift when Kanye jumped on the stage?" - points to more complex queries that have been beyond its reach before.
Mr Nayak also said that in some instances the new algorithm had been shown to produce worse results than before, making it a constant work in progress. "I don't think we're anywhere close to solving language, but this is a good step," said Jeff Dean, Google's head of AI.
Google's periodic updates to its ranking algorithm have sometimes resulted in big changes in the amount of traffic directed to outside websites, angering companies whose businesses depend on the search engine.
One impact of the current change, for instance, could be to reduce the amount of traffic that Google sends to some websites in languages other than English. That is because the improved language understanding should start to answer more questions directly in these languages, producing "snippets" of text at the top of search results, rather than forcing users to click on a link to visit another website.
This method has already been applied to English language searches and is one reason that far fewer searches on Google now result in a click through to another website - though the company says that answering questions directly has been a boon for users.
Ben Gomes, head of search, said that the BERT change would not have much impact on the traffic Google passes on. Instead, he predicted that users would instinctively begin to ask more complex questions as the responses subtly improved. That would make the system more useful, increasing the number of searches and the amount of traffic Google passes on, he added.
All Rights Reserved © 2024
