The search engine can set you free

He’s one of the World Wide Web’s good guys. Someone Kofi Annan would wholeheartedly endorse. Amit Singhal, one of the backroom boys of Google, shines in a world that the former UN secretary-general believes can be liberated by information. Toiling over his algorithms, Singhal makes sure everyone gets the right information at the right time.
The algorithm was developed by the legendry duo of Sergey Brin and Larry Page in 1997, but Singhal rebuilt it in 2001. He is currently one of the world’s four Google Fellows and is a major influence on how Google serves its users.
Tim Berners-Lee, the man who invented World Wide Web, once said, “The Google algorithm was a significant development. I’ve had thank-you emails from people whose lives have been saved by information on a medical website or who have found the love of their life on a dating website.”
COMPLEX WORKING
The internet search is seemingly simple. You put in a keyword and milliseconds later, the information is neatly presented in a list of top search results. But dig a little deeper and things are profoundly complex.
Singhal explains how a search engine works. “At the basic level lies a crawler”, which goes out and collects web pages. These pages are then indexed. “In the beginning, our crawlers used to go out every month. This duration gradually reduced and since the introduction of our new search technology, Caffeine, two months ago, we are now bringing in the new information on a 24x7 basis,” Singhal claims.
He says the PageRank system is the secret sauce that makes search results appealing. “To assign a page rank, we use more than 200 hooks. One of the hooks, for example, is the placement of the keyword. Is it in the title or is it in the body? And this is just one tiny part of how our algorithms process the information”.
WORDS TO CONTEXT
When modern search engines appeared on the Web some 17 years ago, words had literal meanings. Thus, ‘GM’ was just ‘GM’, not General Motors or genetically modified. “This was a problem because all human beings understand the context. Ludwig Wittgenstein, a language philosopher, once wrote that words only mean their context,” says Singhal.
“But today, we have brought the context into search.” That means that if a cricket enthusiast uses the keyword ‘lord’ on Google now, the search engine will use his search history and come up with information on Lord’s rather than the Almighty. On the other hand, for a person with a spiritual bent of mind, the result will be the opposite.
A smart search engine is just the beginning of better times. As more and more information goes online, Singhal envisages a future where it will be “simply available” when you need it. “You won’t have to look for it. You will not have to search. This is where we are heading,” he says with the conviction of a man who has seen the fruition of his dreams over the last two decades.
Before then, search engines have to get past bottlenecks. To begin with, there is the real-time challenge. Despite its expertise, Google was caught unawares by Twitter and Facebook where information is generated and consumed in real-time. “Real-time has been a major focus in the last few years and we have improved a lot with Caffeine,” claims Singhal, demonstrating the way the ‘Updates’ feature on the Google homepage tracks tweets in real-time.
HANDY CELLPHONES
Search engines also need to work on the ‘search beyond text’ option, though they have managed to crack search through voice and image on the smartphone. One brand of smartphones allows a tourist, for example, to simply point the instrument at the Taj Mahal and get information on the monument from the web.
Smartphones are a key element in Singhal’s futuristic world. It’s a world where we won’t encounter the science of search on some webpage but in our daily lives. “I am incredibly excited about the ‘search without searching’, a device like cellphones that will take over the work of looking for information.”
He offers an example. “You want to buy a cricket bat and you put it on your to-do list. So, your phone knows about it. It also knows which route you take to commute from office to home because you use GPS navigation built into it. Using local business listings, it knows that several shops on the route sell cricket bats. I imagine a future when you will get a prompt from your phone telling you to buy the bat just when you are near the concerned shop,” he says. “That’s the future – when we won’t have to look for information. It will be available everywhere just at the moment when you need it.”
WANDERING THE WEB
1990: Archie, the world’s first search engine, is born in McGill University in Montreal
1991: Gopher, a computer protocol for distributing, searching, and retrieving documents, is developed by Mark McCahill at the University of Minnesota
1993: Wanderer is developed at MIT. It crawled the web and created ‘wandex’
1994: WebCrawler allows users to search using keywords
1994-1996: AltaVista, Yahoo, Lycos, Inktomi, etc, burst onto the scene
1997: Google unleashed. PageRank system decimates opponents
2004: Microsoft works on search engine. Readies MSNBot
2007: True Knowledge attempts to go beyond search. It gives you answers.
In 2009, search engine Wolfram Alpha debuts
2009: Microsoft launches Bing
2010: Google strikes back with Caffeine

Javed Anwer


#-Link-Snipped-#