The Man who Answers the Search Engines

Date:   Wednesday , September 01, 2010

If search engine is your gateway to the web, solving and providing the right information you seek for; Rajeev Rastogi can be termed as the genie behind who with his deep expertise and constant research find answers to some of the most intriguing technology challenges of search engines today. “Nothing is more amazing and exciting than the world of search. There are billions of web pages out there. How to provide the user with the most relevant information each time keeping in mind the user behavior or demographic pattern is the most challenging game today,” says Rastogi, VP and Head of Yahoo Innovation Lab in India.

Like a curious kid who would not rest until he solves his last piece of puzzle, Rastogi pursues some of the most puzzling phenomenon of information extraction technology today. “It is like solving a puzzle. Each piece though looks unrelated has to be co-related to make it complete. Each time you fix the right piece, the puzzle gets clearer” adds he. Nothing could be more challenging to a man who realized very early in life that he could never do anything which did not have a challenge. The cut-throat competition among search engines has ensured the players are keenly attuned to keeping their visitors satisfied. But serving up handy links is a tricky business, especially when we searchers often use ambiguous terms.

At Yahoo! his team looks at some of the core computer science problems — specifically around data mining — and create solutions that in the long term will deliver a more relevant experience for users and advertisers. “This is an important area to focus on now, with Internet content and data only continue to expand as more people and machines are connected” says Rastogi.

Prior to joining Yahoo!, Rastogi was the founding director of Bell Labs India. He was a Bell Labs Fellow and Distinguished Member of its Technical Staff. Currently, he also sits on the editorial board of Communications for the Association for Computing Machinery (CACM). He has published over 125 papers on database systems and information storage and management and holds over 40 patents.

Rastogi‘s background and expertise in product development, as well as his deep experience in research plays a key role in leading some of the new approaches to information extraction and machine learning, multimedia and query processing in Yahoo!.

Though a hardcore technologist, Rastogi understands and has deep knowledge about the user behavior and the business side of Yahoo!. He speaks eloquently on adding value to the user and his customers. He believes that without understanding the user one cannot bring about the next wave of innovation on the internet. He is a strong advocate of machine learning research which he believes will play a major role in revolutionizing the search engines and internet in a major way. Machine Learning is a branch of artificial intelligence (AI) in which a computer program analyses huge chunks of data and makes predictions about the future. “Applications include understanding the behavior of users, providing advertisements which have higher click probability and can filter pornographic images, determine fraudulent transactions and help investors make investment decisions.”

An internationally acclaimed scientist, Rastogi is an alumnus of IIT Bombay, who went later to the University of Texas to pursue his Masters and a PhD thereafter. "I realized research was my area when I was doing my PhD” says Rastogi. “It clearly changed my way of thinking and my professors were my role models and decided to take up research as my career."

However Rastogi is not happy about the research scenario in India. He says, “Somehow the Indian companies and the Indian environment do not give premium to the students to take up research as a career. The Indian academia, government and companies together should create an ecosystem and create awareness to the students about the positive aspects of pursuing research. More than anything, research as a career has to be more economically viable to the person pursuing it”

After we sign off after the conversation, Rastogi walks to the seminar room where students are eagerly waiting for his session on Machine Learning. After a successful career in U.S. for more than 15 years, he moved to India to replicate the ecosystem of research and innovation he created there. Looking at the 18 page print copy I have in my hand of his resume (which I got from my search on him in Yahoo!), with his achievements and acclaims, I don’t need to say it’s not a difficult feat for him