Dr. Prabhakar Raghavan wishes to achieve iconic breakthroughs in the search engine space; he hopes to make human-human interactions on the web – one in which the machine and the Internet become irrelevant – a reality, and is propelled by the desire to see his contribution rank beside Xerox Parc’s human-computer interaction breakthrough.
For the uninitiated, Raghavan is the head of Research at Yahoo, his efforts dovetailed towards developing ‘social search’ – a broad effort to enhance computerized Web search tools with insights gained from mining the collective knowledge of its users. After getting on board Yahoo in June 2005, the 45-year old scientist has devoted most of his energies towards identifying and working on the thrust areas that constitute the linchpin of Yahoo’s strategy in taking on its arch rivals:

Search and information retrieval, which is all about matching queries with people.

Machine-learning/ data mining, which would concern discerning user behavior from the 12 terabytes of data that flows into Yahoo on a daily basis. Responding to the need for an expert in the field, Raghavan hired web retrieval and mining expert Dr. Ricardo Baeza-Yates to head the company’s Web search and data mining focused research labs in Spain and Chile.

Microeconomics: A tremendously consequential aspect to the development of online business. Explains Raghavan, “Earlier, research constituted code-writing by an engineer, the marketing arm would then take over.” Today we have moved into an era where monetising aspects are intertwined with engineering advances; the code writing process has to driven by the microeconomic concerns of how the potential product can help ratchet up the revenues. This calls for an involvement of people with microeconomic thinking right up front in the research stage. Realizing this, Raghavan brought on board former Harvard professor and economist Michael Schwarz within months of joining Yahoo. Primary among Schwarz’s contributions have been the query incentive networks, implemented on Yahoo Answers. Under the incentive system, users accumulate points based on their activities; more points leads to recognition within the community, central to keeping the user hooked.

Community Systems: The challenge in this arena was about identifying the right set of middleware that would drive the Internet applications central to Yahoo’s strategy. The middleware would have to make sense of how people in an online community strike friendships, the trust factor, and recommendation on ratings of a book/ movie/ play by users. Raghavan’s thoughts in this area were crystallized after he hired Raghu Ramakrishnan, Professor at the University of Wisconsin-Madison and a researcher on databases. Ramakrishnan has been instrumental in the development of community based search mechanism, currently running in the pilot mode.

Media Experience Research: As opposed to identifying and working on an algorithm based on what users consume / produce online, as did the earlier thrust area, Media Experience Research is all about answering why they do it. Says Raghavan, “It’s essential to analyze upfront what might strike a chord with users based on why they do something, as opposed to guessing what might be hot and developing the technology.” This spells out a need for sociologists and anthropologists in the online business. They must study what elements of an experience makes people want to come back and ‘hang out’ on the portal, and what incentives could be offered to keep them enamored.

In consonance with proceeding on the thrust area development initiative, Raghavan went about sprucing up Yahoo’s research team. His brief from the company’s executive management in this regard was simple: settle for the best. That meant going after luminaries like Yates, Ramakrishnan and Schwarz, ones who were being pursued by all the leading Internet companies. With the help of the management, Raghavan also ushered in former vice president of research and chief scientist at AltaVista Andrei Broder. The exercise set the tone for excellence and sent out signals both within and outside the company: that Raghavan, and consequently Yahoo would not settle for anything but the best.

Getting into the game
Raghavan’s own entry into the company happened at a time when he was being wooed by Microsoft and Google. He chose Yahoo owing to the mission set out for him: to build a top-notch research organization that would drive the next generation of the internet. The profile, he realized, would give him the opportunity to combine his algorithm and information retrieval abilities with the people angle. Yahoo clocked 500 million unique users a month then, and it was a dream come true for him.

“My style is to be up at the whiteboard, brainstorming with colleagues, and failing repeatedly till I come up with a breakthrough,” says Raghavan. This was the reason why the semiconductor design and architecture field could not ignite his passion. As a young graduate pursuing his masters at Berkeley, VLSI design had excited his faculties tremendously, so much so that he had plunged into making semiconductors. But soon, sitting in the lab, waiting for the ‘chips to bake’, he realized that it wasn’t a social place. It didn’t seem ‘fun’, he recalls. He switched fields and took fresh guard in the chip design arena, and then in computer architecture, all within a 2-year span. His moves were in line with hot industry trends. Looking back, Raghavan defends his shifts saying he was too naïve then, and his shifts cannot be labeled big decisions. It was in quest of what he calls ‘passion with pragmatism’. Not only would he need to follow the hot areas, but among them discover what ignited his passion, and more importantly, where he could make a big difference.

When he chanced upon cryptography and algorithms towards the end of his doctorate degree, it gave him all that he sought. The fact that a new algorithm could effect sweeping changes in the lives of people grabbed him. “You can think about an algorithm just about anywhere, without bothering about any apparatus. It possesses you till you find a solution,” he says. Moreover, working on algorithms would enable him to be in the midst of people all the time. He joined IBM’s Research team after completing his PhD from UCB. After 14 years at IBM’s Almaden research center, and captivated by the search engine space, he moved to Verity Inc. in 2000. Eventually came the move to Yahoo in July 2005.

Raghavan has a piece of advice for young engineers. An engineer always builds applications with respect to the company’s product roadmap; they do not get an opportunity to experiment much. But their very status as engineers signifies that they embody a creative spark. “If they can create a cushion and combine their creativity with a pursual of their dreams, they too can achieve iconic breakthroughs,” he quips. As for researchers like him there is enough room to elbow experiments. After all, not every research of a scientist is successful, nor should it be that way. Only some bring about impeccable payoffs, and that’s what Raghavan is chasing at Yahoo!