Browse by year:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

The Smart Techie was renamed Siliconindia India Edition starting Feb 2012 to continue the nearly two decade track record of excellence of our US edition.

April - 2012 - issue > In Conversation

Bucket Testing The best way to optimize a website

Ashish Vikram

VP- Engineering, Heads User Data and Analytics (UDA) Group-Yahoo!

Tuesday, April 10, 2012

Forward

The success of several product changes (such as new features, user interface improvements, content and design concepts) at Yahoo! is measured scientifically, by rolling out the changes to a sample set of population instead of the entire user population, via experimentation a.k.a. bucket testing. This is done keeping various things in mind, like — to test product features, user interface design algorithms on a smaller scale before exposing it to larger audience, to test out ideas to optimize the tradeoff between monetization and user experience, forecast the impact of product changes to the overall page performance, gain insights into user behavior and more.

Ashish Vikram, VP- Engineering heads the User Data and Analytics (UDA) group at Yahoo! India R&D, that analyzes Yahoo!’s customer data to provide deep insights into user behavior, which is then leveraged to personalize digital content and enhance Yahoo!’s products. In a candid conversation he talks about the bucket testing process and its advantages.

Why is bucket testing important?

User data is critical to us for we need to constantly analyze it, show the ROI to the advertisers on the website. The advertisers need the statistics to ensure that by placing ads on Yahoo!, they are able to reach the required demographics. The data is also used to improve the engagement on our website and bucket testing becomes very important for this.

When you have a site that is getting millions of page views a day, it is quite risky because if you make changes that cause your audience to go away, they may never comeback. So at Yahoo!, the way we make changes first introducing the new design to a small percentage of the users, say about five percent. The changed UI is run for a set period of time, varying from weeks to several months, depending up on the data required. The system automatically collects the data from the five percent bucket and with statistical analytics UDA group confirms if the new changes are working better or not and if so how much better.