There’s no denying it, we live in the era of Big Data.

A recent report of McKinsey Global Institute titled “Big Data: The next frontier for innovation, competition and productivity” points out the following:

- There were 5 billion mobile phones in use in 2010 - 30 billion pieces of content are shared on Facebook every month - 235 terabytes of data were collected by the U.S. Library of Congress in April 2011 - 15 out of 17 business sectors in the U.S. have more data stored per company than the U.S. Library of Congress - The amount of global data generated annually is projected to grow by 40 percent. Where is all this data coming from? Companies churn out all types of transactional data about their customers, partners, suppliers and operations. We now have networked sensors embedded in everything from cars and consumer appliances to smart meters and smart phones. Along with smart phones, we now have social media sites and new tablets and other hand-held WiFi devices that have given billions of individuals around the world the ability to contribute to the accumulation of Big Data. As big a challenge as Big Data presents us, the opportunity is even greater. According to that same report, McKinsey estimates that a retailer that can effectively derive the right insight from Big Data has the potential to increase its operating margin by more than 60 percent!

The bigger question is how do you clean or extract that insight and leverage that Big Data to its fullest? After years of helping clients in retail, financial services, high technology and telecommunications improve marketing effectiveness, customer loyalty and risk management by applying business analytics solutions, I feel we have a way forward.

Data warehousing, business intelligence, and business analytics, among other software technologies, along with knowledge/business process outsourcing approaches, have been adopted to tackle and analyze massive datasets for years. But in the era of Big Data, the datasets have grown so large and are growing so fast, these traditional technologies and approaches are not adequate. So, the answer to tapping the full potential of Big Data lies in yet another tectonic shift. Underlying this shift are three fundamental drivers affecting the business landscape - The mainstream adoption of software-as-a-service (SaaS) delivery models; - The need for offshore based service providers to adopt non-linear growth models; and - The increasing trend towards building individual problem-specific software solutions —the so-called “Apps” revolution.