BI & Analytics : Evolution and Future

Date:   Monday , January 02, 2017

Listen. Get Data. Think. Get Data. Evaluate. Get Data. Analyze. Get Data. Synthesize. Decide. Transform.


Given that the analytics function has increasingly higher levels of sponsorship and responsibility in forward-thinking organizations today, it is hard to imagine that just about fifteen years back, it was a very different story. Till about the mid-nineties, the word \"analytics\" was non-existent. The function, if I may call it that, was buried within \"MIS\" teams or a complementary part of the data-warehousing group. If you needed a report that outlined what products you were selling in which markets and what price points you sent a request to the \"IT team\", and you would receive a tabular report after a couple of days (we cannot imagine waiting more than a few seconds today!). This was the time of centralized databases, report requisitions, MIS specialists and the occasional external consultant who would make sense of it all.

Fast forward a few years, and we had the first real industry-impacting shift information federation. This was made possible by the relative ease of availability of corporate data in structured data repositories coupled with 3 critical factors. First, the willingness of organizations to take hard business decisions based on data (and not just \"gut\"); second, the introduction of the \"business analyst\" to the job market and finally the search engine. You now had some very smart people looking at up-to-the-day data (their own, competition and the market), and building relationships with your customers, gaining new clientele, making obscene amounts of money for their employers and then making sure that anyone with a web-enabled browser could find out about it. The banks and mass-market retail corporations led this wave and the rest of the financial industry wasn\'t far behind. Over the last 10-12 years, Enterprise BI has focused on areas like maximizing monetization on marketing campaigns, increasing customer loyalty and purchase propensity, risk and fraud.

Several enterprises across the world are investing in this capability using a variety of engagement models in-house, captive, specialized outsourcing and generic outsourcing. The largest proportion of analytics consulting work is still centered on these \"traditional\" application areas where increasingly advanced techniques are being used to identify ever-elusive target groups and increase monetization. Indian companies are leading the way in providing analytics-as-a-service to global audiences.

Indian businesses have traditionally focused more on increasing penetration through expanding their logistics chains but as the markets become more saturated, they are beginning to understand the value of real-time customer and market information, and the need to drive profitability through data-driven insights. We should see these organizations build out their analytics capability over the next 1-2 years.

With the proliferation of structured and unstructured data and creation of analytics platforms and appliances with advanced visualization/dash boarding tools, those at the cutting edge of this wave are integrating customer data as it gets produced into relatively real-time decision-making. An example of this is the analysis of tweets at a racing event where there was a car crash. Within moments the spectators were tweeting about it, and the company managing the data feeds took two very important and responsible decisions. Through geo-tagging and image analysis, they were able to pinpoint the location and profile of the injured to emergency responders not only saving precious lives, but also preventing inaccurate rumors about the incident from spreading. This was possible due to the convergence of technology, real-time visual analytics, information synthesis and actionable outputs key components of analytics as we envisaged ten years back.

I expect the next major shift to ride on the back of energy efficient, programmable hardware. Most of the statistical and business analysis today is administered through what we generically call \"software programming\" using tools like SAS, SQL or Excel. The limitations of the underlying hardware prevent analytics \"jobs\" from executing in rapid succession on the same hardware/appliance. With the development of ever more advanced chips like the Field Programmable Gate Array (FPGA) developed by Microsoft (and used in Bing), we can switch to a different kind of job in less than a second on the same hardware infrastructure. The investments required for enterprise-grade analytics would see a dramatic reduction, for the same capability.

Analytics consultants are increasingly being recognized as the conscience keepers of the organization and also engaged as strategic partners in charting out sustainable and transformative plans for the future. We need to constantly stay at the leading edge of technology and analytical techniques while simultaneously grounding ourselves in the reality of businesses today. The journey has just begun the next 10-15 years will be incredibly exciting and the best among us will shape that future.