Truviso: Next Generation Analytics in Action Date:   Friday , May 01, 2009 The Internet boom has put traditional data analytic solutions under severe pressure. The business intelligence (BI) approach, which was born in the mid-’70s, no longer scales in the 21st century when organizations are routinely dealing with terabytes of data per day. With volumes of data from user action, pages, ads, tweets, rich media interactions, and other sources mushrooming faster than the efficacy of available hardware or traditional business intelligence approaches, enterprises are hard-pressed to keep up with the flood of data. Trying to address the data analytics scalability problem by throwing hardware at it becomes a losing proposition due to spiraling costs in terms of servers, people, power, cooling, and space. Truviso is taking a different approach. Offering what it calls ‘continuous analytics’ of incoming data, a technique that enables massive scalability and extreme flexibility in data analysis. Co-founded in 2006 by Sailesh Krishnamurthy Ph.D. with UC Berkeley computer science Professor Michael J. Franklin, the U.S. based, California-headquartered Truviso solves the challenge of continuous data analysis for ‘always-on’, data intensive business environments. The company's software solution processes huge volumes of incoming data to enable continuous analysis, visibility, and action across heterogeneous business and IT systems. Truviso, with its next generation business intelligence solutions, changes the way companies’ data is processed, analyzed, and acted upon. Next-gen BI Solution Virtually every organization is facing rapid growth of data volume, with typical growth of data volumes being between 50 and 200 percent annually. More significantly, in network-centric areas such as media companies, social networks, content delivery, security, and others data can sometimes grow in excess of 1,000 percent a year. In conjunction with this unprecedented data volume growth, net-centric organizations that provide digital services across the Internet are under unrelenting pressure to continue making smarter, faster, and more efficient decisions. These companies, in an effort to stay ahead of their competition, are applying increasingly sophisticated analyses over detailed and complex information. All of this has created a ‘perfect storm’ in which traditional data analytics’ approaches to scalability and performance have become increasingly untenable. “Traditional business intelligence methods use an inefficient, sequential, batch approach – store the data first, then run queries for reporting and analysis. This is a Sisyphean task of pushing a large boulder up a hill even as the boulder is getting bigger, heavier, and more unwieldy,” explains Krishnamurthy. Truviso’s continuous analytics flips legacy BI architecture on its head, by having always-on queries continuously analyzing flowing data. This breakthrough solution unlocks actionable insights from dynamic data, resulting in several orders of magnitude in efficiency improvement. Built on core technology that can concurrently execute multiple queries simultaneously across distributed data sources, Truviso's product enables continuous analysis and proactive, data-driven actions with minimal disruption or performance impact on existing IT systems. As an example of the value that Truviso delivers, a top-ranking Internet portal deployed Truviso to process as many as 50 million inquiries (or transactions) per day, replacing a server farm they relied on to analyze the data. This server farm approach was expensive and inefficient, resulting in mounting operating costs and complexities. By incorporating Truviso’s continuous analytics solution into their IT landscape, the company was able to save time and money by getting continuous, proactive data analytics. At the same time, the company also was able to obtain meaningful, timely, relevant information on user traffic and operational activity levels, thereby getting a handle on their SLA, uptimes, and most importantly, user monetization opportunities. “While using BI to try and report on such problems is not new, what is unique about Truviso is that we are specifically architected for today’s always-on, massive data volume world. We see Truviso’s continuous analytics as a critical enabler for our customers to acquire, retain, and satisfy their customers and strategic partners,” states Krishnamurthy. Technology @ Work A significant challenge in commercializing a disruptive technology, such as continuous analytics, is to provide revolutionary performance in a familiar, evolutionary package. The Truviso approach is to permit customers to leverage skills and capabilities that they already have such as the SQL (the standard language for BI and analytics), Java, and Flex/Flash, ensuring that traditional database administration idioms, such as tables, views, and indexes, can be re-used. Truviso accomplishes this by producing a software product that builds on an open technology platform (database engine), making it compatible with traditional data-generating and data-consuming applications, then extending key Truviso intellectual IP to provide customers with game-changing analytic capabilities. This innovative approach enables the analysis of heterogeneous data regardless of whether the data is flowing, staged, or a combination of the two. In Truviso, queries are continuous and always running so new results are delivered on-demand whenever a downstream application or user requires them. Data does not need to be stored, so Truviso can keep up with enormous data volumes. Thousands of concurrent queries can be run continuously and concurrently on a single server, queries can be run over both real time and historical data, and incoming data can be optionally perused for replay, back testing, drill-down, or benchmarking. Truviso vs. Competition Until Truviso came on the scene, companies struggling with data volume issues tried to shoehorn traditional database and data warehouse approaches by throwing more and more hardware at the problem. “Most legacy vendors apply ‘brute force technology’; trying to keep-up with growing data volumes by adding more servers running traditional database engines, trying to extend and adopt architectures from two decades ago,” explains Krishnamurthy. A number of new players, in an attempt to handle the data explosion, are taking a different approach and are getting improvement by modifying some of the fundamental assumptions of how analytics engines work. The column-oriented database approach has generated significant amount of interest. There are some who apply custom hardware to key parts of the query-processing pipeline to make things go faster. Recently, there has been a rising interest in using more general-purpose parallel programming systems, largely from the open source community, as a platform for scaling-out analytics. An example is Apache Hadoop, an open source implementation of the MapReduce technique pioneered by Google. Unfortunately, the Hadoop-based solutions suffer from ease-of-use and compatibility issues that come from using an imperative programming model as opposed to the well-known approaches like SQL that are easier to develop, maintain, and extend. All of these alternative approaches require massive investment in what is fundamentally a decades-old approach and a constantly growing cluster of computing nodes to try and keep up with the avalanche of data – a losing proposition that increases the cost of operation and complexity of deployment. By contrast Truviso, having completely re-imagined business intelligence, applies continuous, stream-oriented query processing to data as it flows by in order to revolutionize the cost-benefit assumptions for analyzing massive amounts of data. Opportunities Abound The future seems to store a big promise for Truviso. UPS, a global logistics company and a user of the Truviso solution, saw such tremendous potential in it that it decided to make an investment in the company. Other strategic investors are the venture capital firms Diamondhead Ventures and ONSET Ventures. Truviso is working with a dozen customers that are mainly from the rich media, social networking, advertising, and content delivery businesses. In April, Truviso was recognized by InformationWeek as one of the ‘Top 50’ startups of 2009. The research firm IDC estimates the business intelligence market to be worth $13 billion, offering major growth opportunities for Truviso. The efforts pioneered by Prof Franklin’s team at Berkeley are radical and disruptive, with the potential to change the face of the business intelligence industry. What drives Krishnamurthy is the ability to make a meaningful impact on the industry at large, and success in the market is a sure-fire result of generating that impact. After all, the company was born out of his doctoral research on stream query processing while earning a Ph.D. in computer science from UC, Berkeley and Krishnamurthy is excited to see his ‘baby’ continue to deliver significant, differentiated business value to some of the most demanding customers on earth.