Head: Peering Over the Event Horizon

Author: Ram Menon
Executive Vice President, Worldwide Marketing, TIBCO
As human beings we intuitively understand the importance of events. But in an IT environment, understanding business events – and analysing what actions should be taken as a result of a certain train of events occurring – is a new development. The logical pathways which lead an application to interpret meaning are still being formed, and the understanding of how certain activities should ensue is growing.

A more complete understanding of the event stream comes at the intersection of the real-time event-driven architecture and the traditional world of business intelligence. It’s a border between two territories that we are only just beginning to cross, but it’s clear that business intelligence is going to play a key role in the predictive business. And it requires a completeness of vision and a level of usability that few solutions offer today.

So while event processing vendors have concentrated on looking for recognized patterns in the real-time event stream, the BI vendors’ approach tends to focus on looking at historical data, which has often been likened to driving a car by looking in the rear-view mirror. Bring those two worlds together though, and you have a powerful new capacity for exploratory analysis in real-time. And make that functionality easy enough for business users to pick up and play with and for perhaps the first time, you are handing the tools to the people who really understand what’s happening, can react accordingly and plan for what may happen next.

Traditional BI products have often been used as a data source, with business users running reports, then dumping the data into Excel for further analysis in an environment they feel comfortable with. Once business people are given an intuitive environment that quickly spots patterns and relationships, and learn more about the application they want to monitor, then they will be in a position to interact directly with the data and build the rules themselves.

The next generation of business management must focus on this vision of being able to react (or predict), not just to today’s pressures but to understand what’s coming next and plan accordingly. There are some fundamental flaws in the approaches businesses are using to build applications that spot patterns in the events around and activities of their customers, employees and suppliers. These come firstly because of their inability to combine the right historical data with real-time events and secondly because most BI tools lack the rich and rapid discovery capabilities needed by business users.

Next-generation business intelligence sits well with the power to predict because it can improve the quality and richness of what ought to be monitored, integrated and connected, and the patterns IT and business systems should be looking for. And when particular conditions show that things are happening, it can be used to discover and diagnose why they occurred and offer a path to corrective action. Used correctly, it can increase our understanding of what has happened, then feed back into the model to form a virtuous circle.

Fighting fraud

Take the example of a fraud application for a bank. An event-driven approach today will monitor activity and if it spots a set of activities that matches a set pattern of fraudulent activity, will kick off an exception process to go and manually discover whether in fact the activity was in some way illegal. The trouble is that fraudsters will quickly change their patterns of operating to stay ahead of this kind of detection. And as fraudulent activity increases, so does the number of instances that require manually sorting at the end of the trading period. This quickly becomes prohibitive.
With a much deeper level of functionality, an application can go one step further. But that requires:

* Business intelligence that takes data and maps different relationships to spot particular scenarios
* Bn integration infrastructure where multiple sources of data can be pulled together, including traditional data warehouses
* A complex event processing architecture to look for sophisticated patterns of activities
* And business process management to kick off both machine and human oriented exception processes
* Rich analytics capabilities to empower end users to ask and answer any question.

Business intelligence plays a vital role in generating discoveries in information, new insights, opportunities and risks in various data sources, then subsequently determining which of those models appeared to be of value and, downstream of the application, pooling all of the data together from multiple sources to see what has actually happened.

Analytic applications are also able to determine different scenarios that are worth modelling and by integrating with a lot of different data sources come up with a set of hypotheses that look for relationships between data and the different patterns that occur that might indicate fraudulent activity. That analysis can be fed into a series of rules that are running on a product line which are monitored as trading activity happens. Then when certain conditions are met, the system can trigger a number of exception processes. One of these processes might initiate a series of analysis sessions that look at particu3lar situations to determine whether they were actually fraudulent and further inform the model.

Intelligent business

The trouble is that traditional business intelligence applications are rarely up to the job. They tend to rely on data warehouses or data cubes, and routines run nightly to crunch data so that systems can easily access it in a fast, supposedly on-line manner in preformatted reports.

That’s fine in a production reporting environment, where they allow users to drill into a particular path in the data, following logical drill down paths. Using a sales example, the user might want to look at national sales, then regional, territory, rep sales, and finally drill into an individual’s performance. But at any point in that analysis if the user wanted to know what was affecting sales and look at other variables that weren’t prepared for them, they would need to go back to the IT department and ask them to add some data for analysis or reconfigure a new data cube for them. In which case the user is held up for hours, if not days or weeks.

Traditional BI systems are architected this way because they assume that users are interested in a certain set of predefined variables and are looking at known issues and questions. But take this into a more complex environment, and users don’t necessarily know the questions they want to ask, nor the data sources they want to interrogate. They want to be able to pull different data sources into their analytic application on the fly and easily visualise the data that the system is presenting to them. And, needless to say, they want to do it fast. In addition, the predictive business wants to put these tools into the hands of business users in their day-to-day work without having recourse to the IT department whenever they want to add a new data source.

We’re entering a new era of pervasive business intelligence which will see BI everywhere as it relates to users – with experiences as simple and intuitive as using Google Earth or iTunes – and BI everywhere as it is built into processes, systems and applications. That’s a powerful combination.

Previous  article
Next article
Write your comment now

Email    Password: 
Don't have SiliconIndia account? Sign up    Forgot your password? Reset
Reader's comments(5)
1: From: Mrs. Mary David

This mail may be a surprise to you because you did not give me the permission to do so and neither do you know me but before I tell you about myself I want you to please forgive me for sending this mail without your permission. I am writing this letter in confidence believing that if it is the will of God for you to help me and my family, God almighty will bless and reward you abundantly. I need an honest and trust worthy person like you to entrust this huge transfer project unto.

My name is Mrs. Mary David, The Branch Manager of a Financial Institution. I am a Ghanaian married with 3 kids. I am writing to solicit your assistance in the transfer of US$7,500,000.00 Dollars. This fund is the excess of what my branch in which I am the manager made as profit last year (i.e. 2010 financial year). I have already submitted an annual report for that year to my head office in Accra-Ghana as I have watched with keen interest as they will never know of this excess. I have since, placed this amount of US$7,500,000.00 Dollars on an Escrow Coded account without a beneficiary (Anonymous) to avoid trace.

As an officer of the bank, I cannot be directly connected to this money thus I am impelled to request for your assistance to receive this money into your bank account on my behalf. I agree that 40% of this money will be for you as a foreign partner, in respect to the provision of a foreign account, and 60% would be for me. I do need to stress that there are practically no risk involved in this. It's going to be a bank-to-bank transfer. All I need from you is to stand as the original depositor of this fund so that the fund can be transferred to your account.

If you accept this offer, I will appreciate your timely response to me. This is why and only reason why I contacted you, I am willing to go into partnership investment with you owing to your wealth of experience, So please if you are interested to assist on this venture kindly contact me back for a brief discussion on how to proceed.

All correspondence must be via my private E-mail ( for obvious security reasons.

Best regards,
Mrs. Mary David.
Posted by: mary lovely david - Monday 26th, September 2011
My Name is Tata I was impressed when i saw your profile at and will like you to email me back to my inbox so that i can send you my picture for you to know who i am.i belive we can establishe a long lasting relation ship with you.In addition,i will like you to reply me through my private e mail box (
This is because i dont know the possibilities of
remainning in forum for a long time.
Thanks,waiting to hear from you soonest.
Posted by: tata tatababy os - Friday 30th, October 2009
3: The current economic slowdown has opened the eyes of analysts that policies and strategies for a long run is essential than for a short term.we should be well prepared to face this kind of situations in future if it ever occurs.what is the next best option to revive this? what measures have the govt taken?
Posted by: Sri sai - Saturday 28th, March 2009
4: "When a man dies, if he can pass enthusiasm along to his children, he has left them an estate of incalculable value."
Posted by: shakti mahapatra - Sunday 02nd, November 2008
5: Business intellligence is a great tool in the path of business development. In this centuary listening is an important part in the business process. We have to sell or promote only the useful or tailor made products or concepts.
close to the customer; open your eyes; go deep in to the ocean; and in business process nothing is predictive
Posted by: gireesankochi varma - Thursday 07th, August 2008
More articles
by Kaushal Mehta - Founder & CEO, Motif Inc..
The retail industry is witnessing an increased migration of customers from traditional brick and mortar retail to E-commerce (online retail)...more>>
by Samir Shah - CEO, Zephyr .
You probably do because you are on the phone with them! For all of you working in some technical management capacity here in Silicon Valley,...more>>
by Raj Karamchedu - Chief Operating Officer, Legend Silicon .
These days are a mixed bag for me. Of late I have been considering "doing something bigger and better," in my life, perhaps seriously though...more>>
by Madhavi Vuppalapati - CEO of Prithvi Information Solutions .
IT Services Rise of Tier II companies The Indian IT outsourcing industry is going through very exciting phase in its business life...more>>
by Bhaskar Bakthavatsalu- Country Manager, India and SAARC of Check Point Software Technologies.
Data loss occurs every day through corporate email. In fact, given the sheer number of emails an organization sends every day, data loss inc...more>>