IT’S THE ROAD NOT TAKEN THAT ALWAYS RANKLES a restless mind. Coulda…Shoulda…Woulda…If only we had a divine way to foresee the future! Talk to any CIO whose company was affected by the Nimda virus or the ‘Love Bug’, and he or she will certainly tell you this – ‘If only we had the foresight to spent the money to shore up our defenses.’

As things stand today, it is virtually impossible to get any project funded unless there is substantial return on investment (ROI). Sadly (for IT, that is) the technology arms race is over, and reality has finally caught up. Used to be that during the tech boom, IT projects somehow always managed to dodge the budget bullet, thanks to overzealous organizations pouring millions into questionable products with dubious payoffs. The glory days are over now, and the chickens have come home to roost. CFOs now take a jaundiced eye view of any and every technology initiative, scrutinizing every dollar spent and questioning the worth of every IT project.

So what’s a CIO to do? Quite simple - Demonstrate superior ROI for technology projects, and prove that technology investments have tangible, measurable, and sustainable returns. Nebulous gains such as ‘increased customer satisfaction’, and ‘better organizational flexibility’ should be translated into hard dollars. This alone will restore credibility to technology spending, and prove that technology initiatives are aligned with the overall business strategy.

A ‘ROSI’ Picture
A 2002 survey conducted by Computerworld magazine revealed that companies spent almost 11 percent of their total 2002 IT budget on security investments. Yet, it is not often easy to tie in revenue gains or productivity improvements to technology spending, simply because there is not often a first-order relationship between the two. Technology investments typically have a second or third order effect on the bottom line. To make matters even more difficult, returns on security investment (ROSI) are even harder to quantify, simply because it’s a pure guesstimate when dollar figures are used to quantify the cost associated with a publicized security intrusion and its adverse impact on corporate reputation.

There is no denying that a variety of ROI measures exist today. From EVA (Economic Value Added) to Balanced Scorecard, Applied Information Economics, Real Options Valuation, and Portfolio Management, every consulting company seems to have its favorite ROI measure, but none seem to accurately capture the bottomline impact of security spending. Instead, the Chief Security Officers (CSO) have relied all too often on the ‘FUD’ factor (Fear, Uncertainty, and Doubt), to scare the CFOs into submission. However, using such soft ROSIs to justify security investments may soon become the thing of the past, thanks to a bunch of researchers who found this challenge too hard to resist. Some of them have developed surprisingly robust methodologies of late, using scientific methods to calculate ROSI.

Utilizing risk analysis tools that are based on principles of probability theory and statistical analysis, these researchers have made significant progress in coming up with hard numbers to back up soft ROSIs. A study at Carnegie Mellon University’s (CMU) Software Engineering Institute researched and analyzed the ‘Survivability’ of networked systems to measure how survivability increases as security spending increases. Another study at the Center for Secure and Dependable Software at the University of Idaho attempted to gauge the cost-effectiveness of intrusion detection in detecting and responding to attacks. A third group of researchers at @Stake (a Cambridge, MA based security consulting firm) went about trying to prove what was intuitively obvious but never substantiated – that the earlier the security processes are built into the software engineering process, the higher the ROI.

CMU’s Survivabilty Experiment
The CMU team performed a regression analysis to create a curve showing the relationship between survivability and security spending. The data they used for the experiment was gathered by CMU between 1988 and 1995, and it essentially captured details of all voluntarily reported security breaches and threats. Sifting through the data, they studied a variety of factors such as the time of the attack, the frequency of attacks, the probability that an attack would occur, the damage caused by the attack etc. These details were then used to build an “attack engine” that struck a simulated networked enterprise on a scale and frequency similar to what was observed in the real world.

As they experimented with the variables, they observed how the network survived the attacks. In some simulations they armed the enterprise with iron-clad defenses which obviously carried a hefty bill. In other simulations, factors such as attack probability and extent of damage were varied. Ultimately the results were plotted on a graph, with the cost on the X-axis and survivability on the Y-axis. The resulting curve illustrated an interesting aspect about security spending. As spending increased so did survivability – up to a point. After this inflection, survivability levels off, revealing a diminishing return on investment.

Non Alcoholic ALE
The team at Idaho approached the question with a simple premise: If the cost of a tool that prevents an attack is known, and the dollar savings gained as a result of using the tool is also known, then the difference between the two is the ROSI.

To substantiate this premise with hard numbers, the team first calculated the value of the assets that it was trying to guard. Next, it figured out the Annual Loss Expectancy (ALE), which is the product of the loss due to an attack and the frequency of such an attack. For instance, if an attack happens every two years and wreaks $100,000 worth of damage, the ALE is $50,000. So if the annual cost to recover from all security incidents is $200,000, then the ROSI is simply $200,000 – ALE = $150,000.

The Earlier The Better
By treating a security hole as just another bug in the system, researchers at MIT, Stanford and @Stake calculated that the earlier the security details are built into the systems design, the greater the returns. They observed that incorporating security features at the design stage yielded a 21 percent ROSI. The same features implemented during development stage netted a 15 percent ROSI. The return falls to 12 percent at the testing stage.

Arguments Against Risk Analysis
All research efforts to pin down ROSI have their foundation in risk analysis. From identifying information assets and their vulnerabilities, to estimating the likelihood of an attack and the computation of annual loss expectancy, every step is a methodical exercise in risk analysis, which ultimately leads to a security plan. But despite its widespread usage in all spheres of business, risk analysis has its set of detractors who argue against its use.

First of all, it’s the inherent lack of precision that is cited as a significant weakness. According this argument, assumptions and approximations have a way of ‘fuzzying’ the math behind these calculations, and this is a flaw that could seriously skew any ultimate result.

A second arguments against risk analysis is that zeroing in on hard numbers may satisfy the bean counters, but it gives rise to a false sense of security. In other words, concluding that the ALE is $200,000 is no guarantee that the downside risk is capped at that level. It is only an ‘expectation’, as the name implies. But often, this fact is lost as soon as the ALE is used to calculate savings during a cost-benefit analysis.

The third argument is that in many organizations, risk analysis sometimes assumes the status of immutability, and this ultimately sows the seeds of its own downfall. Even though it is important to review and revise the previous years’ analyses, organizations tend to reuse dated data and figures instead of a performing a fresh annual exercise in risk estimation every year.

Guesstimates No More
It may be true that absolute precision is not yet a hallmark of the results of any of these risk analysis techniques. Nevertheless, the point of performing risk analysis is to get a handle on the magnitude of risk that an organization faces, not to forecast the exposure to the last penny. Security spending is often non-discretionary. Companies really do not have an option to forego essential security infrastructure. But in this era of belt-tightening, the onus is on the IT executives to prove that security spending is not an exercise in unbridled profligacy in the midst of a tight economy, but a sheer necessity in the face of networked reality. With rapidly maturing ROSI calculators, CIOs now have the tools to justify security spending using hard numbers, not smoke and mirrors.