The Smart Techie was renamed Siliconindia India Edition starting Feb 2012 to continue the nearly two decade track record of excellence of our US edition.

April - 2008 - issue > Technology

Test Your Test Metrics

Venkat Ramana
Thursday, June 26, 2008
Venkat Ramana
It might sound obvious, but one of the most important tasks of the software quality assurance life cycle is collection and analysis of test results. Lacking metrics, the stake holders would have no idea if they were making progress in removing software defects.

Yet, as any experienced test specialist will testify, “Test metrics are similar to corporate balance sheets; what they show is very interesting, but what they hide is vital”. While collecting test metrics might be anything but simple, ensuring that you are collecting the right metrics and interpreting it correctly — could be even more challenging.

For instance, your team might say that all defects are logged with adequate comments, and that defect severity levels are trending down over time. Yet when the application is released to production, its performance proves dismal while users unexpectedly report many bugs. In effect, the surgery was successful, but the patient died.

Consequently, the burden is on the test manager to understand what test results reveal, and more importantly what they don’t reveal.

To provide an idea of what to look for, let’s review some examples on how commonly used metrics sometimes generate misleading results, and then we’ll discuss how to overcome these shortfalls.

Share on Twitter
Share on LinkedIn
Share on facebook