In this information age, when product development is struggling to keep up with standards definition, one can visualize the challenges faced by development teams working with technology leaders. However, it takes hands-on experience to appreciate the uphill task faced by the system validation teams.

Engineers enjoy and take pride in ‘developing’ products. Try getting them to validate the product and you will hit a wall. Only the experienced would understand the importance and value of a good, proven test plan for the commercial success of the product in the dog-eat-dog market place. As the schedule of a project invariably continues to slip through the schedules set initially, the final delivery date remains fixed and the crunch is upon the system validation phase, it being the last. Unplanned patch releases only demand shorter time between releases, resulting in shorter test cycles. All this makes the system validation engineer wish, “If only the routine test execution were automated, life would be less of a struggle.”

Every high tech product that is developed today comes with innumerable user-configurable parameters and will mandate testing of products under all valid and invalid combinations of these parameters. Manually creating all these combinations for testing would be a grueling and time consuming task. On the other hand, repeating the same limited number of test cases again and again for regressions, while not really adding much to the test coverage, takes up valuable time and energy. Automated test suites offer themselves to easy extension by way of increasing these combinations with very little effort in a short while. They not only enable quick regression testing of products and new releases but allow unattended execution of most of the tests so they can be run overnight with results being available the next morning to be fed back to the development teams. In summary, test automation allows for faster testing of the product and more test coverage at a lower cost.

The activity of automating test cases logically follows at least one successful execution of the test case. Hence, it is by nature a repetitive job – ‘not challenging’, ‘not enjoyable’. Naturally, this task of test automation continually gets put aside and in reality never gets done. With the experienced validation team members being busy with first understanding the product and then conceiving the test strategies, defining test setups and plans, and later being overwhelmed with deriving and executing test cases, they think of automated test cases only when they have to execute them a second time. Basically, the validation engineer is asking for the test cases he or she had defined and proved earlier to be automated. The question then is, “Who should be automating the proven test cases?”

It should not be a surprise to engineering managers that more resources are required for proper on time validation of the product rather than development. However, anyone who has tried will tell you it is impossible to convince top management of this. Management’s priority is to control the cost impact and hiring full time employees exclusively for test automation is not an option. An alternative is to outsource this activity, but that would also be costly and is associated with a different set of issues like long term ownership and maintenance of the automated test suites.

The author is Director, Centillium Communications. He can be reached at djk@centillium.com