Software Testing - A Perspective on Pricing Models Date:   Monday , January 02, 2012 The planning of a software project begins with identifying the customer requirements. Then we determine the features to meet the requirements. The development team plans to design the coding architecture for the same and testing team starts preparing the Feature test plan (FTP). FTP is then further divided into several test cases which will be ultimately executed to get all those features tested. Effort is estimated for testing and then a testing team consisting of testers and a lead is created to get the testing work done. This drilling down from requirements to testers is shown in the figure below. The software testing project can be priced at any point on this ladder. Let us discuss the same one by one. Pricing by 'Testers' In this method, a testing team is formed with a set of testers and is blocked for client’s requirement. The client pays on the basis of duration of blocking and the no. of team members. This is the traditional Time and Material (T&M) model. For example, if the testing team is kept for four months and consists of eight testers at the cost of $4000 per tester per month, then the client has to pay $4000 * 8 * 4 = $128000. Client may insist on differential pricing based on the expertise of the testers or experience. The Testing organization has the least risk in this method from a revenue perspective. At the same time, there is little incentive to perform better. Irrespective of the performance, money is paid. Also, there is little incentive for innovation. On the other hand, because of differential pricing, when more novice testers are present in the team the revenue may become less. At the same time, the client has his cost unpredictable since he has to keep the team till the testing exercise is completed. Testing being a downstream activity, its schedule is dependent on other upstream activities and may slip. He also does not have any control on the output of testing activity since what the vendor assures is keeping of the testing team. Any cost due to delay in the starting of testing activity is passed on to the client. To have a hold on the project, client has to get into lots of reviews with the testing team throughout the project duration. As seen above, this method has more disadvantages than advantages for both the parties. Pricing by 'Test effort' Here organization charges the client based on the effort estimated for testing phase. This is the traditional ‘Fixed Price Model’. The main difference this method has from the ‘Pricing by Tester’ model is that here the charge is related only to the effort put in for testing and not on how many days or weeks the testing team is kept. So client is not required to pay for any delay or for just keeping an idle team. For example, if the test effort is calculated as 25 Person Months of effort and the Testing Organization charges at $4000 per Person Month, then the total bill will be $100000. Since the client is paying for a predetermined effort, the organization has some incentive for devising methods to reduce the actual effort and make the revenue nonlinear and also be innovative. The revenue is no more expertise linked and hence the team can consists of new talent as well. The organization also has the flexibility in moving the team members around as the situation demands. Here the client is no more paying for an idle team and is only paying for the team’s effort. But, there is still an uncertainty on the output given by the team as his bill is proportional to the effort and not the output. Pricing by 'Test cases' Test cases are the most important and objective measure of a testing team’s work. Test cases can be classified into low, medium and high complexity. Depends on how many in each category a team executes, the price is calculated and charged to the client. Here the client gets some sense of the team’s internal activities and a relation is established between that activity and the price he pays. For example, assume that the product has 500 low complex test cases, 1000 medium complex test cases and 750 high complex test cases. Also assume that the execution of Low complex test cases are charged @ $20, Medium complex test cases @ $30 and high complex test cases @ $40. Then the total price will be $70000. Since the client pays at the rate of test cases, the organization has incentive to innovate to devise methods to execute test cases faster and thus spend less effort per test case. Still the revenue is the same. At the same time, team can find new test cases to thoroughly test each of the features and thus earn more revenue as well provided it is approved by the client. In this method, client has got more insight to the testing team’s activities. The output certainty increases as he pays for the team’s output i.e. no. of test cases executed. Nevertheless, one problem still remains - he does not have any guarantee of a defect-free feature as the pricing is just on the test cases. Pricing by 'Features' This is the next level of Output based pricing. Disregarding the no. of testers, effort put in or test cases used, this method suggests that the payment has to be based on no. of features thoroughly tested and gone to the field without any defects. This gives the testing organization a good incentive for devising good testing methods, methodologies and techniques and making sure that each feature gone to the field is without defects. The organization is forced to capture all the defects very early in the life cycle and thus avoid more effort and also bad reputation. The client’s risk is very less here. His cost is almost certain and leaves the rest to the vendor organization. Also, the coordination between Development team and Testing team will be more in this method as both teams work together to reduce the defects going to the field. Pricing by 'Requirement' Here the entire testing is a black box for the client and his only demand is to test the product till design and testing teams agree that it is defect free and deployment ready. Prioritization of features and test cases will be the job of the vendor organization. This method is the riskiest from testing organization point of view; but brings most of the opportunities too - opportunity to innovate maximum, independent planning and execution as far as the same is within the target dates of product release. If carefully executed, this method brings maximum profit for the testing organization. From client’s perspective, this is the least risky method. Cost certainty is more which helps him in his overall plan. Client is also guaranteed a zero-defect product at the end of testing. Tough competition between different Testing players in the software industry has already pushed the prices down. The testing industry has to innovate in the pricing models as well to keep the innovation going in the testing execution phase.