Challenges Faced by Organizations/CIOs in the Realm of Test Automation Date:   Tuesday , March 15, 2016 Headquartered in Bengaluru, Tyto Software provides business-ready, cutting-edge software automation products for web application testing, thus enabling organizations cut-down their process costs Test automation has considerably matured over the past decade. Most organizations have at least tried automating their tests. However the success rate of automation projects is still low. Challenges faced are in prioritization, team constitution, tool selection and development/test practices. Based on our over a decade long experience in helping automation teams, we have put together some challenges and suggested solutions. Prioritization of Automation: The priority of automation should be understood from the top management to the bottom tester. Automation yields results when it is frequently run. Allocation of resources, choice of tools, allocation of time in sprints, machines for playback, budget for test machines or cloud instances - all these need to be understood and planned for. Choosing the Right Team: Testers should understand the business of the product and be able to find problems that an end user may face. Having a very technical automation team may get scenarios automated but may not find business related problems. Similarly having a pure manual team which does not want to do any programming or scripting will prevent automation. The ideal would be a testing team that understands the business functionality very well. They also understand that automation can ease their work. They may not be very technical, but can write simple scripts or synthesize steps logically to create a functional flow. Given the right tools, such a team will be able to maintain quality of the product. The development team should be able to pitch in and help testers in the automation process. However, they should not be unnecessarily burdened and side-tracked because the chosen automation tool is too demanding. Testers lose a lot of respect and developers\' time because their automation tool demands that the tester ask the developers to add/modify ids to the developer code base. The right choice of tools should prevent such interactions. While hiring for automation, care should be taken to hire \'A\' grade testers and quality engineers and teach them scripting if needed, rather than hire B grade developers. This ensures a healthy attitude towards testing and quality and alignment of company/team goals with personal ambitions. Choosing the Tools: Whether the application has a GUI or not, developers should ensure that ample unit tests are written. This is easier said than done. Unit testing is a skill and developers may need to be taught to design code properly to allow unit testing. When dealing with GUIs, the right automation tool can make the testing process easy. Recorders automate the task of scenario flow creation. Tools help extract keywords, which can be fed into an inbuilt framework which helps the business testers to automate without delving too much into code. Reports are automatically generated and the tool can be integrated with Continuous Integration systems. The playback is fast and reliable as such a tool would allow test automation from day one and ensure steady progress. Often teams confuse test automation of their application with building a test automation tool which will solve all the world\'s problems. In the name of the oft abused word \'Framework\', a lot of effort is wasted even before the first automation script is run. One of the reasons for this is who decides on the automation tool. If it is left to a spare developer to choose, he will choose a tool where he can use his programming skills and build something from scratch. If it is left to the upper management who are not hands on, reporting may take precedence over tester usability. It is advisable to have multiple stake holders participate in the tool evaluation process. Once a tool is chosen, one may be stuck with it for a long time. Some parameters to look for in the automation tool are - ease of creation and maintenance of scripts, capability to support complex interactions and technologies, reliable playback without needing wait, fast playback, good resilience to GUI modifications, good reporting and over all ease of use. Automation Code Maintenance: Often we see teams whose scripts are stored in a shared network folder. Scripts are run periodically from the tester\'s machine and the results are reported to management. A better approach is to check in all automation scripts into version control systems and have a Continuous Integration system pull and execute tests. This ensures that multiple team members can work on the test code base and local changes do not affect automation. Version Control Systems are a must for automation too and not only scripts, but also the test data should be checked in. Frequent Execution: The surest way to kill an automation project is to not run tests for a few days. For frequent feedback, automated tests should be plugged into Continuous Integration systems. Provisioning of machines, virtualized instances or cloud instances may be needed for GUI tests. The person/team taking care of build processes should be able to handle such provisioning. Another aspect to consider is the set up of test data. Database with test data may be resurrected before each automated suite run. During a large suite playback, expect some random errors and failures. If possible create an automated failed suite and rerun it before publishing results. Some tools already are capable of auto rerun of failed scripts. This can save significant tester time. While it may appear daunting, belief in automation, adequate planning and use of good engineering practices can make test automation a success. If done correctly, it can make product delivery efficient, defect free and stress free.