Test Automation Methodologies

The following is a description of two methods that have proven effective in implementing an Automated Testing Solution:

"Functional Decomposition" Method:
The main concept behind the "Functional Decomposition" script development methodology is to reduce all test cases to their most fundamental tasks, and write User-Defined Functions, Business Function Scripts, and "Sub-routine" or "Utility" Scripts which perform these tasks independently of one another. In general, these fundamental areas include:

Navigation (e.g. "Access Payment Screen from Main Menu")
Specific Business Function (e.g. "Post a Payment")
Data Verification (e.g. "Verify Payment Updates Current Balance")
Return (e.g. "Return to Main Menu")

In order to accomplish this, it is necessary to separate Data from Function. This allows an automated test script to be written for a Business Function, using data-files to provide the both the input and the expected-results verification. A hierarchical architecture is employed, using a structured or modular design.

The highest level is the Driver script, which is the engine of the test. The Driver begins a chain of calls to the lower level components of the test. Drivers may perform one or more test case scenarios by calling one or more Main scripts. The Main scripts contain the test case logic, calling the Business Function scripts necessary to do the application testing. All utility scripts and functions are called as needed by Drivers, Main, and Business Function scripts.

Driver Scripts: Perform initialization (if required), then call the Main Scripts in the desired order.
Main Scripts: Perform the application test case logic using Business Function Scripts.
Business Function Scripts: Perform specific Business Functions within the application.
Subroutine Scripts: Perform application specific tasks required by two or more Business Function scripts.
User-Defined Functions: General, Application-Specific, and Screen-Access Functions.

(Note that Functions can be called from any of the above script types.)

Processing Note:
If an error occurs in the processing, such that we cannot continue with the Test Case, a "FALSE" condition is returned to the calling script. This script, in turn, returns the "FALSE" condition to its calling script, etc., until control is returned back to the Driver script. If the Test Case dependencies have been properly controlled, the Driver can then continue with the next Test Case, otherwise the Driver would have to exit.

Advantages:
* Utilizing a modular design, and using files or records to both input and verify data, reduces redundancy and duplication of effort in creating automated test scripts.

* Scripts may be developed while application development is still in progress. If functionality changes, only the specific "Business Function" script needs to be updated.

* Since scripts are written to perform and test individual Business Functions, they can easily be combined in a "higher level" test script in order to accommodate complex test scenarios.

* Data input/output and expected results is stored as easily maintainable text records. The user's expected results are used for verification, which is a requirement for System Testing.

* Functions return "TRUE" or "FALSE" values to the calling script, rather than aborting, allowing for more effective error handling, and increasing the robustness of the test scripts. This, along with a well-designed "recovery" routine, enables "unattended" execution of test scripts.

Disadvantages:
* Requires proficiency in the tool's Scripting Language;

* Multiple data-files are required for each Test Case. There may be any number of data-inputs and verifications required, depending on how many different screens are accessed. This requires data-files to be kept in separate directories by Test Case.

* Tester must not only maintain the Detail Test Plan with specific data, but must also re-enter this data in the various data-files.

* If a simple "text editor" such as Notepad is used to create and maintain the data-files, careful attention must be paid to the format required by the scripts/functions that process the files, or processing-errors will occur.

• "Test Plan Driven" Method:
• The "Test Plan Driven" method preserves most of the advantages of the "Functional Decomposition" method, while eliminating most of the disadvantages. In this method, the entire testing process is data-driven, including functionality. The Detail Test Plan is written in a specific format, then saved in a particular record-format which the pre-written "Utility" scripts use to control the entire processing of the Automated Test.
• Architecture:
  The architecture of the "Test Plan Driven" method looks similar to that of the "Functional Decomposition" method, but in fact, they are substantially different:
• Driver Script:
  - Performs initialization (if required);
  - Calls the Application-Specific "Controller" Script, passing to it the file-name of the Test Case file (which has been saved in a "tab-delimited" format from the tester-generated Detail Test Plan spreadsheet);
• Controller Script:
  - Reads and processes the "tab-delimited" file, using a "switch-case" statement to break on each Key Word;
  - Builds a list of "processing actions" from the data within the rows / columns within the Key-Word group;
  - Calls a specific "Utility" script associated with the Key Word, passing to it the created "processing list";
• Utility Scripts:
  - Process input-list received from the "Controller" script;
  - Perform specific tasks (e.g. press a key or button, enter data, verify data, etc.), calling "User Defined Functions" if required;
  - Report any errors to a customized Test Report for the Test Case;
  - Return to "Controller" script;
• Functions:
  - General and Application-Specific functions may be called by any of the above script-types in order to perform specific tasks;
• Advantages:
  This method has most of the advantages of the "Functional Decomposition" method, as well as the following:
  * Detail Test Plan can be written in Spreadsheet format containing all input and verification data. Therefore the tester only needs to write this once, rather than, for example, writing it in Word, and then creating input and verification files as is required by the "Functional Decomposition" method.
• * Test Plan does not necessarily have to be written using a "spreadsheet" program. Any format can be used from which either "tab-delimited" or "comma-delimited" files can be saved (e.g. Access Database).
• * If "utility" scripts can be created by someone proficient in the tool's Scripting Language, prior to the Detail Test Plan being written, then the tester can use the tool immediately and not be required to learn the TSL Scripting language. The tester need only learn the "Key Words" required, and the specific format to use within the Test Plan.
• * If the Detail Test Plan already exists in some other format, it is not difficult to translate this into the "spreadsheet" format.
• * Since a number of "generic" Utility scripts have already been created, we can usually get the user up and running (for most applications) with this method within a few days, rather than weeks.
• Disadvantages:
• * Development of "customized" (Application-Specific) Functions and Utilities still requires proficiency in the tool's Scripting Language. Note that this is also true of the "Functional Decomposition" method, and, frankly of any method including "Record/Playback".
• * If application requires more than a few "customized" Utilities, this will require the tester to learn a number of "Key Words" and special formats. This is preferable, however, to the tester having to learn an entire programming language in order to utilize a Test Tool.
• * Spreadsheet Test Cases written in this manner are not re-usable. If, for example the "Post a Payment" Business Function changes, then all test scripts that require the "Post a Payment" sequence will have to be updated.