Test Automation is an integral part of any software development and sustenance project. However, research shows that 80% of test automation projects fail while the other 20% do not yield enough ROI. In this article, we share our experience of automating testing in several projects and how some of these best practices will make your next test automation project successful.
1. Treat Test Automation like a Software Development project:
We have seen that Test Automation projects have greater success if they are treated with the same discipline and rigour of software development. A successful testing automation project:
- Focuses on design (reusability, extensibility etc.)
- Takes a framework driven approach
- Has team members who are experts in writing code and not just testing or test automation tools.
- Has a clear cut development and governance plan
- Maintains detailed documentation
- Prepares a plan to track bugs and rectify them
- Identifies reusable functions in automation scripts
2. Set measurable goals for your Test Automation project:
It is essential to set clear, quantifiable objectives for the test automation project. These goals should clearly bring out:
- Percentage of test cases to be automated in various Test Suites
- % Improvement in the test coverage
- % reduction in the time to release the build
- Reduction of test cycle time of new releases by improving their time-to-market (management-level goal)
For testing metrics like those above, tools such as Automated Execution Efficiency Ratio (AEER) could be used.
3. Choose a testing tool that answers your Automation needs:
Selecting the right tools is obviously a very important foundation for success in your Test Automation project. We suggest employing the following criteria in selecting the right test tool:
- The tool must posses a built- in facility to run multiple test cases
- The tool should be able to classify tests being run, basing on set criteria while simultaneously running multiple test cases
- The tool should be able to run the desired test cases without human intervention
- The tool should be able to generate execution reports or logs at the end of each test case
- The tool should be able to compare two values, so that the difference between real and expected results can be estimated.
4. Get Your Design of Test Automation Framework Right
Test Automation framework allows us to define the scripting methodology to automate tests, creates mechanism to hook into the application under test, execute the tests and report the results. Framework establishes a common standard for the team to achieve the goals.
While designing a test automation framework, the key goal is maintainability. The tests will change and good automation framework will support and facilitate change. If there is any change in any test, it should be very easy to change the automation suite, and in some cases not required to change at all. In addition, performance is also important. In a typical automated test suite, several tests will be executed. If each test can be executed faster by a second or so, it will matter! Ultimately we are talking about cutting down the time to test!
The following list will serve as a checklist (not exhaustive) to design your test automation framework:
- Define your scripting standards for business components, system communications, data check points, loggers, reporters etc.
- Hide the complexities and critical aspects of the code from Test engineers and expose them to the implemented Libraries and tests.
- Separate Scripts and Data – automated tests scripts must be separated from the input data store (could be in text, XML or Excel files). Ideally only input data gets manipulated for testing with multiple values and the script does not change.
- Think through the applications future enhancements and modifications before building your test framework library, and build in the extensibility in it.
- Use the standard process for script versioning.
- Your framework will have components/libraries to handle
- Application Functionality
- Types and sources of Input data
- Communication between utilities, components and systems such as communication between distributed components, check points, communication with logger etc
- Tool Extending capabilities – features which are not supported by the tool that you have selected.
5. Ensure high quality of Test Data:
A good investment of time and effort should go into creating well-structured test data during the early stages of the automation testing project. This makes the process of writing automated tests easier, in the later stages of the project life-cycle. The quality of test data also determines its reusability and maintainability.
6. Build the right team and invest in training them
Apart from the right tools and a proven testing process, a testing automation project requires a dedicated, and skilled team who are trained to understand the complexity of the project and work accordingly . The challenge in such scenarios is to find skilled, dynamic personnel who should be trained to:
- Handle ambiguous scenarios and test cases
- Conduct manual testing along with developers
- Write automated regression tests
- Execute automated regression packages.
- Handle integration and performance testing
- Plan and gather requirements
7. Conduct early and frequent tests
The more frequent and earlier the testing automation process is applied in the project life-cycle, the earlier are the bugs detected in the process. The earlier the bugs are identified, the more economical it is to remove them from the system at a preliminary stage. This process also helps refine the automated test suite for further enhancements.
