Automated Testing: Process, Planning, Tool Selection

Manual testing is performed by a human sitting in front of a computer carefully executing the test steps. Automation Testing means using an automation tool to execute your test case suite. The automation software can also enter test data into the System Under Test, compare expected and actual results, and generate detailed test reports.

Test Automation demands considerable investments of money and resources. Successive development cycles will require execution of the same test suite repeatedly. Using a test automation tool, it's possible to record this test suite and re-play it as required. Once the test suite is automated, no human intervention is required. This improves the ROI of Test Automation.

The goal of Automation is to reduce the number of test cases to be run manually and not eliminate manual testing altogether.

Why Automated Testing?

Automated testing is important due to the following reasons:

• Manual Testing of all workflows, all fields, all negative scenarios is time and cost consuming.
• It is difficult to test for multilingual sites manually.
• Automation does not require Human intervention. You can run automated tests unattended (overnight).
• Automation increases speed of test execution.
• Automation helps increase Test Coverage.
• Manual Testing can become boring and hence error-prone.

Which Test Cases to Automate?

Test cases to be automated can be selected using the following criteria to increase the automation ROI:

• High Risk - Business Critical test cases.
• Test cases that are executed repeatedly.
• Test cases that are very tedious or difficult to perform manually.
• Test cases which are time-consuming.

The following category of test cases are not suitable for automation:

• Test Cases that are newly designed and not executed manually at least once.
• Test Cases for which the requirements are changing frequently.
• Test cases which are executed on an ad-hoc basis.

Test Tool Selection

Test Tool selection largely depends on the technology the Application Under Test is built on. For instance, QTP does not support Informatica. So, QTP cannot be used for testing Informatica applications. It's a good idea to conduct Proof of Concept of Tool on AUT.

Define the Scope of Automation

Scope of automation is the area of your Application Under Test which will be automated. Following points help determine scope:

• Features that are important for the business.
• Scenarios which have a large amount of data.

Planning, Design and Development

During this phase, you create an Automation strategy & plan, which contains the following details:

• Automation tools selected.
• Framework design and its features.
• In-Scope and Out-of-scope items of automation.
• Schedule and Timeline of scripting and execution.
• Deliverables of automation testing.

Test Execution

Automation Scripts are executed during this phase. The scripts need input test data before they are set to run. Once executed, they provide detailed test reports. Execution can be performed using the automation tool directly or through the Test Management tool which will invoke the automation tool.

Example: Quality Center is the Test Management tool which in turn will invoke QTP for execution of automation scripts. Scripts can be executed on a single machine or a group of machines. The execution can be done during the night, to save time.

Maintenance

As new functionalities are added to the System Under Test with successive cycles, Automation Scripts need to be added, reviewed, and maintained for each release cycle. Maintenance becomes necessary to improve the effectiveness of Automation Scripts.

Automation Tools

Following are the most popular test tools:

• QTP: HP's Quick Test Professional (now known as HP Functional Test) is the market leader in Functional Testing Tool. The tool supports a plethora of environments including SAP, Java, Delphi amongst others. QTP can be used in conjunction with Quality Center, which is a comprehensive Test Management Tool.
• Rational Robot: An IBM tool used to automate regression, functional, and configuration tests for client-server, e-commerce, as well as ERP applications. It can be used with Rational Test Manager which aids in Test Management Activities.
• Selenium: An open-source Web Automation Tool that supports all types of web browsers. Despite being open-source, it's actively developed and supported.

How to Choose an Automation Tool?

Selecting the right tool can be a tricky task. The following criteria will help you select the best tool for your requirement:

• Easy to debug the automation software scripts.
• Ability to recognize objects in any environment.
• Extensive test reports and results.
• Minimize training costs of selected tools.

Tool selection is one of the biggest challenges to be tackled before going for automation. First, identify the requirements, explore various tools and their capabilities, set the expectations from the tool, and go for a Proof Of Concept.

Framework in Automation

A framework is a set of automation guidelines which help in:

• Maintaining consistency of Testing.
• Improves test structuring.
• Minimum usage of code.
• Less Maintenance of code.
• Improve reusability.
• Non-Technical testers can be involved in code.
• Training period of using the tool can be reduced.
• Involves Data wherever appropriate.

There are four types of framework used in software automation testing:

1. Data Driven Automation Framework
2. Keyword Driven Automation Framework
3. Modular Automation Framework
4. Hybrid Automation Framework

Automation Best Practices:

• Scope of Automation needs to be determined in detail before the start of the project. This sets expectations from Automation right.
• Select the right automation tool: A tool must not be selected based on its popularity but its fit to the automation requirements.
• Choose appropriate framework.
• Scripting Standards: Standards have to be followed while writing the scripts for Automation. Some of them are:
  • Create uniform scripts, comments, and indentation of the code.
  • Adequate Exception handling - How errors are handled on system failure or unexpected behavior of the application. User-defined messages should be coded or standardized for Error Logging for testers to understand.
• Measure metrics: Success of automation cannot be determined by comparing manual effort with automation effort but by also capturing the following metrics:
  • Percent of defects found.
  • Time required for automation testing for each and every release cycle.
  • Minimal Time taken for release.
  • Customer satisfaction Index.
  • Productivity improvement.

The above guidelines, if observed, can greatly help in making your automation successful.

Benefits of Automated Testing

Following are benefits of automated testing:

• 70% faster than manual testing.
• Wider test coverage of application features.
• Reliable in results.
• Human Intervention is not required while execution.
• Test Frequently and thoroughly.
• Ensure Consistency.
• Increases Efficiency.
• Improves accuracy.
• Saves Time and Cost.
• More cycles of execution can be achieved through automation.
• Early time to market.

Conclusion

Right selection of automation tool, testing process, and team are important players for automation to be successful. Manual and automation methods go hand-in-hand for successful testing.