Have you ever spent weeks or months building an automation framework with solid coverage, helpful for regression tests, and integration with CI? But when it comes to the execution phase, nobody runs it and wraps up the test cases with manual execution.
Here comes the question, why??
Constant Issues while Running Tests
If you have ever worked as a QA, then you can relate to the below pointers.
- “CI test run has failed and debugging the logs to figure out the failure is too time-consuming.”
- “When you require to just run one test case but still have to set up data, environment, and trigger manually”
- “Spent efforts and time in implementing automated test cases, but only using it during regression purposes?
- “Updating the test data for the automated test cases and data set up can be frustrating and time-consuming”
- “Few test cases may work fine and pass when run locally but fail when run in pipeline”
- “When the person handling the pipelines and scripts is absent, the manual team may find it hard to comprehend and run scripts”
Are you finding it similar??
While automating the test cases is supposed to speed up the execution process, reduce manual efforts, and improve software quality, it can be too exhausting which leads to avoiding it.
At some point, automated testing can be frustrating and it slows everything down. Here we often tend to:
- Test the functionalities manually to wrap up the test cases fast.
- Pass the test cases by manually intervening in the script run wherever required.
- Ignore the failure reports without proper analysis and blame it on data or environmental issues.
Why Do Automated Tests Get Ignored?
There are three key reasons why automated test suites are being avoided. They are as follows.
1. Running Tests can be Annoying
Executing a test case manually can be very easy even if it takes some time. You just have to follow the test steps, take few screenshots, document it and upload the document. Then it’s done.
When we compare the same with executing in automation:
- Check if the environment is set up properly.
- Handle the loading time of the UI.
- Debug the test cases that fail randomly and are not replicable.
- Constant efforts in fixing the scripts whenever we integrate the code for a new feature.
It is basic human tendency to avoid the complicated things and take up an easy route. In that case, it means we skip automation.
2. Test Failures in Pipeline
Analyzing the CI logs and finding out nothing that explains the failure? You have to go through hundreds of lines of logs, trying to figure out what happened. By the time you figure out the issue, it already would have taken up your time and you again have to work on fixing it without impacting the rest of the code.
It’s frustrating and that is why many teams don’t bother looking into the failures until there is a blocker. They just rerun the tests, hoping it will work in the next run.
3. Automation ≠ Usability
Building an automation framework is totally different from making it usable during execution.
If only one person on the team knows how to trigger tests, debug failures, or set up test data, the whole system falls apart in the absence of that person. The automation test suite should be an asset to the team, not a burden. But when it’s complex, slow, and frustrating, nobody wants to deal with it.
Automation does require EFFORTS.
We always discuss how benefitting the automation scripts can be like:
- Identifying bugs early
- Improve the reliability and quality of software
- Save manual efforts and complete release cycles on time
It is theoretically true. However, we often fail to discuss how difficult can be to run and maintain the scripts. This on the contrary takes up time and can be frustrating. Hence most of the automated frameworks are unused and do not have any proper maintenance.
Conclusion
We all would have an impression that automating test cases would make things easier: faster releases; fewer bugs; less manual work. But if you ask any QA team regarding this, they will agree that:
- They have tests, but no one executes them.
- Executing automated test cases is exhausting.
- People don’t run tests until something breaks.
- Debugging a failed test case is a worst nightmare.
How is something that is supposed to save us time becoming another hurdle for us? If a test is automated, why is it not simple and straightforward? Executing automated testcases seem to take more effort of yours than manual execution. This isn’t the right way. The real value of test automation is not just having tests, but assuring they are actually used. Test automation should be for you, not against you. Surely there is a better way!!