Often times when we talk about Test Automation in the Product Engineering jargon, one of the first questions from the stakeholders / project sponsors is “What is the Return on Investment”. In my opinion this is asked because Test Automation is an expensive proposition. Unless careful thought is given to what to automate, when to automate, how and how much to automate, the returns adversely impact the team in terms of time, cost and test coverage. So, it is rather important that the stakeholders ask this question upfront which makes the team think about the business and technical value from the test automation effort, present their data points to the team and get their buy in instead of diving into a random effort.
In this blog post, I want to share a few core areas of returns of investment whenever test automation is undertaken. Hopefully, this post will come in handy for anyone who is undertaking such an ROI analysis to ensure they have covered all areas of ROI.

Think about ROI from both tangible and intangible sources. Several people stop at the tangible side, which I agree is very important to show measurable returns to the management, but fail to understand that it is their responsibility to explain the intangible returns too.

On the tangible side typically, the returns focus on:

1. Time Savings – savings that can be quantified easily explaining how much time would be saved by having automated tests, which would otherwise have to be run manually

2. Cost Savings – measurable savings from reduced time to test, reduced number of manual testers needed to complete the job, reduced risk of untested areas and reduced risk of product release with undetected bugs

3. Extra test coverage – with the current development practices where there is a lot of system and service level communication, manual testing alone is not going to suffice to get the required test coverage. Areas such as web services, voluminous databases to be tested, complex performance test scenarios all are going to be impractical if not impossible to be tested manually. The extra test coverage that test automation delivers can be easily measured through instrumented means and code coverage techniques. Other areas typically considered for test automation include: regression tests, build verification/smoke tests, localization tests, hot fix / patch test suites

4. Risk Reduction – by closely working with the marketing and business team, the test team can determine the risks that test automation helps reduce by quantifying the impact of bugs found by automation. These are bugs which could have been found only by automation; showcasing the impact of such bugs going undetected, will really help the management appreciate the value of such automated tests. There is an element of intangible customer satisfaction that is involved here, but a rough measure of the impact is often good enough to explain the returns.

On the intangible side, it is important to consider the following while making a case to the management / stakeholder team:

1. Employee Satisfaction – Test automation often provides bigger and better opportunities to the testers on the job and makes the work more interesting and challenging, rather than being bogged by mundane manual tests. This is especially true when automation is taken up on tests that are repetitive in nature. While the satisfaction is often not measurable, this can be represented to the management team through creative ways such as “feedback from employees, satisfaction index that shows their job satisfaction levels before and after they were exposed to test automation, employee retention percentages that can be attributed to test automation opportunities etc.”

2. Creative, Out of Box Thinking amongst teams – Automation is not a very straight forward testing ammunition. One has to understand the business and technical sides of it to ensure its returns are justifiable. To be able to do this, one has to often think of creative workarounds, new frameworks that they may need to create, freeware/open source tools to leverage etc. Such creative scenarios can be a major morale boost making the job very rewarding for the tester. That said, the tester’s manager herein plays an important role in guiding them in the right direction to give ample scope for creativity yet ensuring the automation effort stays on track

When such intangible benefits are also considered in explaining the ROI on test automation, the management team will really appreciate the holistic thought that the team has provided increasing the chances of their support and buy in, for the overall test automation effort. Hope these come in handy in your teams in helping make a case for test automation and if you have additional points that you would like to share, I would be happy to hear from you. Until next time, happy testing!