Continuous integration, deployment/delivery, testing are all terms in much demand and usage in the software development industry today. There are a bunch of groups that have adapted to the needs of the “continuous” world fairly well and are mature in their overall delivery process. There are a few others who are in the process of getting there or are still over-whelmed by what it takes to get there. As testing services providers including test automation across all testing attributes most questions that we see are around the following:
- What does it take to move into the world of continuous testing with core functional test automation and what happens to the manual test efforts?
- What about continuous testing of other attributes such as performance, security, accessibility etc.
Let’s look at both one by one. For an organization that is fairly new to test automation and has largely been focusing on its test efforts only manually, the first step is to build a robust stand-alone test automation suite. This is a suite that includes the overall test automation scenarios based on what percentage the organization’s test strategy has been finalized on. Within such a suite, the right prioritization around smoke/sanity, regression etc. will be identified. The goal here is two-fold – build a very robust automation suite on core automation best practices making maintenance effort as low as possible and secondly bundling them in the most effective way so they can be run in an unattended manner in the continuous integration world. While organizations may choose to do these in an “n-1” sprint sequence, the whole idea is to very soon get into an “in sprint” automation cycle to handle the automation engineering and execution within the current sprint. Just as how the whole idea of continuous integration is to bring in as much agility and granularity as possible to catch issues early on when they are easy to find and cheap to fix, the testing scope and ownership should also be broken down into pieces where the testers are able to bring in self-discipline once the core goals have been identified across the test team. Once this is done, the tester is also able to determine the extra coverage on top of the automation suite to handle manually – this also calls for all testers especially manual testers, to gain proficiency in the space of automation, especially with how easy automation tools and frameworks have evolved to be enabling everyone take up automation. Today the key to a successful tester is not how good a manual tester you are, but how good a balanced tester you have become, able to manage both automated tests and take up creative user centric flows manually.
The other area companies are pondering with, is building a reliable automation suite for non-functional workflows to also fit into the CI/CD space. This is also not rocket science. At the end of the day it is all about how you have made your choices – your choice around which are the core tests you want included in the regression suite since not every test needs to be run with every build (from cost of run and time of run standpoints), how reliable the regression suite is and which are the ones that can be covered in a non-regression cycle. Once these are determined it is a matter of integrating the select tests from the standalone suite into the integrated suite.
At the end of the day, CI/CD driven test automation calls for as much discipline and awareness amongst testers, as it does for developers – once this has been understand and applied, adapting is just a process that follows.