Time to shake up your obsolete performance test strategy
Performance test strategy has always been considered an important area of quality, despite being a non-functional area. Quality and the rest of the product teams have equally given importance to this space including setting aside the required budget, hiring the right talent, investing in tools, R&D, time to take up performance testing etc. However, the challenge today is that the product landscape is changing dramatically by the day – a lot of on demand testing services are in the mix, and unfortunately some pieces of performance testing are still stuck in time. On the bright side most teams have moved to leveraging the cloud as an IaaS to perform and provide load testing services. However, how much of that is on demand, load testing services is the question to consider – as in, how nimble are the scripts, how realistic are they to reflect true production like end user scenarios, what load testing monitoring is taken up post release, what is included in the scope of a performance test strategy are all questions that cannot be ignored.
I was reading a nice article this morning that talked about, how it is time to relook at what we cover in a performance test strategy and need to think beyond just concurrency testing. We also had a talk recently that highlighted the need for additional areas of test such as mobile device performance, besides just the core application or website’s load testing scenarios. This is a topic of ours, which was very well received at an internal conference and has also been chosen to be presented at STC QAI, a leading software testing conference globally and in India.
In summary, a performance test strategy needs to consider the following:
- Realistic usage patterns
- Live scenarios post release – module wise performance testing, ideally even performance testing at unit level to catch issues early on
- A nimble automated suite that can run on demand
- Cloud offerings to scale
- Core application performance parameters in compatibility with other system parameters such as device performance and also other quality attributes
- Live monitoring post release to look for not just functional feedback but also performance feedback and predictive analysis based on the same for future product recommendations
It is certainly time to review and shake up (if need be) the performance test strategy the testing community has been using all along.
