Performance Testing has over the years become an area of specialization in software testing. With the right combination of test automation, analysis and debugging skills, it has become a niche giving testers more challenging and rewarding opportunities compared to several other test areas. Testers have over a period of time devised more complex performance tests including some of the forward looking ones such as capacity planning along with the ones that are hard to simulate such as long haul tests. Mobile devices thrown into the mix have brought in their own challenges on what portion of the testing should be done on them, and what can be done with a basic client-server setup. Undoubtedly, performance testing requires a lot of patience, an eye to catch defects which are not very straightforward and hands on experience that is built over time to understand the overall landscape.

While all of these together lie on one side, there is another aspect of performance testing which is probably not that difficult, which is very valuable in the results it could yield, but is still not often taken up by teams either due to lack of value awareness or lack of time – this is one, where core performance tests (such as response times / page load times) are run on competing products to understand how one’s product fare against them. While feature richness, time to market, brand value, product quality, marketing taken up, are all important in determining product acceptance amongst users, another very important aspect is product performance. Performance will be accounted for under the head of quality to a large extent, but several teams do not understand the importance of comparing their performance with that of competition – they need to understand that if a competitors page load time across say a heavy page is 1 second, while theirs is 3, over time, they would lose a significant user base primarily due to slowness in page load. These comparative tests don’t take much time and are not often apple to apple (because different products are being compared). But what can be done to bring in some level of consistency is to run them at specific times of the day, repeat tests over different time slots to compare results, take a similar page for comparison (e.g. a search results page) etc. Such actions will help them draw meaningful inferences and provide more data for understanding where product improvements are needed, if any. The tests by themselves do not take much time, but it is the comparison and analysis followed by any implementation changes that consume chunk of the time. However, the tester herein, collaborates with the development team to a very large extent to arrive at what changes need to be made (whether configuration level, database setting level or something more detailed at design or technology level). Such comparative testing is becoming an important part of the overall performance testing effort and if the test team voluntarily does this and takes back actionable results to the product team, it would be a very welcome change to help the entire team collaborate on a very important area of “product performance improvement in comparison to competition”

Security Testing has traditionally been a core testing type as a product gets tested and ready to be released. Test teams have typically worked with development, business and product groups to create a threat model analysis to understand what the product’s vulnerabilities are, what the threat entry points would likely be, what kind of data a hacker may be using to compromise the system, to determine what ethical hacking (aka security testing) they would need to undertake to make the product more rigid and hardened to handle real world security attacks.
At a basic level security testing is handled by the core functional team itself. Checklists and guidelines from organizations that promote web security, such as Open Web Application Security Project (OWASP) have proven invaluable to help in planning and implementing a security test effort. However, beyond the basic security testing, to take it up in further depth, be it pen testing, white hat testing, security test experts are sought after. Based on their tests they submit a report on the product’s security status, which almost is a certification for the organization that the product is secure for release to web (RTW). Typically, these specialist security teams could be a third party independent testing team or could be the organization’s resident specialist team. Regardless of who they are, they are often an on-call team, who evaluate the product for a specific period of time, submit their report and move on. Granted, they are very expensive resources, but the question to consider is whether it helps to have such a floating team take on this very important task of verifying the security of the system. There has been no dearth for web security breaches in the recent times. The most recent news is that of Target Corporation’s CEO resigning from his role, taking ethical responsibility for the compromise of 70 million user data records that happened late last year. While there is a useful blog that was posted recently on why security testing should become a part of the core testing effort to be contained within software releases, in my opinion, security testing spans across product releases. While the team plans and executes the security testing effort within a given release, the landscape is so vast and dynamic that it requires an on-going resident expert team that is watching for newer patters, hacks and trends in the world of software security, relating them to their product at hand and working with the developers to harden the system to handle them. Security testing thus becomes out of band in nature with a need for on-going monitoring, analysis and fixing and in this regard differentiates itself from other types of software testing.