Software as a Service (SaaS) has been an evolving trend we’ve been seeing over the last 2 years or so. Its increasing popularity can be attributed to several reasons such as: evolving cloud technology, need for optimization of resources and on-demand usage, global expansion of business that necessitates a platform to bring its people together, to name a few.
In this blog, I will be talking about how testing for SaaS applications is different from testing a traditional on premise implementation or rather what more to additionally focus on, in SaaS applications. To do this, let us look at a simple definition of SaaS and derive testable elements from it.
SaaS is a software delivery model, where the software is made available to the end customers over the Internet. The software vendor hosts the software for the customer, without the customer having to build up a hardware or software environment at their end. The customer herein, just provides software customization requirements to the vendor and follows a “pay per use model” to enable optimum use of resources. SaaS offers the customer the flexibility to scale up or down the resource usage on an “on-demand” basis. Depending on the SaaS implementation model, there are varying levels of SaaS model maturity ranging all the way from:
- Single instance – single tenant (customer)
- Customizable multi tenancy on the base source code
- Multi tenancy on the base source code without any customizations
Let us now draw the testable elements from this definition:
Take away #1 from definition: Software made available to end customers over the Internet: Software has come to enable us carry on a zillion different activities all the way from day to day shopping, banking, networking to possibly onetime events like home purchase, finding a match for yourself etc. If you are going to do all of this over the Internet, what is the first concern that comes to your mind? Privacy, confidentiality, security. Are all my transactions going to be secure especially when multiple other tenants are using the same instance of the software? Security testing, especially, web application security testing is very important for SaaS applications. Take the initiative of doing the threat model analysis, or if your development team owns it, actively participate to ensure you understand the potential threats so you can test them to ensure they are mitigated.
Take away #2 from definition: The software vendor hosts the software for the customer: Given the scale of operations involved in the SaaS model, performance is something to definitely plan for in advance and test. This is something that the end customers have come to expect, however, this can become challenging for the vendor when multiple instances are being planned for. Discuss the performance requirements with the business or product planning team upfront and start this testing early on in the product life cycle to conduct incremental tests. Besides just performance testing, scalability and capacity planning testing is also important in SaaS given that it works on an “on demand” model. Scalability and capacity planning does not always mean throwing in more hardware – doing so, may potentially result in wasted resources. Work with the development team to understand how scalable the product architecture is, to see if any optimizations can be done. Start all these discussions early on, since architectural changes may be very expensive if not impossible to implement at later stages.
Take away #3 from definition: The customer herein just provides software customization requirements to the vendor and follows a “pay per use model” to enable optimum use of resources: Besides the core product functionality, two other important aspects to test in SaaS implementations are: right metering and billing and the right customer customizations.
One of the inherent benefits of the SaaS model is the pay-per-use feature. Unless this has been implemented correctly, there is no way to keep track of the customer’s accurate usage and this may affect the success of the entire application. Test for all the payment and usage algorithms, including boundary cases to ensure the implementation is correct.
Since often times the same application instance is customized and offered to meet the customer requirements, ensuring all of the customizations have been done correctly is very important. This may sound simple but any bug that is missed here, affects the customer’s reputation badly. Imagine having a logo of a competitor on your page – this could be disastrous. This is an area to be specially looked at, unlike in the traditional on-premise applications where this does not pose any risk.
SaaS applications, when implemented correctly and adequately tested for, have a lot of benefits to offer. Reap those benefits, to make your software a success by proactively planning for these above areas well in advance. Any feedback, comments, questions? – I’ll be happy to hear from you.