SaaS (Software-as-a-Service) businesses are seeing hyper-growth, and with more and more organizations adopting SaaS instead of on-premise software, the R&D organizations of SaaS companies have their task cut out. The QA/Testing team are especially challenged owing to the short release cycles and the fact that thousands of customers will be served by the same infrastructure. Most organizations report impediments to SaaS testing like – short notice periods for QA notification, frequent testing of live upgrades, short validation cycle times, impact on multiple subscriber organizations, privacy violations, errors due to rapid addition of new features, time taken for data migration, concerns over data security & integrity etc. cloud the obvious benefits of SaaS testing.

In this article, we will look at the specific challenges faced by QA organizations of SaaS companies. In our next article, we will highlight the testing strategy to address these challenges.

  1. Handling Changes through Frequent Releases – Every time the Application is upgraded, the users have to understand the impact of the change, validate it against the existing system & ensure that the impact on the existing features of the application is minimal. Managing and executing all these activities within a short time span (1-2 weeks) is challenging. When SaaS upgrades involve interface upgrading, compatibility and integration issues across old and new interfaces crop up for the subscribers. Live upgrades being simulated or tested on the SaaS application impedes the activity of the existing users.
  2. Security Testing – Maintaining data security, accessibility & integrity on a single SaaS application across multiple tenants. To understand individual privacy requirements, privilege levels, behavioural patterns and provide adequate privacy to the data can be a daunting task.
  3. Integration Challenges – When subscribers integrate their internal enterprise applications with SaaS, inbound and outbound data integration validations from client networks to the SaaS providers is needed. In such cases it is very difficult to conduct thorough validation simultaneously ensuring 100% data security and privacy
  4. Data Migration Issues – Data migration across different SaaS applications or from other applications to SaaS can be challenging in terms of time taken for understanding the requirements and the exhaustive integration validation processes
  5. Licensing: The SaaS app licensing may vary by functionality, usage (such as volume of transactions or amount of specific data) or # of named/concurrent users. All this needs to be tested across every release.
  6. Performance testing: Successfully modelling the most-used business transactions, application usage and user mix may require greater diligence than an on-premise application.