Cloud testing

Cloud testing is a form of software testing in which web applications use cloud computing environments (a "cloud") to simulate real-world user traffic.

Overview

Cloud testing uses cloud infrastructure for software testing.[1] Organizations pursuing testing in general and load, performance testing and production service monitoring in particular are challenged by several problems like limited test budget, meeting deadlines, high costs per test, large number of test cases, and little or no reuse of tests and geographical distribution of users add to the challenges. Moreover, ensuring high quality service delivery and avoiding outages requires testing in one's datacenter, outside the data-center, or both. Cloud Testing is the solution to all these problems. Effective unlimited storage, quick availability of the infrastructure with scalability, flexibility and availability of distributed testing environment reduce the execution time of testing of large applications and lead to cost-effective solutions.

Need for cloud testing

Traditional approaches to test a software incurs high cost to simulate user activity from different geographic locations.[2] Testing firewalls and load balancers involves expenditure on hardware, software and its maintenance.[3] In case of applications where rate of increase in number of users is unpredictable or there is variation in deployment environment depending on client requirements, cloud testing is more effective.[4]

Types of testings supported

Types of testing

Stress

Stress Test is used to determine ability of application to maintain a certain level of effectiveness beyond breaking point. It is essential for any application to work even under excessive stress and maintain stability.[5] Stress testing assures this by creating peak loads using simulators. But the cost of creating such scenarios is enormous. Instead of investing capital in building on-premises testing environments, cloud testing offers an affordable and scalable alternative.

Load

Load testing of an application involves creation of heavy user traffic, and measuring its response. There is also a need to tune the performance of any application to meet certain standards. However a number of tools are available for that purpose.

Performance

Finding out thresholds, bottlenecks & limitations is a part of performance testing.[6] For this, testing performance under a particular workload is necessary.[7] By using cloud testing, it is easy to create such environment and vary the nature of traffic on-demand. This effectively reduces cost and time by simulating thousands of geographically targeted users.

Functional

Functional testing of both internet and non-internet applications can be performed using cloud testing. The process of verification against specifications or system requirements is carried out in the cloud instead of on-site software testing.

Compatibility

Using cloud environment, instances of different Operating Systems can be created on demand, making compatibility testing effortless.

Browser performance

To verify application's support for various browser types and performance in each type can be accomplished with ease. Various tools enable automated website testing from the cloud.

Latency

Cloud testing is utilized to measure the latency between the action and the corresponding response for any application after deploying it on cloud.

Steps

Companies simulate real world Web users by using cloud testing services that are provided by cloud service vendors such as Advaltis, Compuware, HP, Keynote Systems, Neotys, RadView and SOASTA. Once user scenarios are developed and the test is designed, these service providers leverage cloud servers (provided by cloud platform vendors such as Amazon.com, Google, Rackspace, Microsoft, etc.) to generate web traffic that originates from around the world. Once the test is complete, the cloud service providers deliver results and analytics back to corporate IT professionals through real-time dashboards for a complete analysis of how their applications and the internet will perform during peak volumes.

Steps for Cloud Testing

Keys to successful testing
  1. Understanding a platform provider's elasticity model/dynamic configuration method
  2. Staying abreast of the provider's evolving monitoring services and Service Level Agreements (SLAs)
  3. Potentially engaging the service provider as an ongoing operations partner if producing commercial off-the-shelf (COTS) software
  4. Being willing to be used as a case study by the cloud service provider. The latter may lead to cost reductions.

Applications

Cloud testing is often seen as only performance or load tests, however, as discussed earlier it covers many other types of testing. Cloud computing itself is often referred to as the marriage of software as a service (SaaS) and utility computing. In regard to test execution, the software offered as a service may be a transaction generator and the cloud provider's infrastructure software, or may just be the latter. Distributed Systems and Parallel Systems mainly use this approach for testing, because of their inherent complex nature. D-Cloud[8] is an example of such a software testing environment.

For testing non-internet applications, virtual instances of testing environment can be quickly set up to do automated testing of the application. The cloud testing service providers provide essential testing environment as per the requirement of the application under test. The actual testing of applications is performed by the testing team of the organization which owns the application or third party testing vendors.

Tools

Leading cloud computing service providers include, among others, Amazon, Advaltis, 3-terra, Microsoft, Skytap, HP and SOASTA.[9]

Benefits

The ability and cost to simulate web traffic for software testing purposes has been an inhibitor to overall web reliability. The low cost and accessibility of the cloud's extremely large computing resources provides the ability to replicate real world usage of these systems by geographically distributed users, executing wide varieties of user scenarios, at scales previously unattainable in traditional testing environments. Minimal start-up time along with quality assurance can be achieved by cloud testing.

Following are some of the key benefits:

Issues

The initial setup cost for migrating testing to cloud is very high as it involves modifying some of the test cases to suit cloud environment. This makes the decision of migration crucial.[12] Therefore, cloud testing is not necessarily the best solution to all testing problems.[13]

Legacy systems & services need to be modified in order to be tested on cloud. Usage of robust interfaces with these legacy systems may solve this problem.[14] Also like any other cloud services, cloud testing is vulnerable to security issues.

The test results may not be accurate due to varying performance of service providers’ network and internet.[15] In many cases, service virtualization can be applied to simulate the specific performance and behaviors required for accurate and thorough testing.

References
  1. Tilley, Scott; Parveen, Tauhida (2010). "Migrating software testing to the cloud". 2010 IEEE International Conference on Software Maintenance. p. 1. doi:10.1109/ICSM.2010.5610422. ISBN 978-1-4244-8630-4.
  2. Girmonsky, Alon. "Cloud-testing compared to traditional testing with-in the corporate LAN". Archived from the original on 25 April 2012. Retrieved 12 October 2011.
  3. Ariola, Wayne. "The Next Generation of Test Environment Management". Retrieved 12 October 2011.
  4. Dubie, Denise (2008-08-06). "Poor application performance translates to lost revenue, research shows". Retrieved 12 October 2011.
  5. "Using the Cloud to Stress Test Your Web Applications". Archived from the original on 19 December 2011. Retrieved 12 October 2011.
  6. "Performance and Load Testing Services using Cloud Computing". Archived from the original on 2 November 2011. Retrieved 12 October 2011.
  7. Ganon, Zohar; Zilbershtein, Itai E. (2009). "Cloud-based Performance Testing of Network Management Systems". 2009 IEEE 14th International Workshop on Computer Aided Modeling and Design of Communication Links and Networks. pp. 1–6. doi:10.1109/CAMAD.2009.5161466. ISBN 978-1-4244-3532-6.
  8. Banzai, Takayuki; Koizumi, Hitoshi; Kanbayashi, Ryo; Imada, Takayuki; Hanawa, Toshihiro; Sato, Mitsuhisa (2010). "D-Cloud: Design of a Software Testing Environment for Reliable Distributed Systems Using Cloud Computing Technology". 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing. pp. 631–636. doi:10.1109/CCGRID.2010.72. ISBN 978-1-4244-6987-1.
  9. "Top 10 Cloud Computing Load Test and Performance Monitoring Companies".
  10. "Test infrastructure in the Cloud business case; a cost reduction, or not?". Retrieved 12 October 2011.
  11. Jaideep. "Five Benefits of Software Testing On Cloud". Archived from the original on 18 September 2011. Retrieved 12 October 2011.
  12. Parveen, Tauhida; Tilley, Scott (2010). "When to Migrate Software Testing to the Cloud?". 2010 Third International Conference on Software Testing, Verification, and Validation Workshops. pp. 424–427. doi:10.1109/ICSTW.2010.77. ISBN 978-1-4244-6773-0.
  13. Tilley, Scott; Parveen, Tauhida (2010). "Migrating software testing to the cloud". 2010 IEEE International Conference on Software Maintenance. p. 1. doi:10.1109/ICSM.2010.5610422. ISBN 978-1-4244-8630-4.
  14. Roodenrijs, Ewald (1 November 2010). "Testing on the Cloud" (PDF). Archived from the original (PDF) on 21 October 2012. Retrieved 20 October 2011. Cite journal requires |journal= (help)
  15. MacVittie, Lori. "Cloud testing: The next generation". Archived from the original on 3 September 2012. Retrieved 12 October 2011.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.