1. N ORTH D AKOTA S TATE U NIVERSITY D EPARTMENT OF C OMPUTER S CIENCE © NDSU S OFTWARE T ESTING R ESEARCH G ROUP Enabling Automated Integration Testing of Cloud Application Services in Virtualized Environments Tariq M. King, Annaji S. Ganti, and David Froslie

2. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP Outline  Introduction  Background  Research Problem  Approach  Prototype  Discussion  Related Work  Conclusion & Future Work 2

3. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP  Cloud provides ubiquitous, on-demand access to computing resources “as services” via the Internet  Operational Perspective  Availability, Scalability, Efficiency, …  Development Perspective  Reusable Components, Info. Hiding  Highly Complex Local Infrastructure  Testing Perspective? Introduction 3

4. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP  Cloud poses several new testing challenges but also brings a number of benefits to software testing Challenges of Testing Cloud Applications  High Complexity, Remoteness, Information Hiding, Autonomy, High Performance and Dependability… Benefits of Software Testing In The Cloud (STITC)  Computational Power and Storage  Virtualization Theme: Use Benefits to Overcome Challenges Introduction 4 MOTIVATION

5. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP “Enabling Automated Integration Testing of Cloud Application Services in Virtualized Environments”  STITC is an emerging discipline with potential to significantly change how we do testing  STITC 2010 Workshop focused on three key areas:  Testing IN the Cloud – leveraging the cloud for test execution and testing in a virtualized environment  Testing OF the Cloud – validating applications that are hosted and deployed in the cloud  Testing TO the Cloud – moving the testing process and other assets to the cloud Introduction 5 RESEARCH POSITION

6. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP  “Services” in cloud computing typically fall into three categories: Software-as-a-Service | Google Docs, Office Web Apps Platform-as-a-Service | Google App Engine, Windows Azure Infrastructure-as-a-Service | Amazon EC 2, Rackspace Background 6 CLOUD COMPUTING

7. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP  Many cloud platforms offer Virtualization support  Creating abstract computing resources from more powerful physical resources Hosted Architecture Background 7 VIRTUALIZATION Hypervisor Architecture

8. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP  To narrow the research scope, we analyze a specific cloud application development scenario  One host (Provider B) is developing an application service that will extend the functionality of a service offered remotely by another host (Provider A) Research Problem 8 SCOPE

9. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP Research Problem 9 ISSUES

10. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP  Our approach to the research problem introduces the notion of Test Support as-a-Service (TSaaS)  Prior to deploying Service A, Provider A would have tested it, which typically involves creating:  Test Artifacts – Automated Test Drivers and Stubs  A Test Bed – Hardware/Software Environment  TSaaS seeks to reuse such existing test automation for the development and delivery of a set of test support services for collaborating partners Approach 10

11. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP  Provider B has full access to a test copy of Service A  Test hooks, instrumentation, and scaffolding reuse  Test data (non-confidential) can be made available  Testing does not interrupt service in production  Concerns: Maintenance, Scalability, Security Approach 11 OVERVIEW Provider A exposes TSaaS to Provider B

12. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP Hypervisor-Based Virtualization and Secure Access Approach 12 ARCHITECTURE

13. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP Approach 13 CONTROL FLOW Control Flow When Realizing Test Support Services

14. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP  Designed with goal of making its components easily deployable to Windows Azure ®  Web & Worker Roles (ASP.NET, IIS 7, WCF Services)  VM Role (Windows 2008 R 2 Server Image)  Application Domain: Credit Reporting/Banking Prototype 14 PROOF OF CONCEPT

15. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP The primary reason for developing the prototype was to investigate the feasibility of implementing TSaaS Successes  Able to quickly develop and expose a cross section of test support operations for the credit service by reusing its existing test artifacts and tooling  Test authoring and execution, code coverage, error reporting, and server-side diagnostics  Facilitates independently setting up preconditions, applying inputs, making assertions, doing cleanup Discussion 15

16. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP Technical Challenges  Locating and Integrating Required Technologies  Combines programming knowledge, test automation skills, and runtime virtualization expertise  Configuring Virtual Hard Disk Images of Servers  Enable Auto-Login, Install Security Patches, Disable Reboot after Auto-Update, Set User Privileges  Emphasized the importance and need for VM tools for manipulating and updating these images Discussion 16 ACCIDENTAL DIFFICULTIES

17. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP Discussion 17

18. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP  Formal modeling and model-based testing criteria for cloud applications (Chan 2009)  Model-driven approach to security testing of cloud applications (Zech 2011 )  Industry challenges for runtime integration and testing for software services (Greiler 2009 )  Many works on testing and testability of SOA (Mei 2008, O’Brien 2007, Tsai 2006 ) Related Work 18

19. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP  Presented an approach to support integration testing of cloud applications using virtualized environments – Test Support as-a-Service (TSaaS)  Described implementation of a TSaaS Prototype  Provides some evidence that approach is feasible  Future work calls for Evaluation and Extension  Azure Experiments, Alternative Implementation, Standardization, Expand Problem Scope, Benefits Virtualization Benefits/Implications, Performance  Building Self-Testing into the Cloud (STITC 2011 ) Conclusion and Future Work 19

20. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP Acknowledgements 20  Neha Kale, MS Student (NDSU)  Dr. Scott Tilley, Professor (Florida Tech), and the participants of STITC 2010  Alisson Sol, Knowledge Engineer (Microsoft) and the participants of 2011 Microsoft Fargo Engineering Day Excellence Expo  CASCON 2011 Reviewers

21. © NDSU S OFTWARE T ESTING R ESEARCH G ROUP Thank You! 21 Questions? ¿Preguntas? 問題 Sawwal вопросы 質問 domande ερωτήσεις