1. Engineering Multi-Tenant Software-as-a-Service Systems Bikram Sengupta (IBM Research India) Abhik Roychoudhury (National University of Singapore) ICSE 2011 workshop PESOS 2011, Honolulu1

2. What is SaaS? ICSE 2011 workshop PESOS 2011, Honolulu2  Software as a service (SaaS) is a software delivery model in which software and its associated data are hosted centrally (typically in the Internet/cloud) and are typically accessed by users using a thin client, normally using a web browser over the Internet. — wikipedia  Gartner predicts that the SaaS market will grow at a compound annual growth rate (CAGR) of 22.1% through 2011  Gartner/Dataquest Insight: “SaaS Demand Set to Outpace Enterprise Application Software Market Growth.”

3. SaaS vs. On-Premise Deployment ICSE 2011 workshop PESOS 2011, Honolulu3 SaaSOn-Premise Simplicityweb-based, simple deployment installation, maintenance Flexibilityless flexible, requires adaption full ownership, customizable Accessibilityinternet reliabilityhigher availability CostLow initial cost, ongoing subscription Software license, infrastructure

4. A. Lenk et al, ICSE 2009 WS on SE-Cloud ICSE 2011 workshop PESOS 2011, Honolulu4

5. Challenges in building Multi-tenant SaaS ICSE 2011 workshop PESOS 2011, Honolulu5 vendor-offered customizations tenant-specific changes Fixed-set SLA: High commonality Ease development/maintenance Little variability Tenants need to compromise Diverse requirements: Improve Flexibility Meet customer needs Degrade level of sharing Increase vendor’s overhead

6. Broader perspective ICSE 2011 workshop PESOS 2011, Honolulu6  Evolving programs / program versions  Testing [ASE08, ASE10, FSE10, …]  Test suite augmentation to stress changes  Debugging [ESEC-FSE99, ESEC-FSE’09, FSE10, …]  DARWIN method [ESEC-FSE99] uses symbolic execution for regression debugging.  Design and development of multi-tenant SaaS software can benefit from research on evolving programs.  Customization of SaaS to accommodate new tenants – can be systematized using testing / debugging research.

7. SaaS Market ICSE 2011 workshop PESOS 2011, Honolulu7  Data from IDC Total public cloud services revenue in 2010 SaaS Market 71%, $17.5b Expected SaaS Market in 2014 $40.9b 25.9% compound annual growth  SaaS applications  ERP, CRM  Security products  …

8. A Scenario that did not happen ICSE 2011 workshop PESOS 2011, Honolulu8 ?? http://www.asxgroup.com.au/asx-sgx-merger.htm The Federal Treasurer, The Honourable Wayne Swan, has made a statement that he has rejected the proposed merger of ASX Limited (ASX) and Singapore Exchange (SGX). In these circumstances the parties have agreed to terminate the Merger Implementation Agreement entered into on 25 October 2010.

9. A Business Case ICSE 2011 workshop PESOS 2011, Honolulu9  Suppose SGX and ASX make an agreement to offer joint online trading services for a range of stock transactions. This has to be made operational very soon to exploit a favorable economic climate.  Several new business processes have to be supported with a high number of variants tailored to the needs of different grades of investors  A number of third party services may need to be leveraged, some of which support large-scale data analytics off remote cloud infrastructures  Grades of investors have to be offered specialized SLAs based on their usage patterns and needs (performance expectations, security, isolation, recoverability requirements etc)  A highly-scalable multi-tenant SaaS solution has to be developed very rapidly, and tested thoroughly before being deployed on a cloud.

10. Challenges ICSE 2011 workshop PESOS 2011, Honolulu10  Propose a SaaS system management model (from the vendor’s perspective )  Enable quantitative analysis of SaaS system  Tenant management  On-boarding decision (return-on-investment)  Variability/commonality management  Application evolving  Add new functionalities  Testing  Re-factoring

11. SaaS Management Model: Overview ICSE 2011 workshop PESOS 2011, Honolulu11 … Service II Service I Service X Feature 2 v1 v2 Feature 1 v1 v3 v2 … T1 T2 T3 subscription

12. SaaS System ICSE 2011 workshop PESOS 2011, Honolulu12  Service: a group of offered functionalities  service invariants  Feature: a particular functionality  feature invariants  Variant v = : a variation of the feature to support certain customization/changes

13. Case study ICSE 2011 workshop PESOS 2011, Honolulu13  Apache Roller  Free open source Java blog server roller.apache.org/roller.apache.org/  Powers the USA government blog, employee blogs at Sun and IBM, Java blogging community …  Convert to a multi-tenancy SaaS system  Tenants: individual blogging sites  End user: site admin, blogger, reader…  Services:  Admin services  Weblog services  Features: write blog, edit blog, delete blog, category management …  Comment services  …

14. Write Weblog Variants ICSE 2011 workshop PESOS 2011, Honolulu14 Pre-conditions: 1.logged-in blogger 2.Word limit 2000 3.Rich/plain text Implementation Meth(); Post-conditions: 1.Publish time is displayed 2.Email notifications are sent to owner and newsfeed subscribers Existing variant (v0) New request/variant (v1) Pre-conditions: 1.Logged-in blogger 2.Word limit 1000 3.Plain text only Implementation Configuration + Meth(); Post-conditions: 1.Email notifications are sent to newsfeed subscribers Less impl. cost v1 vs. v0 1.Stronger pre-condition 2.Weaker post-condition 3.v0 is a refinement of v1

15. Write Weblog Variants ICSE 2011 workshop PESOS 2011, Honolulu15 Pre-conditions: 1.logged-in owner 2.Word limit 2000 3.Rich/plain text Implementation Meth(); Post-conditions: 1.Publish time is displayed 2.Email notifications are sent to owner and newsfeed subscribers Existing variants (v0 & v1) New request/variant (v2) Pre-conditions: 1.Authorized blogger 2.Select a category 3.Word limit 2000 4.Rich/plain text Implementation Meth1() Post-conditions: 1.Weblog is stored in the selected category 2.Publish time is displayed v2 vs. existing variants: 1.No refinement  new impl. Is needed 2.Derive the impl. from the closest existing variant 3.Create a new variant or feature in the model (depends on the distance) v0 v1 cost ∝ distance

16. On-boarding Cost ICSE 2011 workshop PESOS 2011, Honolulu16  Accommodate a new service request  Direct on-boarding: exist a same variant, or a refinement  No new implmentation  constant cost  Feature expansion: create a new variant of an existing feature  Associate cost with weakening/strengthening of pre/post conditions  Assumption: weaken post strengthen post  Feature set augmentation: create a new feature  Service set augmentation: create a new service

17. Onboarding Profit ICSE 2011 workshop PESOS 2011, Honolulu17 Profit of onboarding a new tenant Subscription fee Expected pay-by-use revenue Aggregated accommodation cost for each request cost

18. Commonality ICSE 2011 workshop PESOS 2011, Honolulu18  Commonality characterizes the level of reuse of a SaaS system  Higher commonality  higher degree of sharing of features by tenants  We expect that for each feature, a significant fraction of tenants should subscribe to it (including its variants)

19. Tenant Selection Problem ICSE 2011 workshop PESOS 2011, Honolulu19  A set of tenants to be onboarded on an existing system  Multi-objective optimization problem  maximize profit and maximize commonality  Find optimal subset of the tenants commonality profit Commonality threshold Pareto front

20. SaaS Testing ICSE 2011 workshop PESOS 2011, Honolulu20  When a new tenant is on-boarded, how to test that existing tenants are not impacted by the changes introduced?  No update of existing operations  not relevant in our approach  How do we efficiently test that the SaaS system meets the needs of the different tenants that have been onboarded ?  large degree of commonality  test only the changes introduced by a tenant  Test suite augmentation problem [ASE10, for evolving programs]  exploits similarities amongst tenants during testing  Test tree

21. Test Tree ICSE 2011 workshop PESOS 2011, Honolulu21 100 tests 300 tests 150 tests 30 tests50 tests Tenant {t1}Tenants {t2, t5} Tenants {t3, t4, t6}

22. Re-factoring SaaS Systems ICSE 2011 workshop PESOS 2011, Honolulu22  Merging  Useful for moving legacy services to the cloud  Analyze specification, detect commonalities/differences  Improve sharing, increase overall system commonality Feature 2 v4 v5 Feature 1 v1 v3 v2 Feature 1’ v1v2 v4v5 * v4 is identified as a refinement of v3, hence tenants subscribing v3 can be moved to v4 (with necessary configuration)

23. Re-factoring SaaS Systems ICSE 2011 workshop PESOS 2011, Honolulu23  Splitting  dual of the merging operation  break a large SaaS into smaller ones  reduce variability and improve maintenance.  Pruning:  retiring entities (services, operations) that the vendor perceives to be of low utility  based on financial motives

24. Initial Experiments ICSE 2011 workshop PESOS 2011, Honolulu24  Conference Online Management System  http://www.conference-service.com/index.html http://www.conference-service.com/index.html  Chair/ Reviewer/ Author/ Participant/ …  Randomly generated tenant sets (20) and requests  Create two sets (set1 and set2) of costs associated with each type of the onboarding cost factors  direct onboarding < weaken post cond. < strengthen pre cond. < weaken pre cond. < strengthen post cond. < add new feature  consider only the implementation cost (no hardware or maintenance cost)  Compute the Pareto optimal for the tenant selection problem

25. Experimental Results ICSE 2011 workshop PESOS 2011, Honolulu25

26. Concluding Remarks ICSE 2011 workshop PESOS 2011, Honolulu26  Towards a formal model for SaaS system  Variant management  Cost model  Tenant onboarding management  Based on the model, we propose various vendor operations:  System testing  Test suite augmentation, Exploit commonality of test suites  System maintenance/re-factoring  Larger scale experiments – future work.