1. 1 Performance and Availability Models for IaaS Cloud and Their Applications Rahul Ghosh Duke High Availability Assurance Lab Dept. of Electrical and Computer Engineering Duke University, Durham, NC 27708 www.ee.duke.edu/~rg51 Collaborators: Vijay K. Naik, Murthy Devarakonda (IBM), Kishor S. Trivedi, DongSeong Kim and Francesco Longo (Duke) IBM Student Workshop for Frontiers of Cloud Computing Hawthorne, NY, USA September 10, 2010

2. 2 Key problems of interest:  Characterize cloud services as a function of arrival rate, available capacity, service requirements, and failure properties  Apply these characteristics in SLA analysis and management, admission control, cloud capacity planning, cloud economics Approach:  Performability (Performance + Availability) analysis  We use stochastic interacting stochastic sub-models based approach Lower relative cost of solving the models while covering large parameter space compared to measurement based analysis Introduction Two key quality-of-service measures for IaaS cloud: (1) service availability and (2) provisioning response delay

3. 3 Novelty of our approach Single monolithic model vs. interacting sub-models approach -Even with a simple case of 6 physical machines and 1 virtual machine per physical machine, a monolithic model will have 126720 states. -In contrast, our approach of interacting sub-models has only 41 states. Clearly, for a real cloud, a naïve modeling approach will lead to very large analytical model. Solution of such model is practically impossible. Interacting sub-models approach is scalable, tractable and of high fidelity. Also, adding a new feature in an interacting sub-models approach, does not require reconstruction of the entire model. What are the different sub-models? How do they interact?

4. 4 Main Assumptions  All requests are homogenous, where each request is for one virtual machine (VM) with fixed size CPU cores, RAM, disk capacity.  We use the term “job” to denote a user request for provisioning a VM.  Submitted requests are served in FCFS basis by resource provisioning decision engine (RPDE).  If a request can be accepted, it goes to a specific physical machine (PM) for VM provisioning. After getting the VM, the request runs in the cloud and releases the VM when it finishes.  To reduce cost of operations, PMs can be grouped into multiple pools. We assume three pools – hot (running with VM instantiated), warm (turned on but VM not instantiated) and cold (turned off).  All physical machines (PMs) in a particular type of pool are identical. System model

5. 5 Provisioning and servicing steps: (i) resource provisioning decision, (ii) VM provisioning and (iii) run-time execution Life-cycle of a job inside a IaaS cloud We translate these steps into analytical sub-models Resource Provisioning Decision Engine Run-time Execution Instance Creation Deploy Job rejection due to buffer full Job rejection due to insufficient capacity ArrivalQueuing Provisioning Decision Instantiation VM deployment Actual ServiceOut Provisioning response delay Admission control

6. 6 Resource provisioning decision Resource Provisioning Decision Engine Run-time Execution Instance Creation Deploy Job rejection due to buffer full Job rejection due to insufficient capacity ArrivalQueuing Provisioning Decision Instantiation VM deployment Actual ServiceOut Provisioning response delay Admission control

7. 7 A request is provisioned on a hot PM if pre-instantiated but unassigned VM exists. If none exists, a PM from warm pool is used. If all warm machines are busy, a PM from cold pool is used. Resource provisioning decision

8. 8 Continuous Time Markov Chain (CTMC) Resource provisioning decision model i = number of jobs in queue, s = pool (hot, warm or cold) Provisioning decision of a single job

9. 9 Output measures -Job rejection probability due to buffer full (P block ) -Job rejection probability due to insufficient capacity (P drop ) -Total job rejection probability (P reject = P block + P drop ) -Mean queuing delay (E[T q_dec ]) -Mean decision delay (E[T decision ]) Reward rate based approach (attach a reward rate to each state of Markov chain) Little’s law (connecting mean number in the queue with mean waiting time) 3-stage Coxian distribution

10. 10 VM provisioning Resource Provisioning Decision Engine Run-time Execution Instance Creation Deploy Job rejection due to buffer full Job rejection due to insufficient capacity ArrivalQueuing Provisioning Decision Instantiation VM deployment Actual ServiceOut Provisioning response delay Admission control

11. 11 VM provisioning model Service out Resource Provisioning Decision Engine Accepted jobs Running VMs Idle resources in hot machine Idle resources in warm machine Idle resources in cold machine Hot pool Warm pool Cold pool Hot PM

12. 12 VM provisioning model for each hot PM 0,0,00,1,0L h,1,0 0,0,1(L h -1),1,1L h,1,1 1,0,m L h,0,m L h,1,(m- 1) 0,0,m (L h -1),1,(m-1) 0,0,(m-1)0,1,(m-1) … … … … … … … … … … L h is the buffer size and m is max. # VMs that can run simultaneously on a PM i,j,k i = number of jobs in the queue, j = number of VMs being provisioned, k = number of VMs running

13. 13 VM provisioning model for each warm PM 0,0,0 0,1*, 0 L w,1*,0 0,0,1 (L w - 1),1,1 L w,1,1 … … 1,0,m L w,0,m L w,1,(m-1) 0,0,m (L w -1),1,(m- 1) 0,0,(m-1)0,1,(m-1) … … … … … … … … 0,1,0L w,1,0 0,1**,0 L w, 1**,0 …

14. 14 Output measures from VM provisioning models Prob. that a job can be accepted in the hot/warm/cold pool (P h /P w /P c ) Weighted mean queuing delay for VM provisioning (E[T vm_q ]) Weighted mean provisioning delay (E[T prov ])

15. 15 Run-time execution Resource Provisioning Decision Engine Run-time Execution Instance Creation Deploy Job rejection due to buffer full Job rejection due to insufficient capacity ArrivalQueuing Provisioning Decision Instantiation VM deployment Actual ServiceOut Provisioning response delay Admission control

16. 16 Run-time model Model outputs: Mean job service time / resource holding time

17. 17 Output measures from pure performance models All these models are used for pure performance analysis since we do not consider any failure Output of resource provisioning decision model: -Job rejection probability due to buffer full (P block ) -Job rejection probability due to insufficient capacity (P drop ) -Mean queuing delay (E[T q_dec ]) -Mean decision delay (E[T decision ]) Output of VM provisioning models: -Probability that a atleast one machine in hot /warm/cold pool can accept a job for provisioning -These probabilities are denoted by P h, P w and P c for hot, warm and cold pool respectively -Weighted mean queuing delay for VM provisioning (E[T q_vm ]) -Weighted mean provisioning delay (E[T prov ]) Output of run-time model: -Mean job service time Output of pure performance models -Total job rejection probability (P reject = P block + P drop ) -Net mean response delay (E[T resp ]=E[T q_dec ]+E[T decision ]+E[T q_vm ]+E[T prov ])

18. 18 Availability model Model outputs: Probability that the cloud service is available, downtime in minutes per year

19. 19 Model interactions: Performability

20. 20 Numerical Results

21. 21 Effect of increasing job arrival rate

22. 22 Effect of increasing job service time

23. 23 Effect of increasing # VMs

24. 24 Effect of increasing MTTF of a PM

25. 25 Applications of the models

26. 26 Admission control Arrival rate (jobs/hr) Distribution of PMs across different pools (all delays are in seconds) (15, 15, 15)(30, 30, 30)(45, 45, 0)(90, 0, 0) E[T resp ]E[T prov ]E[T resp ]E[T prov ]E[T resp ]E[T prov ]E[T resp ]E[T prov ] 250484.37477.83314.26310.27304.03300.24303.79300.00 500697.98656.92354.87347.83312..00306.62305.14300.00 5505146.12666.07363.95355.66315.00309.06305.54300.00 60013825.85670.52373.99364.03318.42311.80306.00300.00 What is the maximum job arrival rate that can supported by the cloud service? Increasing arrival rate increases response delay. Putting more PMs reduces this delay.

27. 27 Response time – energy trade-off Arrival rate (jobs/hr) Distribution of PMs across different pools (all delays are in seconds) (15, 15, 15)(30, 30, 30)(45, 45, 0)(90, 0, 0) E[T resp ]E[T prov ]E[T resp ]E[T prov ]E[T resp ]E[T prov ]E[T resp ]E[T prov ] 250484.37477.83314.26310.27304.03300.24303.79300.00 500697.98656.92354.87347.83312..00306.62305.14300.00 5505146.12666.07363.95355.66315.00309.06305.54300.00 60013825.85670.52373.99364.03318.42311.80306.00300.00 What is the optimal # PMs across different pools that minimizes response time for a given energy budget? Increasing capacity reduces the gap between actual provisioning delay and response delay.

28. 28 SLA driven capacity planning What should be the size of each pool, so that total cost is minimized and SLA (maximum rejection probability or response delay) is upheld?

29. 29 Recent work on IaaS cloud resiliency

30. 30 Resiliency Analysis Definition of resiliency Resiliency is the persistence of service delivery that can justifiably be trusted when facing changes* changes of interest in the context of IaaS cloud Increase in workload, faultload Decrease in system capacity Security attacks Accidents or disasters Our contributions: Quantifying resiliency of IaaS cloud Resiliency analysis approach using performance analysis models *[1] J. Laprie, “From Dependability to resiliency”, DSN 2008 [2] L. Simoncini, “Resilient Computing: An Engineering Discipline”, IPDPS 2009

31. 31 Effect of changing demand

32. 32 Effect of changing capacity

33. 33 Conclusions Stochastic model can be an inexpensive alternative to measurement based evaluation of cloud QoS To reduce the complexity of modeling, we use an interacting sub-model approach - Overall solution of the model is obtained iteration over individual sub-model solutions The proposed approach is general and can be applicable to variety of IaaS clouds Results show that IaaS cloud service quality is affected through variations in workload (job arrival rate, job service rate), faultload (machine failure rate) and available system capacity This approach can be extended to solve specific cloud problems such as capacity planning of public, private and hybrid clouds In future, models will be validated using real data collected from cloud

34. 34 Thanks!