1. Copyright 2010 ITRI 工業技術研究院 11 ITRI Cloud OS & Virtual Resource Management Patrick Fu System Software Division, CCMA/ITRI patrickfu@itri.org.tw

2. Copyright 2010 ITRI 工業技術研究院 2 Agenda Cloud OS introduction Physical resource provisioning Virtual resource management Adaptive provisioning and power management

3. Copyright 2010 ITRI 工業技術研究院 3 What is Cloud OS ? Physical Node Storag e Server Physical Node Storage Server Mail Virtual Cluster Compute Nodes Backup Virtual Cluster HC Virtual Cluster AppX Virtual Cluster Data Nodes Service Nodes System Service daemons Cloud OS agents System Management Software layer –Physical Resource Provisioning –Virtual Resource Management Improve manageability of massive Cloud Data Center Enhance self-provisioning Optimize physical resource utilization High Availability for any single point of failure Energy management –Highly Available Distributed Storage Management –Service Load Balancing –Security –High Speed Networking What is it not? –It’s not Operating System –It’s not Virtualization Hypervisor

4. Copyright 2010 ITRI 工業技術研究院 4 Service/Technology Mapping IaaS PaaS Servers Storage Arrays Power Distribution Switches + Scalable System Architecture System Management Cooling Cloud Hardware Platform Hypervisor Virtualization Mgmt Storage Mgmt Security Backup/Replication Data Center Automation Energy Management Cloud System Software Platform LAMP.NET WebSphere WebLogic Google App Engine Cloud Application Middleware Platform SaaS Automated Cloudification Technology Applications

5. Copyright 2010 ITRI 工業技術研究院 5 Software Stack for Cloud OS Physical Cluster Deployment Tool Virtual Machine Management Virtual Cluster Provisioning Power Management Intra-Virtual-Cluster Load Balancing System/Network Management Security Virtual DataCenter Mgmt Console Physical Compute Servers All-layer-2 Network Distributed Main/Secondary Storage

6. Copyright 2010 ITRI 工業技術研究院 6 Cloud OS Service Model Provisioning & Runtime monitoring of Virtual Resources –Virtual Instance  Hypervisor construct  An image of a guest OS –Virtual Cluster  A group of VM instances providing same service, front-ended by a network load balancer  Configuration -# of virtual machines and its configuration -Storage space requirement -External network bandwidth requirement -Load balancing policy -Firewall/IDS setting -Network configuration, including DNS and DHCP -OS image and application image –Virtual Data Center  One or more virtual cluster working in coordination (multi-tier web services, EMR’s, VDI’s, etc)

7. Copyright 2010 ITRI 工業技術研究院 7 CloudOS Virtualization Level … PM OS APs vm OS APs vm OS APs vm OS APs vm … PM OS APs vm OS APs vm OS APs vm OS APs vm … PM OS APs vm OS APs vm OS APs vm OS APs vm … PM OS APs vm OS APs vm OS APs vm OS APs vm … VCluster VDC CloudOS

8. Copyright 2010 ITRI 工業技術研究院 8 Resource Provisioning To prepare VMs with appropriate resources and make them ready for user applications –Allocating resources to VMs to match the workloads To prepare a virtual cluster with appropriate instances and make it ready for virtual cluster computation –Consolidating VMs onto physical servers Goals: –High resource utilization –Energy efficiency –Low performance interference

9. Copyright 2010 ITRI 工業技術研究院 9 Provisioning Challenges VM size estimation –Static SLA model/forecasting future use Placement –Deploy a VM onto physical servers (initially) –Policy: immediate, best effort, advance reservation, etc. Consolidation and Load Balancing –High consolidation ration and resource utilization – low cost of running data center –Statically Heuristic based Average resource utilization –Dynamic replacement Measure-Forecast-Remap (MFR) Live migration Balancing overloaded and underloaded nodes –Constrained bin packing problem w/ SLA Performance isolation –Cohosting VMs on a server creates performance interference –How to model and prevent the interference

10. Copyright 2010 ITRI 工業技術研究院 10 RPM Static Resource Provisioning Statically provision from SLA SLA w/ historical data? –No, conservatively allocation –Yes, forecasting joint-VM provisioning Immediate provisioning model (before instantiation of virtual machine) Placement policy –Proprietary –Virtual cluster affinity placement policy Performance gain from locality Place VMs from the same virtual cluster as possible Need experiments to support CloudOS

11. Copyright 2010 ITRI 工業技術研究院 11 Our Motivation Source: Cost of power in Large-Scale Data Center, James Hamilton Blog, 11/28/2008

12. Copyright 2010 ITRI 工業技術研究院 12 Joint Provisioning via VM Multiplexing Dataset from a commercial data center –15,897 VMs –1325 physical hosts 94% of the hosts have more than one VM Joint provisioning averagely saves 40% of the capacity Meng, X. et al. Efficient Resource Provisioning in Compute Clouds via VM Multiplexing. ICAC ‘10

13. Copyright 2010 ITRI 工業技術研究院 13 Joint-VM Provisioning at Runtime Hypervisor PM VM Hypervisor PM VM 100% Under provisioning Over provisioning CloudOS Capacity tt tt

14. Copyright 2010 ITRI 工業技術研究院 14 Load balancing and DVMM Consolidation manager/ DVMM PM Over provisioning Under provisioning Joint-VM histogram VM victim histogram Resource Provisioning Manager (RPM) PM reconfiguration Utilization ratio Reach reconfiguration point CloudOS

15. Copyright 2010 ITRI 工業技術研究院 15 Adaptive physical resource provisioning PRM Power on/off PMs Reconfiguration map New PM map Utilization rate reaches threshold, sending reallocation request Static joint-VM provisioning DVMM Consolidation manager Placement VM monitoring Runtime joint-VM provisioning Performance interference Utilization change Victims RPM core New PM map Cloud OS RPM Software Components Cloud OS

16. Copyright 2010 ITRI 工業技術研究院 16 Load balancing

17. Copyright 2010 ITRI 工業技術研究院 17 Consolidation plan

18. Copyright 2010 ITRI 工業技術研究院 18 Migration plan

19. Copyright 2010 ITRI 工業技術研究院 19 Runtime Reallocation VM3 VM2 VM1 PM1 VM3 VM2 VM1 PM2 VM3 VM2 VM1 PM3 VM1 PMi VM1 PMj VM1 PMk VM3 VM1 PM1 VM3 VM1 PM2 VM3 VM2 VM1 PM3 VM1 PMi VM1 PMj VM1 PMk VM2

20. Copyright 2010 ITRI 工業技術研究院 20 Adaptive Physical Resource Provisioning Power Management PM Pool Provisioned Utilization threshold Low utilizationHigh utilization Over provisioningUnder provisioning PM reallocation algorithm PRM Power on/off PMs Reconfiguration map VM placement DVMM live migration Load balancer Consolidation manager Utilization monitor New PM map Utilization rate reaches threshold, sending reallocation request CloudOS

21. Copyright 2010 ITRI 工業技術研究院 21 Challenges Triggering mechanism –No workload consolidation “recently” (e.g. past hour) –No physical machine load balancing going on –No physical server was powered on “recently” (e.g. past hour) –Avoid oscillation Cost of migration –Network load –Cache effects –Domain in suspension Multi-dimensional bin packing –CPU, Memory, Network, Disk I/O Migration plan –Only 1 migration per Physical server @ a time –# of cores vs. # of VMs

22. Copyright 2010 ITRI 工業技術研究院 22 Backup

23. Copyright 2010 ITRI 工業技術研究院 23 Software architecture

24. Copyright 2010 ITRI 工業技術研究院 24 Procedure of Power management in Monitoring thread Receive data from Dom0 Calculate the data Workload Trigger? Yes Perform Consolidation Plan M*K < N ? No Receiving CCinstance data per 30 sec Calculate instMonitorThreadData->cputotal instMonitorThreadData->memorytotal instMonitorThreadData->count Do Consolidation Change instance state instDvmmBloc->state=doingpwm instMonitorThreadData->state=doingpwm instDvmmBloc->destHost=resource->hostName Change external state Call PRM to shut down Machine Change instance state instDvmmBloc->pwmtime=now instDvmmBloc->state=pwmdone instMonitorThreadData->state=pwmdone instMonitorThreadData->pwmtime=now Stop receiving data from Dom0 Check instMonitorThreadData->state instMonitorThreadData->pwmtime instMonitorThreadData->lbtime instMonitorThreadData->cputotal instMonitorThreadData->memorytotal instMonitorThreadData->count Done Yes Do Load balancing Change instance state instDvmmBloc->state=doinglb instMonitorThreadData->state=doinglb instDvmmBloc->destHost=resource->hostName Change external state Call PRM to turn on Machine If necessary

25. Copyright 2010 ITRI 工業技術研究院 25 Data Mining: VM Resource Usage Patterns of each VC Find VM resource usage patterns for each VC Aid to predict the trend of resource usages Medium L Low High (or unpredictable) Time CPU usage Monday

26. Copyright 2010 ITRI 工業技術研究院 26 Q&A Thank you!