CCMA & Cloud OS 符儒嘉 工研院雲瑞運算中心 系统軟體組 組長 1

Agenda Introduction CCMA @ ITRI (工研院雲端運算行動應用科技中心 ) Cloud OS Summary Virtual Data Center & Virtual Clusters Virtualized Storage Networking in a Cloud Data Center Runtime Virtual Machine Management Security System Management Summary Introduction to Cloud Computing and what’s position of CCMA Why we need Cloud OS and go into some details

Cloud Computing Definition Provisioning of dynamically scalable and virtualized resources as a service over the Internet. Multi-tenancy Device & Location independence Ability to obtain virtual computing resource on demand Provides the Illusion of infinite computing resources Self-Provisioning of virtual resources Eliminates the need for up-front commitment by Cloud developers Provides the ability to pay as you go for use of computing resources Reliability, Scalability, Security, Manageability Cloud Computing is pretty board in definition CCMA focus on server side, specifically on how to build a Cloud Computing Center with Container Computer and Cloud OS projects Allow multiple clients to get to their data from anywhere with no dependency on Client end device Elasticity Provisioning without reliance on IT Pay as you go Secure, Highly scalable, available, and manageable

Cloud Computing vs. Utility Services 電力供應 發電廠 超高壓變電所 一次變電所 二次變電所 配電變電所 家庭用戶 大規模場所 工廠 大樓 SaaS Providers PaaS IaaS 雲端運算 End Users ISVs 企業 資料中心 資料來源：IEK (2010/02)

Timing is right Technology Push Market Pull Broadband network connectivity getting faster and more reliable Internet service availability significantly improved Sufficient trust in infrastructure providers By many measures, Google is already a critical service for most of the world, and it is in the cloud! Market Pull Big Data Software install on premise  Software as a service (SaaS) Information technology (IT) on premise  IT service as a rented utility (as in electricity) “IT should not and will not be a core competence for most corporations” Nicholas Carr’s - “Does IT matter?” and “The Big Switch” Lowering up-front and day-to-day IT cost: pay only as much as actual resource usage Similar issues before – IT outsourcing; Application Service Provider Why is this time different? Other than technology push, Market pull is also another reason By Technology push, I mean…. Virtualizaiton, bandwidth, security enhancements (IDS/IPS, WAF, etc) Market Pull, data explosion-- 300,000 petabytes of public data produced in 2009 alone Enterprise now seeing much more pain for DataCenter and IT cost Prevalence of SaaS – People like Elasticity

Cost of Data Center Power Usage

Types of Clouds Hybrid Cloud Service Providers Cloud Providers Public Cloud Service Users Private Cloud Service Providers Cloud End-User Services (SaaS) Cloud Platform Services (PaaS) Cloud Infrastructure Services (IaaS) Cloud Providers In 2010, IaaS, PaaS, and SaaS will no longer be just for Public Clouds. Startup companies are Cloudera and VMOps are absolutely cashing in on this. VMOps just closed an $11M series B funding Building a Computing Cloud is very similar to building a Data Center. You have to mind about SLA, storage, Networking, etc. Except the cost for IaaS is much lower than buying HW to build the Data Center There’s no reason why companies like Merrill Lynch, Walmart can’t operate their own data center like Cloud data centers. Physical Infrastructure 7

Infrastructure as a Service Example Players Amazon GoGrid RightScale Rackspace VMOps Eucalyptus ElasticHosts … Example Players

Platform as a Service Example Players Microsoft Azure Google App Engine Force.com Rackspace Cloud Heroku QuickBase Caspio …

Software as a Service Example Players SaleForce.com Adobe.com Autodesk WebEx Microsoft Office Gmail & other Google Apps Flicker …

DataCenter as a Computer Majority of cloud computing infrastructure consists of reliable services delivered through data centers Traditional colocation Datacenters Multiple servers and communications gear collocated due to common environmental & security needs Hosts a large number of relatively small or medium-sized applications, each running on a dedicated hardware infrastructure Datacenters for Cloud Computing platform Belongs to a single organization, Uses a relatively homogeneous hardware and system software platform, and share a common system management layer. Runs a smaller number of very large applications Cloud computing workloads must be designed to gracefully tolerate large numbers of component faults with little or no impact on service level performance and availability. Google and Amazon pretty successful Cloud Server providers. How they build their data center is quite different

Warehouse Scale Computers (WSC) Not just a collection of servers Hundreds to Thousands of servers running in concordance Typically runs on a virtualized platform Fault behavior & energy considerations have significant impact Needs to be considered as a single unit Must be highly manageable Deployment of software updates Monitoring & system management Affordability Currently power Public Cloud such as Google, Amazon, Yahoo, Microsoft’s, etc… Soon to be affordable by Enterprises A rack of servers can easily have > 600 cores WSC is an concept coined by Google developers. The other insteresting point is that this is not as expensive as you think. With the current architecture, 8-core CPU’s are coming out. A rack can contain roughly 30 1U servers, you can have a rack that contains close to 500 cores. The cost is not that high for a rack. There’s no reason why big corporation can also start building their DataCenters this way In other words, big Enterprise will start buying Data Centers, rather than buying servers

Google “Warehouse Style Computer” Data Center This is already shrinking to a container (modular approach)

“Secret Sauce” of Cloud Computing Commodity components Virtualization Servers, Memory, Storage, Network Self Provisioning Programmatic Control Elasticity Data vs. Response time Data and Traffic keeps on growing, but response time must maintain relatively constant Data Center must “scale out” Manageability High Availability “Green” Computing

The New Data Center Industry Container Computer for high efficiency and environmental conservation (Packaging, PUE, …) Bundled software (Cloud OS) for integrated service, high scalability, and availability Large Enterprise will bypass traditional server channels (IBM, HP, Dell, …) Purchase of entire data center directly from ODM manufacturers Significant cost reductions Horizontal scalability High Availability Google already directly purchase from Taiwan manufacturers Alll of the above leading a New Data Center Industry!!!

工研院雲端運算應用科技中心 CCMA@ITRI And that’s why CCMA is formed at ITRI

Mission Statement Deliver an end-to-end data center architecture know-how and a system software suite that will enable a cloud service provider to operate a mega data center that is the most efficient and capable in the world

Cloud Computing Food Chain Build Cloud Data Center the Google Way Hardware DataCenter Know-how Cloud OS 18

Container Computers 19

Data Center Architecture Know-how Treat the entire data center as a computer - Air flow analysis - Cooling architecture (thermal management) - Power/energy management - Focus on ease of system and network management - What cannot be managed/monitored does not get deployed Modular and Scalable (Card to Rack to Container to Warehouse) Explore low power, commodity CPU as a building block Google data center tour (http://www.google.com/corporate/green/datacenters/summit.html)

System Software (Cloud OS) Virtualization Platform CPUs Storage (Filesystems) Network Resource Management Provisioning of virtual clusters Physical machine load balancing Network traffic load balancing Power Management Security Hypervisor protection Compartmentalization between Clusters System Management FCAPS High Availability Physical component failure does not interrupt availability of virtual resources Cloud Applications management Physical Node CCMA Infrastructure SW VM Mail Virtual Cluster Bkup Virtual Cluster HC Virtual Cluster AppX Virtual Cluster

Cloud OS

What’s different about WSC’s? As computation continues to move into the cloud, the computing platform of interest no longer resembles a pizza box or a refrigerator, but a warehouse full of computers. These new large datacenters are quite different from traditional hosting facilities of earlier times and cannot be viewed simply as a collection of co-located servers. Large portions of the hardware and software resources in these facilities must work in concert to efficiently deliver good levels of Internet service performance, something that can only be achieved by a holistic approach to their design and deployment. In other words, we must treat the datacenter itself as one massive warehouse-scale computer (WSC). The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines May, 2009

Commodity Hardware-Only System Architecture Physical Server VM0 VM1 VMn Layer-3 Border Routers Layer-2-Only Data Center Network Server Load Balancer Cluster Compute Server Rack Storage Server

Architecture Prinicples Commodity Hardware A set of compute servers each equipped with homogenous multiple CPUs Requires CPU/memory/IO virtualization support A set of JBOD (just a bunch of disks) storage servers proportionally intermixed with the compute servers Low-power CPU is sufficient; RAID is optional A layer-2-only network connects all servers that consists of top-of-rack switches and core switches Everything is virtualized CPU, Memory, Storage, Network If a resource cannot be remotely managed, it should not be part of the CCMA data center

Software Stack for Cloud OS Cloud Application Management Tool Virtual Cluster Provisioning Network/System Management Physical Cluster Deployment Tool Physical Compute Servers Security Distributed Main/Secondary Storage All-layer-2 Network Intra-Virtual-Cluster Load Balancing Power Management Virtual Machine Management

Virtualization Platform Leverage existing hypervisors Allocation of virtual machine instances Monitor VM Performance Virtual storage provisioning Intra-VirtualCluster load balancing Scalable data center network Isolation between virtual clusters Virtual machine migration Physical Node Storage Server Mail Virtual Cluster Compute Nodes Bkup Virtual Cluster HC Virtual Cluster AppXYZ Virtual Cluster Data Nodes Service Nodes System Service daemons Cloud OS agents

Virtual Resource Provisioning Physical cluster deployment Virtual Cluster A group of VM’s providing same service, front-ended by a network load balancer Configuration Storage space requirement External network bandwidth requirement Load Balancing policy Firewall/IDS setting Network configuration, including DNS and DHCP OS image and application image Virtual Data Center One or more virtual cluster working in coordination (multi-tier web services, EMR’s, VDI’s, etc) Physical Machine Load Balancing Satisfy each virtual cluster’s performance requirement while minimizing the total amount of physical resource reserved

Virtual Storage Management Storage virtualization Service models Dedicated or Shared Volume Shared Filesystem Shared Database Distributed main storage Provides a global storage abstraction on a large number of distributed storage servers Distributed secondary storage Replication, Snapshot, Deduplication Unification of SAN and LAN: 10G Ethernet interconnect Each storage block in a disk volume remains available despite failure in switch, server, or disk drive Thin Provisioning Scales to a very large number of concurrent accesses

Cloud Storage System Architecture DMS DFS Metadata DFS Client VM Volume iSCSI Initiator iSCSI Target DFS DataNode DFS DataNode …

Networking in Cloud OS Scalable Load Balancer Cluster Layer 2 only Inter-VirtualCluster load balancing Each member of SLB cluster responsible for load balancing one or more VC’s Load balance based on current load on virtual machines Layer 2 only How to scale up to 100,000 physical servers with commodity Ethernet switches Load balance of Network packet routing Support for fast fail-over Pre-computed main and alternative routes Fast failure detection and re-routing Use Valiant load balancing to avoid congestion or bottlenecks

Layer-2-Only Data Center Network Network load balancing Server Server load balancing Fast failure detection and re-routing Valiant load balancing Network load balancing Server Fail Over Server Core (Layer 2 switch) Region (Layer 2 switch) Top Of Rack (Layer 2 switch) IP1, MAC1 Node #1 Node #2 Node #3 Node #4 Node #10 Node #20 Node #30 Node #40 Node #100 Node #200 Node #300 Node #400 IP2, MAC2 VM #1 VM #1 VM #1 VM #1 VM #1 VM #1 VM #1 VM #1 VM #1 VM #1 VM #1 VM #1 VM #2 VM #2 VM #2 VM #2 VM #2 VM #2 VM #2 VM #2 VM #2 VM #2 VM #2 VM #2 Compute Server Rack VM #3 VM #3 VM #3 VM #3 VM #3 VM #3 VM #3 VM #3 VM #3 VM #3 VM #3 VM #3 VM #24 VM #24 VM #24 VM #24 VM #24 VM #24 VM #24 VM #24 VM #24 VM #24 VM #24 VM #24

Virtual Machine Management Objective Power Management Physical Machine Load Balancing Monitor runtime VM statistics Heuristic calculation to predict workload for virtual clusters Determine power down/up of machines 2 dimensional bin packing VM migration algorithm Physical machine load balancing Migration of VM’s to other physical machine to balance out CPU and I/O load CONSIGNEE CONSIGNOR = PM to be turn off CONSIGNMNET = VM to be migrated

Fail-over & Load Balancing Virtual Machine Manager VM Die Monitor One VM die System is busy I am the new one! Hypervisor 1.1 Restart the dead VM 2.1 Migrate to meet load balancing

Security Multi-tenancy architecture Inter-virtual-cluster compartmentalization Works in the presence of constant VM motion Virtual appliance-based firewall and IDS/IPS Leverages open-source firewall/IDS/IPS technology Support for AAA, VPN, and standard access control

Container Computer Network Operating System System Management Leverages open-source network/system monitor tool and server configuration tool Discovery of comprehensive inter-service dependency map: How an arbitrary service depends on other services and in what temporal order Provides application-level performance monitoring support to cloud application management tool Comprehensive resource usage accounting for SLA or billing purpose Virtualization-aware, temperature aware and power-aware Container Computer Network Operating System Configuration CFENGINE Performance GANGLIA Fault MANTIS Security LDAP Accounting RADIUS SNMP IPMI Agent

Summary

Why do we need Cloud OS? Warehouse Style Computer (WSC) takes a holistic view of the entire data center to make sure it works as if it is a single server Cloud OS integrates server virtualization, storage virtualization, and network virtualization to provide: Resource management for Virtual Data Centers and Virtual Clusters Scalable Data Center Networking Load Balancing of Virtual Cluster, Network Traffic, and Physical Machines Ease of management for all Data Center resources Highly Available services End-to-end security and QoS guarantee Taiwan ODM manufacturers is uniquely positioned to take advantage of growth Data Center Industry due to Cloud Computing WSC will be used in both Public clouds and Private clouds Cloud OS will significantly enhance the value of WSC’s We are not building Hypervisor. We are not building Operating Systems. We are building a management layer and integrates virtualization of CPU, storage, and network. Our competitor is like vCloud and XenCloud, Cloudera, Eucalyptus, or VMOPs. Customer will start buying data centers instead of servers. Taiwan ODM is uniquely ready to position for that.

Q&A Thank you!