1. CCMA & Cloud OS
符儒嘉
工研院雲瑞運算中心
系统軟體組 組長 1

2. Agenda Introduction CCMA @ ITRI (工研院雲端運算行動應用科技中心 ) Cloud OS Summary
Virtual Data Center & Virtual Clusters
Virtualized Storage
Networking in a Cloud Data Center
Runtime Virtual Machine Management
Security
System Management
Summary
Introduction to Cloud Computing and what’s position of CCMA
Why we need Cloud OS and go into some details

3. Cloud Computing Definition
Provisioning of dynamically scalable and virtualized resources as a service over the Internet.
Multi-tenancy
Device & Location independence
Ability to obtain virtual computing resource on demand
Provides the Illusion of infinite computing resources
Self-Provisioning of virtual resources
Eliminates the need for up-front commitment by Cloud developers
Provides the ability to pay as you go for use of computing resources
Reliability, Scalability, Security, Manageability
Cloud Computing is pretty board in definition
CCMA focus on server side, specifically on how to build a Cloud Computing Center with Container Computer and Cloud OS projects
Allow multiple clients to get to their data from anywhere with no dependency on Client end device
Elasticity
Provisioning without reliance on IT
Pay as you go
Secure, Highly scalable, available, and manageable

4. Cloud Computing vs. Utility Services
電力供應
發電廠
超高壓變電所
一次變電所
二次變電所
配電變電所
家庭用戶
大規模場所 工廠 大樓
SaaS
Providers
PaaS
IaaS
雲端運算
End Users
ISVs 企業
資料中心
資料來源：IEK (2010/02)

5. Timing is right Technology Push Market Pull
Broadband network connectivity getting faster and more reliable
Internet service availability significantly improved
Sufficient trust in infrastructure providers
By many measures, Google is already a critical service for most of the world, and it is in the cloud!
Market Pull
Big Data
Software install on premise  Software as a service (SaaS)
Information technology (IT) on premise  IT service as a rented utility (as in electricity)
“IT should not and will not be a core competence for most corporations”
Nicholas Carr’s - “Does IT matter?” and “The Big Switch”
Lowering up-front and day-to-day IT cost: pay only as much as actual resource usage
Similar issues before – IT outsourcing; Application Service Provider
Why is this time different? Other than technology push, Market pull is also another reason
By Technology push, I mean…. Virtualizaiton, bandwidth, security enhancements (IDS/IPS, WAF, etc)
Market Pull,
data explosion-- 300,000 petabytes of public data produced in 2009 alone
Enterprise now seeing much more pain for DataCenter and IT cost
Prevalence of SaaS – People like Elasticity

6. Cost of Data Center
Power Usage

7. Types of Clouds Hybrid Cloud Service Providers Cloud Providers
Public Cloud
Service Users
Private Cloud
Service
Providers
Cloud End-User Services (SaaS)
Cloud Platform Services (PaaS)
Cloud Infrastructure Services (IaaS)
Cloud Providers
In 2010, IaaS, PaaS, and SaaS will no longer be just for Public Clouds.
Startup companies are Cloudera and VMOps are absolutely cashing in on this. VMOps just closed an $11M series B funding
Building a Computing Cloud is very similar to building a Data Center. You have to mind about SLA, storage, Networking, etc.
Except the cost for IaaS is much lower than buying HW to build the Data Center
There’s no reason why companies like Merrill Lynch, Walmart can’t operate their own data center like Cloud data centers.
Physical Infrastructure 7

8. Infrastructure as a Service
Example Players
Amazon
GoGrid
RightScale
Rackspace
VMOps
Eucalyptus
ElasticHosts …
Example Players

9. Platform as a Service Example Players Microsoft Azure
Google App Engine
Force.com
Rackspace Cloud
Heroku
QuickBase
Caspio …

10. Software as a Service Example Players SaleForce.com Adobe.com Autodesk
WebEx
Microsoft Office
Gmail & other Google Apps
Flicker …

11. DataCenter as a Computer
Majority of cloud computing infrastructure consists of reliable services delivered through data centers
Traditional colocation Datacenters
Multiple servers and communications gear collocated due to common environmental & security needs
Hosts a large number of relatively small or medium-sized applications, each running on a dedicated hardware infrastructure
Datacenters for Cloud Computing platform
Belongs to a single organization,
Uses a relatively homogeneous hardware and system software platform, and share a common system management layer.
Runs a smaller number of very large applications
Cloud computing workloads must be designed to gracefully tolerate large numbers of component faults with little or no impact on service level performance and availability.
Google and Amazon pretty successful Cloud Server providers. How they build their data center is quite different

12. Warehouse Scale Computers (WSC)
Not just a collection of servers
Hundreds to Thousands of servers running in concordance
Typically runs on a virtualized platform
Fault behavior & energy considerations have significant impact
Needs to be considered as a single unit
Must be highly manageable
Deployment of software updates
Monitoring & system management
Affordability
Currently power Public Cloud such as Google, Amazon, Yahoo, Microsoft’s, etc…
Soon to be affordable by Enterprises
A rack of servers can easily have > 600 cores
WSC is an concept coined by Google developers.
The other insteresting point is that this is not as expensive as you think. With the current architecture, 8-core CPU’s are coming out. A rack can contain roughly 30 1U servers, you can have a rack that contains close to 500 cores. The cost is not that high for a rack.
There’s no reason why big corporation can also start building their DataCenters this way
In other words, big Enterprise will start buying Data Centers, rather than buying servers

13. Google “Warehouse Style Computer” Data Center
This is already shrinking to a container (modular approach)

14. “Secret Sauce” of Cloud Computing
Commodity components
Virtualization
Servers, Memory, Storage, Network
Self Provisioning
Programmatic Control
Elasticity
Data vs. Response time
Data and Traffic keeps on growing, but response time must maintain relatively constant
Data Center must “scale out”
Manageability
High Availability
“Green” Computing

15. The New Data Center Industry
Container Computer for high efficiency and environmental conservation (Packaging, PUE, …)
Bundled software (Cloud OS) for integrated service, high scalability, and availability
Large Enterprise will bypass traditional server channels (IBM, HP, Dell, …)
Purchase of entire data center directly from ODM manufacturers
Significant cost reductions
Horizontal scalability
High Availability
Google already directly purchase from Taiwan manufacturers
Alll of the above leading a New Data Center Industry!!!

16. 工研院雲端運算應用科技中心 CCMA@ITRI
And that’s why CCMA is formed at ITRI

17. Mission Statement
Deliver an end-to-end data center architecture know-how and a system software suite that will enable a cloud service provider to operate a mega data center that is the most efficient and capable in the world

18. Cloud Computing Food Chain
Build Cloud Data Center the Google Way
Hardware
DataCenter
Know-how
Cloud OS 18

19. Container Computers 19

20. Data Center Architecture Know-how
Treat the entire data center as a computer - Air flow analysis - Cooling architecture (thermal management)
- Power/energy management - Focus on ease of system and network management - What cannot be managed/monitored does not get deployed
Modular and Scalable (Card to Rack to Container to Warehouse)
Explore low power, commodity CPU as a building block
Google data center tour (

21. System Software (Cloud OS)
Virtualization Platform
CPUs
Storage (Filesystems)
Network
Resource Management
Provisioning of virtual clusters
Physical machine load balancing
Network traffic load balancing
Power Management
Security
Hypervisor protection
Compartmentalization between Clusters
System Management
FCAPS
High Availability
Physical component failure does not interrupt availability of virtual resources
Cloud Applications management
Physical Node
CCMA Infrastructure SW VM
Mail Virtual Cluster
Bkup Virtual Cluster
HC Virtual Cluster
AppX Virtual Cluster

22. Cloud OS

23. What’s different about WSC’s?
As computation continues to move into the cloud, the computing platform of interest no longer resembles a pizza box or a refrigerator, but a warehouse full of computers. These new large datacenters are quite different from traditional hosting facilities of earlier times and cannot be viewed simply as a collection of co-located servers. Large portions of the hardware and software resources in these facilities must work in concert to efficiently deliver good levels of Internet service performance, something that can only be achieved by a holistic approach to their design and deployment. In other words, we must treat the datacenter itself as one massive warehouse-scale computer (WSC).
The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines
May, 2009

24. Commodity Hardware-Only System Architecture
Physical Server
VM0
VM1
VMn
Layer-3
Border
Routers
Layer-2-Only Data
Center Network
Server Load
Balancer
Cluster
Compute Server
Rack
Storage
Server

25. Architecture Prinicples
Commodity Hardware
A set of compute servers each equipped with homogenous multiple CPUs
Requires CPU/memory/IO virtualization support
A set of JBOD (just a bunch of disks) storage servers proportionally intermixed with the compute servers
Low-power CPU is sufficient; RAID is optional
A layer-2-only network connects all servers that consists of top-of-rack switches and core switches
Everything is virtualized
CPU, Memory, Storage, Network
If a resource cannot be remotely managed, it should not be part of the CCMA data center

26. Software Stack for Cloud OS
Cloud Application Management Tool
Virtual Cluster Provisioning
Network/System
Management
Physical
Cluster
Deployment
Tool
Physical Compute Servers
Security
Distributed Main/Secondary Storage
All-layer-2 Network
Intra-Virtual-Cluster
Load Balancing
Power
Management
Virtual Machine Management

27. Virtualization Platform
Leverage existing hypervisors
Allocation of virtual machine instances
Monitor VM Performance
Virtual storage provisioning
Intra-VirtualCluster load balancing
Scalable data center network
Isolation between virtual clusters
Virtual machine migration
Physical Node
Storage Server
Mail Virtual Cluster
Compute Nodes
Bkup Virtual Cluster
HC Virtual Cluster
AppXYZ Virtual Cluster
Data Nodes
Service Nodes
System Service daemons
Cloud OS agents

28. Virtual Resource Provisioning
Physical cluster deployment
Virtual Cluster
A group of VM’s providing same service, front-ended by a network load balancer
Configuration
Storage space requirement
External network bandwidth requirement
Load Balancing policy
Firewall/IDS setting
Network configuration, including DNS and DHCP
OS image and application image
Virtual Data Center
One or more virtual cluster working in coordination (multi-tier web services, EMR’s, VDI’s, etc)
Physical Machine Load Balancing
Satisfy each virtual cluster’s performance requirement while minimizing the total amount of physical resource reserved

29. Virtual Storage Management
Storage virtualization
Service models
Dedicated or Shared Volume
Shared Filesystem
Shared Database
Distributed main storage
Provides a global storage abstraction on a large number of distributed storage servers
Distributed secondary storage
Replication, Snapshot, Deduplication
Unification of SAN and LAN: 10G Ethernet interconnect
Each storage block in a disk volume remains available despite failure in switch, server, or disk drive
Thin Provisioning
Scales to a very large number of concurrent accesses

30. Cloud Storage System Architecture
DMS
DFS
Metadata
DFS
Client VM
Volume
iSCSI
Initiator
iSCSI
Target
DFS
DataNode
DFS
DataNode …

31. Networking in Cloud OS Scalable Load Balancer Cluster Layer 2 only
Inter-VirtualCluster load balancing
Each member of SLB cluster responsible for load balancing one or more VC’s
Load balance based on current load on virtual machines
Layer 2 only
How to scale up to 100,000 physical servers with commodity Ethernet switches
Load balance of Network packet routing
Support for fast fail-over
Pre-computed main and alternative routes
Fast failure detection and re-routing
Use Valiant load balancing to avoid congestion or bottlenecks

32. Layer-2-Only Data Center Network Network load balancing Server
Server load balancing
Fast failure detection and re-routing
Valiant load balancing
Network load balancing Server
Fail Over Server
Core (Layer 2 switch)
Region (Layer 2 switch)
Top Of Rack (Layer 2 switch)
IP1, MAC1
Node #1
Node #2
Node #3
Node #4
Node #10
Node #20
Node #30
Node #40
Node #100
Node #200
Node #300
Node #400
IP2, MAC2
VM #1
VM #1
VM #1
VM #1
VM #1
VM #1
VM #1
VM #1
VM #1
VM #1
VM #1
VM #1
VM #2
VM #2
VM #2
VM #2
VM #2
VM #2
VM #2
VM #2
VM #2
VM #2
VM #2
VM #2
Compute Server
Rack
VM #3
VM #3
VM #3
VM #3
VM #3
VM #3
VM #3
VM #3
VM #3
VM #3
VM #3
VM #3
VM #24
VM #24
VM #24
VM #24
VM #24
VM #24
VM #24
VM #24
VM #24
VM #24
VM #24
VM #24

33. Virtual Machine Management
Objective
Power Management
Physical Machine Load Balancing
Monitor runtime VM statistics
Heuristic calculation to predict workload for virtual clusters
Determine power down/up of machines
2 dimensional bin packing
VM migration algorithm
Physical machine load balancing
Migration of VM’s to other physical machine to balance out CPU and I/O load
CONSIGNEE
CONSIGNOR
= PM to be turn off
CONSIGNMNET
= VM to be migrated

34. Fail-over & Load Balancing
Virtual Machine Manager VM
Die
Monitor
One VM die
System is busy
I am the new one!
Hypervisor
1.1 Restart the dead VM
2.1 Migrate to meet load balancing

35. Security Multi-tenancy architecture
Inter-virtual-cluster compartmentalization
Works in the presence of constant VM motion
Virtual appliance-based firewall and IDS/IPS
Leverages open-source firewall/IDS/IPS technology
Support for AAA, VPN, and standard access control

36. Container Computer Network Operating System
System Management
Leverages open-source network/system monitor tool and server configuration tool
Discovery of comprehensive inter-service dependency map: How an arbitrary service depends on other services and in what temporal order
Provides application-level performance monitoring support to cloud application management tool
Comprehensive resource usage accounting for SLA or billing purpose
Virtualization-aware, temperature aware and power-aware
Container Computer Network Operating System
Configuration
CFENGINE
Performance
GANGLIA
Fault
MANTIS
Security
LDAP
Accounting
RADIUS
SNMP
IPMI Agent

37. Summary

38. Why do we need Cloud OS?
Warehouse Style Computer (WSC) takes a holistic view of the entire data center to make sure it works as if it is a single server
Cloud OS integrates server virtualization, storage virtualization, and network virtualization to provide:
Resource management for Virtual Data Centers and Virtual Clusters
Scalable Data Center Networking
Load Balancing of Virtual Cluster, Network Traffic, and Physical Machines
Ease of management for all Data Center resources
Highly Available services
End-to-end security and QoS guarantee
Taiwan ODM manufacturers is uniquely positioned to take advantage of growth Data Center Industry due to Cloud Computing
WSC will be used in both Public clouds and Private clouds
Cloud OS will significantly enhance the value of WSC’s
We are not building Hypervisor. We are not building Operating Systems.
We are building a management layer and integrates virtualization of CPU, storage, and network. Our competitor is like vCloud and XenCloud, Cloudera, Eucalyptus, or VMOPs.
Customer will start buying data centers instead of servers. Taiwan ODM is uniquely ready to position for that.

39. Q&A Thank you!