1. The University of Adelaide, School of Computer Science
Computer Architecture
A Quantitative Approach, Fifth Edition
The University of Adelaide, School of Computer Science
24 November 2018
Chapter 6
Warehouse-Scale Computers to Exploit Request-Level and Data-Level Parallelism:
Copyright © 2012, Elsevier Inc. All rights reserved.
Chapter 2 — Instructions: Language of the Computer

2. The University of Adelaide, School of Computer Science
24 November 2018
Introduction
Introduction
Warehouse-scale computer (WSC)
Provides Internet services
Search, social networking, online maps, video sharing, online shopping, , cloud computing, etc.
Differences with HPC “clusters”:
Clusters have higher performance processors and network
Clusters emphasize thread-level parallelism, WSCs emphasize request-level parallelism
Differences with datacenters:
Datacenters consolidate different machines and software into one location
Datacenters emphasize virtual machines and hardware heterogeneity in order to serve varied customers
Copyright © 2012, Elsevier Inc. All rights reserved.
Chapter 2 — Instructions: Language of the Computer

3. The University of Adelaide, School of Computer Science
24 November 2018
Introduction
Introduction
Important design factors for WSC:
Cost-performance
Small savings add up
Energy efficiency
Affects power distribution and cooling
Work per joule
Dependability via redundancy
Network I/O
Interactive and batch processing workloads
Ample computational parallelism is not important
Most jobs are totally independent
“Request-level parallelism”
Operational costs count
Power consumption is a primary, not secondary, constraint when designing system
Scale and its opportunities and problems
Can afford to build customized systems since WSC require volume purchase
Copyright © 2012, Elsevier Inc. All rights reserved.
Chapter 2 — Instructions: Language of the Computer

4. Prgrm’g Models and Workloads
The University of Adelaide, School of Computer Science
24 November 2018
Prgrm’g Models and Workloads
Batch processing framework: MapReduce
Map: applies a programmer-supplied function to each logical input record
Runs on thousands of computers
Provides new set of key-value pairs as intermediate values
Reduce: collapses values using another programmer-supplied function
Programming Models and Workloads for WSCs
Copyright © 2012, Elsevier Inc. All rights reserved.
Chapter 2 — Instructions: Language of the Computer

5. Prgrm’g Models and Workloads
The University of Adelaide, School of Computer Science
24 November 2018
Prgrm’g Models and Workloads
Example:
map (String key, String value):
// key: document name
// value: document contents
for each word w in value
EmitIntermediate(w,”1”); // Produce list of all words
reduce (String key, Iterator values):
// key: a word
// value: a list of counts
int result = 0;
for each v in values:
result += ParseInt(v); // get integer from key-value pair
Emit(AsString(result));
Programming Models and Workloads for WSCs
Copyright © 2012, Elsevier Inc. All rights reserved.
Chapter 2 — Instructions: Language of the Computer

6. Prgrm’g Models and Workloads
The University of Adelaide, School of Computer Science
24 November 2018
Prgrm’g Models and Workloads
MapReduce runtime environment schedules map and reduce task to WSC nodes
Availability:
Use replicas of data across different servers
Use relaxed consistency:
No need for all replicas to always agree
Workload demands
Often vary considerably
Programming Models and Workloads for WSCs
Copyright © 2012, Elsevier Inc. All rights reserved.
Chapter 2 — Instructions: Language of the Computer

7. Computer Architecture of WSC
The University of Adelaide, School of Computer Science
24 November 2018
Computer Architecture of WSC
WSC often use a hierarchy of networks for interconnection
Each 19” rack holds 48 1U servers connected to a rack switch
Rack switches are uplinked to switch higher in hierarchy
Uplink has 48 / n times lower bandwidth, where n = # of uplink ports
“Oversubscription”
Goal is to maximize locality of communication relative to the rack
Computer Ar4chitecture of WSC
Copyright © 2012, Elsevier Inc. All rights reserved.
Chapter 2 — Instructions: Language of the Computer

8. The University of Adelaide, School of Computer Science
24 November 2018
Storage
Storage options:
Use disks inside the servers, or
Network attached storage through Infiniband
WSCs generally rely on local disks
Google File System (GFS) uses local disks and maintains at least three relicas
Computer Ar4chitecture of WSC
Copyright © 2012, Elsevier Inc. All rights reserved.
Chapter 2 — Instructions: Language of the Computer

9. The University of Adelaide, School of Computer Science
24 November 2018
Array Switch
Switch that connects an array of racks
Array switch should have 10 X the bisection bandwidth of rack switch
Cost of n-port switch grows as n2
Often utilize content addressible memory chips and FPGAs
Computer Ar4chitecture of WSC
Copyright © 2012, Elsevier Inc. All rights reserved.
Chapter 2 — Instructions: Language of the Computer

10. The University of Adelaide, School of Computer Science
24 November 2018
WSC Memory Hierarchy
Servers can access DRAM and disks on other servers using a NUMA-style interface
Computer Ar4chitecture of WSC
Copyright © 2012, Elsevier Inc. All rights reserved.
Chapter 2 — Instructions: Language of the Computer

11. Infrastructure and Costs of WSC
The University of Adelaide, School of Computer Science
24 November 2018
Infrastructure and Costs of WSC
Location of WSC
Proximity to Internet backbones, electricity cost, property tax rates, low risk from earthquakes, floods, and hurricanes
Power distribution
Physcical Infrastrcuture and Costs of WSC
Copyright © 2012, Elsevier Inc. All rights reserved.
Chapter 2 — Instructions: Language of the Computer

12. Infrastructure and Costs of WSC
The University of Adelaide, School of Computer Science
24 November 2018
Infrastructure and Costs of WSC
Cooling
Air conditioning used to cool server room
64 F – 71 F
Keep temperature higher (closer to 71 F)
Cooling towers can also be used
Minimum temperature is “wet bulb temperature”
Physcical Infrastrcuture and Costs of WSC
Copyright © 2012, Elsevier Inc. All rights reserved.
Chapter 2 — Instructions: Language of the Computer

13. Infrastructure and Costs of WSC
The University of Adelaide, School of Computer Science
24 November 2018
Infrastructure and Costs of WSC
Cooling system also uses water (evaporation and spills)
E.g. 70,000 to 200,000 gallons per day for an 8 MW facility
Power cost breakdown:
Chillers: % of the power used by the IT equipment
Air conditioning: % of the IT power, mostly due to fans
How man servers can a WSC support?
Each server:
“Nameplate power rating” gives maximum power consumption
To get actual, measure power under actual workloads
Oversubscribe cumulative server power by 40%, but monitor power closely
Physcical Infrastrcuture and Costs of WSC
Copyright © 2012, Elsevier Inc. All rights reserved.
Chapter 2 — Instructions: Language of the Computer

14. Measuring Efficiency of a WSC
The University of Adelaide, School of Computer Science
24 November 2018
Measuring Efficiency of a WSC
Power Utilization Effectiveness (PEU)
= Total facility power / IT equipment power
Median PUE on 2006 study was 1.69
Performance
Latency is important metric because it is seen by users
Bing study: users will use search less as response time increases
Service Level Objectives (SLOs)/Service Level Agreements (SLAs)
E.g. 99% of requests be below 100 ms
Physcical Infrastrcuture and Costs of WSC
Copyright © 2012, Elsevier Inc. All rights reserved.
Chapter 2 — Instructions: Language of the Computer

15. The University of Adelaide, School of Computer Science
24 November 2018
Cost of a WSC
Capital expenditures (CAPEX)
Cost to build a WSC
Operational expenditures (OPEX)
Cost to operate a WSC
Physcical Infrastrcuture and Costs of WSC
Copyright © 2012, Elsevier Inc. All rights reserved.
Chapter 2 — Instructions: Language of the Computer

16. The University of Adelaide, School of Computer Science
24 November 2018
Cloud Computing
Cloud Computing
WSCs offer economies of scale that cannot be achieved with a datacenter:
5.7 times reduction in storage costs
7.1 times reduction in administrative costs
7.3 times reduction in networking costs
This has given rise to cloud services such as Amazon Web Services
“Utility Computing”
Based on using open source virtual machine and operating system software
Copyright © 2012, Elsevier Inc. All rights reserved.
Chapter 2 — Instructions: Language of the Computer

17. Distributed and Cloud Computing Cloud Platform Architecture over Virtualized Datacenters

18. Public, Private & Hybrid Clouds

19. Public Clouds vs. Private Clouds :
Characteristics
Public clouds
Private clouds
Technology leverage and ownership
Owned by service providers
Leverage existing IT infrastructure and personnel; owned by individual organization
Management of provisioned resources
Creating and managing VM instances within proprietary infrastructure; promote standardization, preserves capital investment, application flexibility
Client managed; achieve customization and offer higher efficiency
Workload distribution methods and loading policies
Handle workload without communication dependency; distribute data and VM resources; surge workload is off-loaded
Handle workload dynamically, but can better balance workloads; distribute data and VM resources
Security and data privacy enforcement
Publicly accessible through remote interface
Access is limited; provide pre-production testing and enforce data privacy and security policies
Example platforms
Google App Engine, Amazon AWS, Microsoft Azure
IBM RC2

20. Cost-Effectiveness in Cloud Computing vs. Datacenter Utilization
(Courtesy of M. Ambrust, et al 2009)

23. Infrastructure as a service (IaaS)
Most basic cloud service model
Cloud providers offer computers, as physical or more often as virtual machines, and other resources.
Virtual machines are run as guests by a hypervisor, such as Xen or KVM.
Cloud users deploy their applications by then installing operating system images on the machines as well as their application software.
Cloud providers typically bill IaaS services on a utility computing basis, that is, cost will reflect the amount of resources allocated and consumed.
Examples of IaaS include: Amazon CloudFormation (and underlying services such as Amazon EC2), Rackspace Cloud, Terremark, and Google Compute Engine.

24. Some IaaS Offerings from Public Clouds :

25. Platform as a service (PaaS)
Cloud providers deliver a computing platform typically including operating system, programming language execution environment, database, and web server.
Application developers develop and run their software on a cloud platform without the cost and complexity of buying and managing the underlying hardware and software layers.
Examples of PaaS include: Amazon Elastic Beanstalk, Cloud Foundry, Heroku, Force.com, EngineYard, Mendix, Google App Engine, Microsoft Azure and OrangeScape.

26. PaaS Offerings from Public Clouds

27. Software as a service (SaaS)
Cloud providers install and operate application software in the cloud and cloud users access the software from cloud clients.
The pricing model for SaaS applications is typically a monthly or yearly flat fee per user, so price is scalable and adjustable if users are added or removed at any point.
Examples of SaaS include: Google Apps, innkeypos, Quickbooks Online, Limelight Video Platform, Salesforce.com, and Microsoft Office 365.

28. Warehouse-Scale Computer (WSC)
The University of Adelaide, School of Computer Science
24 November 2018
Warehouse-Scale Computer (WSC)
Provides Internet services
Search, social networking, online maps, video sharing, online shopping, , cloud computing, etc.
Differences with HPC “clusters”:
Clusters have higher performance processors and network
Clusters emphasize thread-level parallelism, WSCs emphasize request-level parallelism
Differences with datacenters:
Datacenters consolidate different machines and software into one location
Datacenters emphasize virtual machines and hardware heterogeneity in order to serve varied customers
(Courtesy of Hennessy and Patterson, 2012)
Chapter 2 — Instructions: Language of the Computer 28

29. Design Considerations for WSC:
The University of Adelaide, School of Computer Science
24 November 2018
Design Considerations for WSC:
Cost-performance
Small savings add up
Energy efficiency
Affects power distribution and cooling
Work per joule
Dependability via redundancy
Network I/O
Interactive and batch processing workloads
Ample computational parallelism is not important
Most jobs are totally independent
“Request-level parallelism”
Operational costs count
Power consumption is a primary constraint when designing system
Scale and its opportunities and problems
Can afford customized systems since WSC require volume purchase
(Courtesy of Hennessy and Patterson, 2012)
Chapter 2 — Instructions: Language of the Computer 29

30. Typical Datacenter Layout

31. Power and Cooling Requirements
The University of Adelaide, School of Computer Science
24 November 2018
Cooling system also uses water (evaporation and spills)
E.g. 70,000 to 200,000 gallons per day for an 8 MW facility
Power cost breakdown:
Chillers: % of the power used by the IT equipment
Air conditioning: % of the IT power, mostly due to fans
(Courtesy of Hennessy and Patterson, 2012)
Chapter 2 — Instructions: Language of the Computer 31

32. Power and Cooling Requirements(2)
How many servers can a WSC support?
Each server:
“Nameplate power rating” gives maximum power consumption
To get actual, measure power under actual workloads
Oversubscribe cumulative server power by 40%, but monitor power closely
Copyright © 2012, Elsevier Inc. All rights reserved.

33. (Courtesy of Luiz Andre Barroso and Urs Holzle, Google Inc., 2009)

34. Measuring Efficiency of a WSC
The University of Adelaide, School of Computer Science
24 November 2018
Measuring Efficiency of a WSC
Power Utilization Effectiveness (PEU)
= Total facility power / IT equipment power
Median PUE on 2006 study was 1.69
Performance
Latency is important metric because it is seen by users
Bing study: users will use search less as response time increases
Service Level Objectives (SLOs)/Service Level Agreements (SLAs)
E.g. 99% of requests be below 100 ms
(Courtesy of Hennessy and Patterson, 2012)
Chapter 2 — Instructions: Language of the Computer 34

35. Modular Data Center

36. The University of Adelaide, School of Computer Science
Cloud Computing
24 November 2018
WSCs offer economies of scale that cannot be achieved with a datacenter:
5.7 times reduction in storage costs
7.1 times reduction in administrative costs
7.3 times reduction in networking costs
This has given rise to cloud services such as Amazon Web Services
“Utility Computing”
Based on using open source virtual machine and operating system software
(Courtesy of Hennessy and Patterson, 2012)
Chapter 2 — Instructions: Language of the Computer 36

37. Enabling Technologies for The Clouds

38. Cloud Computing as A Service
[9]

40. Virtualized servers, storage , and network for cloud platform construction

42. Challenges/Issues in Cloud Computing42

43. Challenges in Cloud Computing (1)
Concerns from The Industry (Providers)
Replacement Cost
Exponential increase in cost to maintain the infrastructure
Vendor Lock-in
No standard API or protocol can be very serious
Standardization
No standard metric for QoS is limiting the popularity
Security and Confidentiality
Trust model for cloud computing
Control Mechanism
Users do not have any control over infrastructures

44. Challenges in Cloud Computing (2)
Concerns from Research Community :
Conflict to legacy programs
With difficulty in developing a new application due to lack of control
Provenance
How to reproduce results in different infrastructures
Reduction in Latency
No specially designed interconnect used
Very low controllability in layout of interconnect due to abstraction
Programming Model
Hard to debug where programming naturally error-prone
Details about infrastructure are hidden
QoS Measurement
Especially for ubiquitous computing where context changes

45. Public Clouds and Service Offerings

47. Platform as a Service (PaaS): Google App Engine
This platform allows users to develop and host web application in Google datacenters with automatic scaling according to the demand.
It is a free service for a certain limit and it only requires a Gmail account to access the services. After the free limit is exceeded the customers are charged for additional storage, bandwidth and instance hours.
The current version supports Java, Python and Go as the programming languages and Google plans to add more languages in the future.
All billed App Engine applications have a 99.95% uptime SLA. App Engine is designed to sustain multiple datacenter outages without any downtime.
The app engine has a few restrictions - can only execute code called from an HTTP request, Java applications may only use a subset from the JRE standard edition and Java application cannot create new threads.

48. Google AppEgine (GAE)

50. AWS – a leader in providing public IaaS services.
EC2 (Elastic compute cloud allows users to rent virtual computers to run their own computer applications. It allows scalable deployment. A user can create, launch, and terminate server instances as needed, paying by the hour for active servers.
S3 (simple storage service) provides the object-oriented storage service for users.
EBS (Elastic block service) provides the block storage interface which can be used to support traditional applications.
Amazon DevPay is a simple to use online billing and account management service that makes it easy for businesses
MPI clusters uses hardware-assisted virtualization instead of para-virtualization and users are free to create a new AMIs
AWS import/export allows one to ship large volumes of data to and from EC2 by shipping physical discs.

51. Copyright © 2012, Elsevier Inc. All rights reserved.
AWS – a leader in providing public IaaS services
Brokering systems offer a striking model for controlling sensors and providing office support of smartphones and tablets.
Small-business companies can put their business on the Amazon cloud platform. Using AWS they can service a large number of internet users and make profits through those paid services.
Copyright © 2012, Elsevier Inc. All rights reserved.

52. Amazon Web Services (AWS)

53. Amazon’s Lesson Down for 3 days since 4/22/2011
1000x of businesses went offline. E.g. Pfizer, Netflix, Quora, Foursquare,Reddit
SLA contract
99.95% availability (<4.5hour down）
10% penalty, otherwise
SLA = Service Level Agreement 53

54. Microsoft Azure Cloud :
This is essentially a PaaS Cloud.
Windows Azure run its cluster hosted at Microsoft's datacenters that manages computing and storage resources.
One can download Azure development kit to run a local version of Azure. It allows Azure applications to be developed and debugged one the windows 7 hosts.
All cloud services can interact with traditional MS software applications such as Windows Live, Office Live, Exchange Online, etc.
If offers a Windows-based cloud platform using Microsoft virtualization technology.
Applications are built on VM’s deployed on the data-center services.
Azure manages all servers, storage and network resources of the data center.

55. Microsoft Windows Azure

56. Cloud Services and Major Providers
Cloud applications are the key global opportunity for developers during next four years
2009: $8.5 billion
2013: $16.8 billion

58. Security and Trust Barriers in Cloud Computing
Protecting datacenters must first secure cloud resources and uphold user privacy and data integrity.
Trust overlay networks could be applied to build reputation systems for establishing the trust among interactive datacenters.
A watermarking technique is suggested to protect shared data objects and massively distributed software modules.
These techniques safeguard user authentication and tighten the data access-control in public clouds.
The new approach could be more cost-effective than using the traditional encryption and firewalls to secure the clouds. 58

59. Security Aware Cloud Platform

60. Cloud Service Models & Security Measures