Presentation is loading. Please wait.

Presentation is loading. Please wait.

Data Center Scale Computing

Similar presentations

Presentation on theme: "Data Center Scale Computing"— Presentation transcript:

1 Data Center Scale Computing
If computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility The computer utility could become the basis of a new and important industry. John McCarthy MIT centennial celebration (1961) Presentation by: Ken Bakke Samantha Orogvany John Greene

2 Outline Introduction Data Center System Components
Design and Storage Considerations Data Center Power supply Data Center Cooling Data center failures and fault tolerances Data center repairs Current challenges current research, trends, etc Conclusion

3 Data Center VS Warehouse Scale Computer
Provide colocated equipment Consolidate heterogeneous computers Serve wide variety of customers Binaries typically run on a small number of computers Resources are partitioned and separately managed Facility and computing resources are designed separately Share security, environmental and maintenance resources Warehouse-scale computer Designed to run massive internet applications Individual applications run on thousands of computers Homogeneous hardware and system software Central management for a common resource pool The design of the facility and the computer hardware is integrated Data centers typically host a wide variety of small to medium sized applications each on dedicated hardware. These applications are decoupled and protected from each other. WSC belong to a single organization, use homogeneous hardware, systems software and a common management infrastructure. They are focused on cost efficiency.

4 Need for Warehouse-scale Computers
Renewed focus on client-side consumption of web resources Constantly increasing numbers of web users Constantly expanding amounts of information Desire for rapid response for end user Focus on cost reduction delivering massive applications. Increased interest in Infrastructure as a Service (Iaas)

5 Performance and Availability Techniques
Replication Reed-Solomon codes Sharding Load-balancing Health checking Application specific compression Eventual consistency Centralized control Canaries Redundant execution and tail tolerance

6 Major system components
Typical server is 4 CPU - 8 Dual threaded cores yielding 32 cores Typical rack - 40 servers & 1 or 10 Gbps ethernet switch Cluster containing cluster switch and racks A cluster may contain tens of thousands of processing threads

7 Low-end Server vs SMP Latency 1000 time faster in SMP
Less impact on applications too large for single server Performance advantage of a cluster built with large SMP server nodes (128-core SMP) over a cluster with the same number of processor cores built with low-end server nodes (four-core SMP), for clusters of varying size.

8 Brawny vs Wimpy Advantages of wimpy computers
Multicore CPUs carry a premium cost of 2-5 times vs multiple smaller CPUs Memory and IO bound applications do not take advantage of faster CPUs Slower CPUs are more power efficient Disadvantages of wimpy computer Increasing parallelism is programmatically difficult Programming costs increase Networking requirements increase Less tasks / smaller size creates loading difficulties Amdahl’s law impacts

9 Design Considerations
Software design and improvements can be made to align with architectural choices Resource requirements and utilization can be balanced among all applications Spare CPU cycles can be used for process intensive applications Spare storage can be used for archival purposes Fungible resources are more efficient Workloads can be distributed to fully utilize servers Focus on cost-effectiveness Smart programmers may be able to restructure algorithms to match a more inexpensive design.

10 Storage Considerations
Private Data Local DRAM, SSD or Disk Shared State Data High throughput for thousands of users Robust performance tolerant to errors Unstructure Storage - (Google - GFS) Master plus thousnads of “chunk” servers Utilizes every system with a disk drive Cross machine replication Structured Storage Big Table provides Row, Key, Timestamp mapping to byte array Trade-offs favor high performance and massive availability Eventual consistency model leaves applications managing consistency issues

11 Google File System

12 WSC Network Architecture
Leaf Bandwidth Bandwidth between servers in common rack Typically managed with a commodity switch Easily increased by increasing number of ports or speed of ports Bisection Bandwidth Bandwidth between the two halves of a cluster Matching leaf bandwidth requires as many uplinks to fabric as links within a rack Since distances are longer, optical interfaces are required.

13 Three Stage Topology Required to maintain same throughput as single switch.

14 Network Design Oversubscription ratios of 4-10 are common.
Limit network cost per server Offloading to special networks Centralized management

15 Service level response times
Consider servers with 99th, 99.9th and 99.99th latency > 1s vs # required service requests Selective replication is one mitigating strategy


17 Power Supply Distribution
Uninterruptible Power Systems Transfer switch used to chose active power input from either utility sources or generator After a power failure, the transfer switch will detect the power generator and after seconds, provide power This power system has energy storage to provide additional protection between power failure of main utility power and when generators begin providing full load Levels incoming power feed to remove spikes and lags from AC-feed

18 Example of Power Distribution Units
Traditional PDU Takes in power output from UPS Regulates power with transformers to distribute power to servers Handles kW typically Provides Redundancy by switching between 2 power sources

19 Examples of Power Distribution
Facebook’s power distribution system Designed to increase power efficiency by reducing energy loss to about 15% Eliminates the UPS and PDU and adds on-board 12v battery for each cabinet

20 Power Supply Cooling Needs
Air Flow Consideration Fresh Air cooling “Opening the windows” Closed loop system Underfloor systems Servers are on raised concrete tile floors

21 Power Cooling Systems 2-loop Systems
Loop 1 - Hot Air/Cool air circuit (Red/Blue Arrows) Loop 2 - Liquid supply to Computer Room Air Conditioning Units and heat discharging

22 Example of Cooling System Design
3 - Loop System Chiller sends cooled water to CRACs Heated water sent from building to chiller for heat dispersal Condenser water loop flows into cooling tower

23 Cooling System for Google

24 Estimated Annual Costs

25 Estimated Carbon Costs for Power
Based on local utility power generated via the use of oil, natural gas, coal or renewable sources, including hydroelectricity, solar energy, wind and biofuels

26 Power Efficiency Sources of Efficiency Loss Improvements to Efficiency
Overheading cooling systems, such as chillers Air movement IT Equipment Power distribution unit Improvements to Efficiency Handling air flow more carefully. Keep cooling path short and separate hot air from servers from system Consider raising cooling temperatures Employ “free cooling” by locating datacenter in cooler climates Select more efficient power system

27 Data Center Failures Reliability of Data Center Fault Tolerances
Trade off between cost of failures, along with repairing, and preventing failures. Fault Tolerances Traditional servers require high degree of reliability and redundancy to prevent failures as much as possible For data warehouses, this is not practical Example: a cluster of 10,000 servers will have an average of 1 server failure/day

28 Data Center Failures Fault Severity Categories Corrupted
Data is lost, corrupted, or cannot be regenerated Unreachable Service is down Degraded Service is available, but limited Masked Faults occur but due to fault tolerance, this is masked from user

29 Data Center Fault Causes
Software errors Faulty configs Human Error Networking faults Faulty hardware It’s easier to tolerate known hardware issues than software bugs or human error. Repairs It’s not critical to quickly repair individual servers In reality, repairs are scheduled as a ‘daily sweep’ Individual failures mostly do not affect overall data center health System is designed to tolerate faults

30 Google Restarts and Downtime

31 Relatively New Class of Computers
Facebook founded in 2004 Google’s Modular Data Center in 2005 Microsoft’s Online Services Division in 2005 Amazon Web Services in 2006 Netflix added streaming in 2007

32 Balanced System Nature of workload at this scale is:
Large volume Large variety Distributed This means no servers (or parts of servers) get to slack while others do the work. Keep servers busy to amortize cost Need high performance from all components!

33 Imbalanced Parts Latency lags bandwidth
Figure from John L. Hennessy, David A. Patterson Computer Architecture, Fifth Edition A Quantitative Approach

34 Imbalanced Parts CPUs have been historical focus
Figures from MSDN blog article, Background and Engineering the Windows 7 Improvements

35 Focus Needs to Shift Push toward SaaS will highlight these disparities
Requires concentrating research: Improving non-CPU components Improving responsiveness Improving end-to-end experience

36 Why does latency matter?
Responsiveness dictated by latency Productivity affected by responsiveness Figure from John L. Hennessy, David A. Patterson Computer Architecture, Fifth Edition A Quantitative Approach

37 Real Estate Considerations
Land Power Cooling Taxes Population Disasters Image from Facebook’s announcement of its new Iowa DC in April 2013 (

38 Google’s Data Centers Pulled from

39 Economical Efficiency
DC is non-trivial cost Does not include land Servers is bigger cost More servers desirable Busy servers desirable Chart pulled from

40 Improving Efficiency Better components Power-saving modes
Energy proportional (less use == less energy) Power-saving modes Transparent (e.g., clock-gating) Active (e.g., CPU throttling) Inactive (e.g., idle drives stop spinning)

41 Changing Workloads Workloads more agile in nature
SaaS Shorter release cycles Office 365 updates several times per year Some Google services update weekly Even major software gets rewritten Google search engine re-written from scratch 4 times Internet services are still young Usage can be unpredictable

42 YouTube Started in 2005 Fifth most popular site within first year
Images from

43 Image from Sandvine via Mashable

44 Adapting Strike balance of need to deploy with longevity
Need it fast and good Design to make software easy to create Easier to find programmers Redesign when warranted Google Search’s rewrites removed inefficiencies Contrast to Intel’s backwards compatibility spanning decades

45 Future Trends Continued emphasis on:
Parallelism Networking, both within and to/from datacenters Reliability via redundancy Optimizing efficiency (energy proportionality) Environmental impact Energy costs Amdahl’s law will remain major factor Need increased focus on end-to-end systems Computing as a utility?

46 “Anyone can build a fast CPU. The trick is to build a fast system.”
-Seymour Cray

47 “Anyone can build a fast CPU. The trick is to build a fast system.”
-Seymour Cray

Download ppt "Data Center Scale Computing"

Similar presentations

Ads by Google