1. Cloud Service Infrastructure
ECE7650

2. Key Components Datacenter Architecture Cloud Management SW
Cluster: compute, storage
Network for DC
Storage
Virtualization
Cloud Management SW
Resource Usage Metering
Automated Systems Management
Privacy and Security Measures
Cloud Programming Env

3. What’s a Cluster?
Broadly, a group of networked autonomous computers that work together to form a single machine in many respects:
To improve performance (speed)
To improve throughout
To improve service availability (high-availability clusters)
Based on commercial off-the-shelf, the system is often more cost-effective than single machine with comparable speed or availability

4. Highly Scalable Clusters
High Performance Cluster (aka Compute Cluster)
A form of parallel computers, which aims to solve problems faster by using multiple compute nodes.
For parallel efficiency, the nodes are often closely coupled in a high throughput, low-latency network
Server Cluster and Datacenter
Aims to improve the system’s throughput , service availability, power consumption, etc by using multiple nodes

5. Top500 Installation of Supercomputers
Top500.com

6. Clusters in Top500

7. An Example of Top500 Submission (F’08)
Location
Tukwila, WA
Hardware – Machines
256 Dual-CPU, quad-core Intel 5320 Clovertown 1.86GHz CPU and 8GB RAM
Hardware – Networking
Private & Public:  Broadcom GigE
MPI:  Cisco Infiniband SDR, 34 IB switches in leaf/node configuration
Number of Compute Nodes
256
Total Number of Cores
2048
Total Memory
2 TB of RAM
Particulars of for current Linpack Runs
Best Linpack Result
11.75 TFLOPS
Best Cluster Efficiency
77.1%
For Comparison…
Linpack rating from June2007 Top500 run (#106) on the same hardware
8.99 TFLOPS
Cluster efficiency from June2007 Top500 run (#106) on the same hardware
59%
Typical Top500 efficiency for Clovertown motherboards w/ IB regardless of Operating System
65-77% 
(2 instances of 79%)
30% impro in efficiency on the same hardware; about one hour to deplay
© 2004 Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

8. Beowulf Cluster
A cluster of inexpensive PCs for low-cost personal supercomputing
Based on commodity off-the-shelf components:
PC computers running a Unix-like Os (BSD, Linux, or OpenSolaris)
Interconnected by an Ethernet LAN
Head node, plus a group of compute node
Head node controls the cluster, and serves files to the compute nodes
Standard, free and open source software
Programming in MPI
MapReduce

9. Why Clustering Today Powerful node (cpu, mem, storage)
Today’s PC is yesterday’s supercomputers
Multi-core processors
High speed network
Gigabit (56% in top500 as of Nov 2008)
Infiniband System Area Network (SAN) (24.6%)
Standard tools for parallel/ distributed computing & their growing popularity.
MPI, PBS, etc
MapReduce for data-intensive computing

10. Major issues in Cluster Design
Programmability
Sequential vs Parallel Programming
MPI, DSM, DSA: hybrid of multithreading and MPI
MapReduce
Cluster-aware Resource management
Job scheduling (e.g. PBS)
Load balancing, data locality, communication opt, etc
System management
Remote installation, monitoring, diagnosis,
Failure management, power management, etc

11. Cluster Architecture
Multi-core node architecture
Cluster Interconnect

12. Single-core computer

13. Single-core CPU chip
the single core

14. Multicore Architecture
Combine 2 or more independent cores (normally CPU) into a single package
Support multitasking and multithreading in a single physical package

15. Multicore is Everywhere
Dual-core commonplace in laptops
Quad-core in desktops
Dual quad-core in servers
All major chip manufacturers produce multicore CPUs
SUN Niagara (8 cores, 64 concurrent threads)
Intel Xeon (6 cores)
AMD Opteron (4 cores)

16. Multithreading on multi-core
David Geer, IEEE Computer, 2007

17. Interaction with the OS
OS perceives each core as a separate processor
OS scheduler maps threads/processes to different cores
Most major OS support multi-core today: Windows, Linux, Mac OS X, …

18. Cluster Interconnect Network fabric connecting the compute nodes
Objective is to strike a balance between
Processing power of compute nodes
Communication ability of the interconnect
A more specialized LAN, providing many opportunities for perf. optimization
Switch in the core
Latency vs bw

19. Goal: Bandwidth and Latency

20. Ethernet Switch: allows multiple simultaneous transmissions
hosts have dedicated, direct connection to switch
switches buffer packets
Ethernet protocol used on each incoming link, but no collisions; full duplex
each link is its own collision domain
switching: A-to-A’ and B-to-B’ simultaneously, without collisions
not possible with dumb hub C’ B 1 2 6 3 4 5 C B’ A’
switch with six interfaces
(1,2,3,4,5,6)

21. switch with six interfaces
Switch Table A
Q: how does switch know that A’ reachable via interface 4, B’ reachable via interface 5?
A: each switch has a switch table, each entry:
(MAC address of host, interface to reach host, time stamp)
looks like a routing table!
Q: how are entries created, maintained in switch table?
something like a routing protocol? C’ B 1 2 6 3 4 5 C B’ A’
switch with six interfaces
(1,2,3,4,5,6)

22. Switch: self-learning
Source: A
Dest: A’
A A’
switch learns which hosts can be reached through which interfaces
when frame received, switch “learns” location of sender: incoming LAN segment
records sender/location pair in switch table A C’ B 1 2 6 3 4 5 C B’ A’
MAC addr interface TTL A 1 60
Switch table
(initially empty)

23. Self-learning, forwarding: example
Source: A
Dest: A’
Self-learning, forwarding: example A
A A’ C’ B
frame destination unknown: 1 2
flood 6 3
A A’
A A’
A A’
A A’
A A’ 4 5
destination A location known: C
A’ A
selective send B’ A’
MAC addr interface TTL A 1 60
Switch table
(initially empty) A’ 4 60

24. Interconnecting switches
Switches can be connected together D E F S2 S4 S3 H I G S1 A B C
Q: sending from A to G - how does S1 know to forward frame destined to F via S4 and S3?
A: self learning! (works exactly the same as in single-switch case!)
Q: Latency and Bandwidth for a large-scale network?

25. What characterizes a network?
Topology (what)
physical interconnection structure of the network graph
Regular vs irregular
Routing Algorithm (which)
restricts the set of paths that msgs may follow
Table-driven, or routing algorithm based
Switching Strategy (how)
how data in a msg traverses a route
Store and forward vs cut-through
Flow Control Mechanism (when)
when a msg or portions of it traverse a route
what happens when traffic is encountered?
Interplay of all of these determines performance

26. Tree: An Example Diameter and ave distance logarithmic Fixed degree
k-ary tree, height d = logk N
address specified d-vector of radix k coordinates describing path down from root
Fixed degree
Route up to common ancestor and down
R = B xor A
let i be position of most significant 1 in R, route up i+1 levels
down in direction given by low i+1 bits of B
Bandwidth and Bisection BW?

27. Bandwidth Bandwidth For a switch with N ports,
Point-to-Point bandwidth
Bisectional bandwidth of interconnect frabric: rate of data that can be sent across an imaginary line dividing the cluster into two halves each with equal number of ndoes.
For a switch with N ports,
If it is non-blocking, the bisectional bandwidth = N * the p-t-p bandwidth
Oversubscribed switch delivers less bisectional bandwidth than non-blocking, but cost-effective. It scales the bw per node up to a point after which the increase in number of nodes decreases the available bw per node
oversubscription is the ratio of the worst-case achievable aggregate bw among the end hosts to the total bisection bw

28. How to Maintain Constant BW per Node?
Limited ports in a single switch
Multiple switches
Link between a pair of switches be bottleneck
Fast uplink
How to organize multiple switches
Irregular topology
Regular topologies: ease of management

29. Scalable Interconnect: Examples
Fat Tree
16 node butterfly
building block

30. Multidimensional Meshes and Tori
3D Cube
2D Mesh
2D torus
d-dimensional array
n = kd-1 X ...X kO nodes
described by d-vector of coordinates (id-1, ..., iO)
d-dimensional k-ary mesh: N = kd
k = dÖN
described by d-vector of radix k coordinate
d-dimensional k-ary torus (or k-ary d-cube)?

31. Packet Switching Strategies
Store and Forward (SF)
move entire packet one hop toward destination
buffer till next hop permitted
Virtual Cut-Through and Wormhole
pipeline the hops: switch examines the header, decides where to send the message, and then starts forwarding it immediately
Virtual Cut-Through: buffer on blockage
Wormhole: leave message spread through network on blockage

32. SF vs WH (VCT) Switching
Unloaded latency: h( n/b+ D) vs n/b+hD
h: distance
n: size of message
b: bandwidth
D: additional routing delay per hop

33. Conventional Datacenter Network

34. Problems with the Cluster Arch
Resource fragmentation:
If an application grows and requires more servers, it cannot use available servers in other layer 2 domains, resulting in fragmentation and underutilization of resources
Poor server-to-server connectivity
Servers in different layer-2 domains to communication through the layer-3 portion of the network
See papers in the reading list of Datacenter Network Design for proposed approaches

35. Datacenter as a Computer
Overview
Workloads and SW infrastructure
HW building blocks
Datacenter basics
Energy and power efficiency
Dealing with Failures and Repairs

36. Datercenter
Datacenters are buildings where servers and comm. gear are co-located because of their common environmental requirements and physical security needs, and for ease of maintenance.
Traditional DCs typically host a large number of relatively small- or medium-sized applications, each running on a dedicated hw infra that is de-coupled and protected from other systems in the same facility

37. Recent Data Center Projects

38. Advances in Deployment of DC
Conquering complexity.
Building racks of servers & complex cooling systems all separately is not efficient.
Package and deploy into bigger units:
Microsoft Generation 4 Data Center

39. Microsoft Generation 4 Datacenter

40. Typical Elements of a DC
Often low-end servers are used
Connected in multi-tier networks

41. Key Aspects Storage Network Fabric Storage Hierarchy
Quantify Latency, BW, and Capacity
Power Usage
Handling Failures

42. Storage Global distributed file systems (e.g. Google’s GFS)
Hard to implement at the cluster-level, but lower hw costs and networking fabric utilization
GFS implement replication across different machines (for fault tolerance); Google deploys desktop-clas dis drives, instead of enterprise-grade disks
Network Attached Storage (NAS), directly connected to the cluster-level switching fabric
Simple to deploy, because it pushes the responsibility for data management and integrity to NAS appliance

43. Amazon’s Simple Storage Service (S3)
Write, read, and delete objects containing from 1 byte to 5 terabytes of data each. The number of objects you can store is unlimited.
Each object is stored in a bucket and retrieved via a unique, developer-assigned key.
A bucket can be stored in one of several Regions: in the US Standard, EU (Ireland), US West (Northern California) and Asia Pacific (Singapore) Regions.
Built to be flexible so that protocol or functional layers can easily be added. The default download protocol is HTTP. A BitTorrent™ protocol interface is provided to lower costs for high-scale distribution.
Designed to provide % durability and 99.99% availability of objects over a given year. Designed to sustain the concurrent loss of data in two facilities.
Designed to provide 99.99% durability and 99.99% availability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.01% of objects. Designed to sustain the loss of data in a single facility.

44. Amazon’s Elastic Block Storage (EBS)
EBS provides block level storage volumes for use with EC2 instances. It allows you to create storage volumes from 1 GB to 1 TB that can be mounted as devices by Amazon EC2 instances.
Each storage volume is automatically replicated within the same Availability Zone. This prevents data loss due to failure of any single hardware component.
The latency and throughput of Amazon EBS volumes is designed to be significantly better than the Amazon EC2 instance stores in nearly all cases.

45. Network Fabric Tradeoff between speed, scale, and cost
Two-level hierarchy (in rack and cluster-levels)
A switch having 10 times the bi-section bw costs about 100 times as much
A rack with 40 servers, each with a 1-Gbps port may have between 4 to 8 1-Gbps uplinks to the cluster switch: an oversubscription factor between 5 and 10 for comm across racks.
“fat-tree” networks built of lower-cost commodity Ethernet switches

46. Next Generation of Network Fabric
Monsoon
Work by Albert Greenberg, Parantap Lahiri, David A. Maltz,Parveen Patel, Sudipta Sengupta.
Designed to scale to 100K+ data centers.
Flat server address space instead of dozens of VLANS.
Valiant Load Balancing.
Allows a mix of apps and dynamic scaling.
Strong fault tolerance characteristics.

47. Storage Hierarchy

48. Latency, BW, and Capacity
Assume: 2000 servers (8GB mem and 1TB disk), 40 per rack connected by a 48-port 1Gpbs switch (8 uplinks)
Arch: bridge the gap in a cost-efficient manner
SW : hide the complexity, exploit data locality

49. DC vs Supercomputers Scale Network Architecture Data Storage
Blue Waters = 40K 8-core “servers”
Road Runner = 13K cell + 6K AMD servers
MS Chicago Data Center = 50 containers = 100K 8-core servers.
Network Architecture
Supercomputers: CLOS “Fat Tree” infiniband
Low latency – high bandwidth protocols
Data Center: IP based Network
Optimized for Internet Access
Data Storage
Supers: separate data farm
GPFS or other parallel file system
DCs: use disk on node + memcache
Fat tree network
Standard Data Center

50. Power Usage
Distribution of peak power usage of a Google DC (circa 2007)

51. Workload and SW Infra
Platform-level SW: presented in all individual servers, providing basic server-level services
Cluster-level infra: collection of distributed systems sw that manages resources and provides services at the cluster level
MapReduce, Hadoop, etc
Application-level sw: implements a specific service
Oneline service like web search, gmail,
Offline computations, e.g. data analysis or generate data used for online services such as building index

52. Examples of Appl-SW Web 2.0 applications
Provide rich user experience including real-time global collaboration
Enable rapid software development
Software to scan voluminous Wikipedia edits to identify spam
Organize global news articles by geographic location
Data-intensive workloads based on scalable architectures, such as Google’s MapReduce framework
Financial modeling, real-time speech translation, Web search
Next generation rich media, such as virtual worlds, streaming videos, Web conferencing, etc.

53. Characteristics Ample parallelism at both data and request levels;
Key is not to find parallelism, but to manage and efficiently harness the explicit parallelism
Data parallelism arises from the large data sets of relatively independent records to be processed
Request-level parallelism comes from hundreds or thousands of requests per second to be responded; the requests rarely invovle read-after-write sharing of data or synchronization

54. Characteristics (cont’)
Workload churn: isolation from users of Internet services makes it easy to deploy new sw quickly.
Google’s front-end web server binaries are released on a weekly cycle
The core of its search services be reimplemented from scratch every 2~3 years!!
New products and services frequently emerge, and their success with users directly affects the resulting workload mix in the DC
Hard to develop a meaningful benchmark
Not too much for HW architects to do
Count on sw rewrites to take advantage of new hw capabilities ?!
Fault-free is challenging, but possible

55. Basic Programming Concepts
Data Replication for both perf and availability
Data Partitioning (sharded) for both p. & a.
Load balancing:
Sharded vs replicated services
Health checking & watchdog timers for availability
No op could rely on a given server to respond to make progress forward
Integrity checks for availability
Application-specific compression
Eventual consistency w.r.t. replicated data
When no updates occur for a long period time, eventually all updates will propagete through the system and all replicas will be consistent

56. Cluster-level SW Resource Management HW abstraction and Basic Services
Map user tasks to hw resources, enforce priorities and quotas, provide basic task managment services
Simple allocation; or automate allocation of resources; fair-sharing of resources at a finer level of granulaity; power/energy consideration
Related Work at Wayne State
Wei, et al, Resource management for end-to-end quality of service assurance [Wei’s PhD disseration’06]
Zhou, et al, Proportional resource allocation in web servers, streaming servers, and e-commerce servers proportional ; see cic.eng.wayne.edu for related publications (02-05)
HW abstraction and Basic Services
E.g. reliable distributed storage, message-passing, cluster-level sync. (GFS, Dynamo )

57. Cluster-level SW (cont’)
Deployment and maintenance
Sw image distributionj, confguration management, monitoring service perf and quality, alarm trigger for operators in emergency situations, etc
E.g. Autopilot of Microsoft, Google’s Health Infrastructure
Related Work at Wayne State
Jia, et al, Measuring machine capacity, ICDCS’08
Jia, et al, Autonomic VM configuration, ICAC’09
Bu, et al, Autonomic Web apps configuration, ICDCS’09
Programming Frameworks
Tools like MapReduce would improve programmer productibity by automatically handling data partitioning, distribution, and fault tolerance

58. Monitoring Infra: An Example
Service-level dashboards
Keep track of service quality (w.r.t target level)
Info must be fresh for corrective actions and avoid significant disruption within sec not min
E.g. how to measure user-perceived pageview response time? (multiple objects, end-to-end)
18 objects

59. Client-Experienced QoS
request-based QoS
connection close
server
last object
waiting for new requests
object 2
object 1
base page
client
Setup connection
client-perceived pageview QoS
Internet
Packet
Capture
Analyzer
Perf
TCP Packets
HTTPS Trans
HTTPS Traffic
Mirrored
Wei/Xu, sMonitor for Measurement of User-Perceived Laency, USENIX’2006

60. Perf Debugging Tools
Help operators and service designers to develop understanding of the complex interactions between programs, often running on hundreds of servers, so as to determine the root cause of perf anomalies and identify bottlenecks
Black-box monitoring: observing network traffic among system components and inferring causal relationships through statistical inference methods, assuming no knowledge of or assistance from appl or sw;
But Info may not accurate
Appl/middleware instrumentation systems, like Google’s Dapper, require to modify applications or middleware libraries for passing tracing across machines and across module boundaries. The annotated modules log tracing info to local disks for subsequent analysis

61. HW Building Blocks
Cost Effectiveness of Low-end Servers

62. Performance of Parallel Apps
Under a model of fixed local computation time, plus the latency penalty of access to global data structure

63. Parallel Apps Perf
Perf advantage of a cluster of high-end nodes (128-cores) over a cluster with the same number of cores built with low-end servers (4 cores) [4 to 20 times difference in price]

64. How Small a Cluster Node should Be?
Other factors need to be considered
Amount of parallelism,
Network requirements
Smaller server may lead to low utilization
etc

65. Datacenter as a Computer
Overview
Workloads and SW infrastructure
HW building blocks
Datacenter basics
Energy and power efficiency
Dealing with Failures and Repairs

66. Power/Energy Issues in DC
Main components

67. UPS Systems
A transfer switch that chooses the active power input, with utility power or generator power.
Typically, a generator takes s to start and assume the full rated load
Batteries to bridge of time of utility power blackout
AC-DC-AC double conversion
When utility power fails, UPS loses input AC power but retains internal DC power, and thus the AC output power
Remove voltage spikes or harmonic distortions in the aC feed
100s kW up to 2MW

68. Power Distribution Units
Takes the UPS output (typically 200~480v) and break it up into many 110 or 220v circuits that feed the actual servers on the floor.
Each circuit is protected by its own breaker
A typical PDU handles 75~225kW of load, whereas a typical circuit handles 20 or 30A at V (a max of 6kW)
PDU provides additional redundancy (in circuit)

69. Datacenter Cooling Systems
CRAC Units: computer room air conditioning
Water-based free cooling

70. Energy Efficiency
DCPE (DC perf efficiency) : Ratio of amount of computation al work to total energy consumed
Total energy consumed:
Power usage effectiveness (PUE): ratio of building power to IT power (currently 1.5 to 2.0)

71. Breakdown of DC Energy Overhead

72. Energy Efficiency (cont’)
Power usage effectiveness (PUE)
Server PUE: ratio of total server input power to its useful power (consumed by the components like motherboard, disks, CPUs, DRAM, I/O bards, etc)
User power excludes losses in power supplies, fans,
Currently, SPUE between 1.6 to 1.8.
With VRM (voltage regulatory module), SPUE can be reduced to 1.2

73. Measuring Power Efficiency
Benchmarks
Green500 in high-performance computing
Joulesort:
measuing the total system energy to perform an out-of-core sort
SPECpower_ssj2008
Compute the perf-to-power ratio of a system running a typical business applicatoin an enterprise Java platform

74. Power Efficiency: An Example
SPECpolwer_ssj2008 on a server with single-chip 2.83GH quad-core Intel Xeon, $GB mem, one 7.2k RPM, s.5’’ SATA disk

75. Activity Profile of Google Servers
A sample of 5000 servers over a period of 6 months
Most of the time, 10-50% utilization

76. Energy-Proportional Computing
Humans at rest consume at little as 70w, while being able to sustain peaks of 1kW+ for tens of mins
For adult male:

77. Causes of Poor Energy Proportionality
CPU used to be dominant factor (60%); currently slighly lower than 50% at peak, drops to 30% at low activity levels

78. Energy Proportional Computing: How
Hardware components: For example
CPU: dynamic voltage scaling
High speed disk drives spend 70% of their total power simply keeping the platters spinning
Need smaller rotational speeds, smaller platters,
Power distribution and cooling

79. SW role: management
Smart use of power management features in existing hw, low-overhead inactive or active low-power modes, as well as implementing power-friendly scheduling of tasks to enhance energy proportionality of hw systems
Two challenges:
Encapsulation in lower-level modules to hide additional infra complexity
Performance robustness: minizing perf variability caused by power management tools
Related Work at Wayne State
Zhong, et al “System-wide energy minimization for hard real-time tasks,” TECS’08.
Zhong, et al “Energy-aware modeling and scheduling for dynamic voltage scaling with statistical real-time guarantee,” TC’’07; Zong’s PhD dissertation ‘2007
Gong, Poewr/Performance Optimization (ongoing)

80. Datacenter as a Computer
Overview
Workloads and SW infrastructure
HW building blocks
Datacenter basics
Energy and power efficiency
Dealing with Failures and Repairs

81. Basic Concepts
Failure: A system failure occurs when the delivered service deviates from the specified service, where the service specification is an agreed description of the expected service. [Avizienis & Laprie 1986]
Fault: the root cause of failures, defined as a defective state in materials, design, or implementation. Faults may remain undetected for some time. Once a fault becomes visible, it is called an error.
Faults are unobserved defective states
Error is “manifestation” of faults
(Source: Salfner’08)

82. Challenges of High Service Availability
High service availability expectation translates into a high-reliability requirement for DC
Faults in hw, sw, and operation are inevitable
In Google, about 45% servers need to reboot at least once over a 6-month window; 95%+ requires less often than once a month, but the tail is relatively long.
The average downtime ~3 hours, implying 99.85% availability
Determining the appropriate level of reliability is fundamentally a trade-off between the cost of failures and the cost of preventing them

83. Availability and Reliability
Availability: A measure of the time that a system was actually usable, as a fraction of the time that it was intended to be usable. (x nines)
Yield: the ratio of requests that is satisfied by the service to the total
Reliability Metrics:
Time to failure (TTF)
Time to repair (TTR)
Mean time to failure (MTTF)
Mean time to repair (MTTR)
Mean time between failures (MTBF)

84. Interplay of HW, SW, Operation
Faults in hw and sw are inevitable. But endeavor will never halt: e.g. RAID disk drive, ECC memory
A fault-tolerant sw infrastructure would help hide much of the failure complexity from application-level sw.
E.g. SW-based RAID system across disk drives residing in multiple machines (as in GFS); MapReduce
Flexibility in choosing the level of hw reliability that maximizes overall system cost efficiency (e.g. inexpensive PC-class hw);
Simplification in common operational procedures (e.g. hw/sw upgrade)
With FT sw, not necessary to keep a server running at all costs. This leads to change in every aspect of the systems, from design to operation and opens opportunity to optimization
As long as the hw faults can be detected and reported to sw in a timely manner

85. Fault Characterization
FT-sw be based on fault sources, their statistical characteristics, correponding recovery behavior
Fault Severity
Corrupted: committed data are lost or corrupted
Unreachable: service is down
Google service no better than 99.9% when its servers are one of the end points
Amazon’s Service Level Agreement is 99.95%
Degraded: service is available but in degraded mode
Masked: faults occur but masked from users by FT hw/sw mechanisms

86. Causes of Service-Level Failures
Field data study I on Internet services: Operator-caused or misconfig errors are larget contriubutors; hw-related faults (server or networking) accounts about 10-25% [Oppenheimer’2003]
Field data study II on early Tandem systems: Hw faults (<10%), sw faults (~60%), op/maintenance (~20%) [Gray’90]
Google’s observation over a period of 6 weeks

87. Observations
Seems sw/hw-based ft techniques do well for independent faults.
SW-, Op-, and maintenance-induced faults have a higher impact on outages, possibly because they are most likely affect multiple systems at once, creating a corrected failure scenario which is hard to overcome.

88. Proactive Failure Management?
Failure Prediction?
Predict future machine failures with low false-positive rates in a short time horizon
Develop models for a good trade-off between accuracy (both in false-positive rates and time horizon) and the penalties involved in failure occurrence and recovery
In DC, the penalty is low, the prediction model must be highly accurate to be economically competitive.
In systems where a crash is disruptive to op, less accurate prediction models would be beneficial
Related Work at Wayne State
Fu/Xu, Exploring spatial/temporal event correlation for failure prediction, SC’07
Fu’s PhD dissertation 2008

89. In Summary Hardware Software: Economics: Cost effectiveness
Building blocks are commodity server-class machines, consumer- or enterprise-grade disk drives, Ethernet-based networking fabrics
Perf of the network fabric and storage subsystems be more relevant to CPU and mem
Software:
FT sw for high service availability (99.99%)
Programmability, Parallel efficciency, Manageability
Economics: Cost effectiveness
power and energy factors
Utilization characteristics require systems and components to be energy efficient across a wide load spectrum, particularly at low utilization level

90. In Summary: Key Challenges
Rapidly changing workload
New applications with a large variety of computational characteristics emerge at a fast pace
Need creative solutions from both hw and sw; but little benchmark available
Building balanced systems from imbalanced components
Processors outpaced memory and magnetic storage in perf and power efficiency; more research should be shifted onto the non-cpu subsystems
Curbing energy usage
Power becomes the first order resources, as speed
Performance under power/energy budget

91. Key Challenges (cont’)
Amdahls’ cruel law
Speedup = 1/ (f_seq + f_par/n) on n-node parallel systems. Sequential part f_seq limits parallel efficiency, no matter how large n
Future perf gains will continue to be delivered mostly due to cores or threads, not so much by faster cpus.
Data-level or request-level parallelism is enough? Parallel computing beyond MapReduce!!

92. Reading List
Barroso and Holzle, “The datacenter as a computer,” Morgan & Claypool, 2009.