Distributed Systems Foundations Lecture 0

Evolution of computing history Main Frame with terminals Network of PCs & Workstations. Client-Server Now, moving forward to Large cloud. 2CS 271

Cloud Reality: Data Centers 3CS 271

Cloud Computing: Why Now? Experience with very large datacenters – Unprecedented economies of scale – Transfer of risk Technology factors – Pervasive broadband Internet – Maturity in Virtualization Technology Business factors – Minimal capital expenditure – Pay-as-you-go billing model CS 2714

Unused resources Economics of Cloud Users CS 271 Pay by use instead of provisioning for peak Static data centerData center in the cloud Demand Capacity Time Resources Demand Capacity Time Resources Slide Credits: Berkeley RAD Lab 5

Unused resources Economics of Cloud Users CS 271 Risk of over-provisioning: underutilization Static data center Demand Capacity Time Resources Slide Credits: Berkeley RAD Lab 6

Economics of Cloud Users CS 271 Heavy penalty for under-provisioning Lost revenue Lost users Resources Demand Capacity Time (days) 1 23 Resources Demand Capacity Time (days) 1 23 Resources Demand Capacity Time (days) 1 23 Slide Credits: Berkeley RAD Lab 7

Cloud Properties Commodity hardware Large Scale Elasticity 8CS 271

App Server Elasticity in the Cloud CS 271 Load Balancer (Proxy) App Server Client Site App Server Client Site 9

Why does this work? As long as requests are stateless, we can add more resources, thus providing: Scale Elasticity 10CS 271

But, most services need DATA! Challenges: – How to scale with the increasing amounts of data – Where to store the data – Accessing data on multiple sites – Failures 11CS 271

Need Fault-tolerance: – Replication Large scale data: – Partition data across multiple servers Multiple Servers: – Time: clocks – Coordination: mutual exclusion, leader election – Consensus: Byzatine agreement, Paxos, etc – Distributed state – P2P 12CS 271

Explosive Data Growth Wikipedia has over 3.5 million pages. Flickr members uploaded over 5 billion photos You Tube has 35 hours of videos uploaded each minute “more video uploaded to YouTube in the past two months than there would have been if ABC, CBS, and NBC had been airing 24/7 since 1948!” Gartner 2010 CS 27113

Twitter 6 th B’day: March 21, 2012 First tweet: "inviting coworkers“, 2006 a record of 6,939 tweets per second immediately after Japan quake on March , and 177 million tweets rest of the day. It took three years, two months and a day for Twitter to get to one billion tweets. Twitter now averages 140 million tweets a day. CS 27114