1. An Overview of Cloud Computing Raghu Ramakrishnan Chief Scientist, Audience and Cloud Computing Research Fellow, Yahoo! Research
Reflects many discussions with:
Eric Baldeschwieler, Jay Kistler, Chuck Neerdaels, Shelton Shugar, and Raymie Stata
and joint work with the Sherpa team, in particular:
Brian Cooper, Utkarsh Srivastava, Adam Silberstein and Nick Puz in Y! Research
Chuck Neerdaels, P.P. Suryanarayanan and many others in CCDI 1

2. CCDI—Research Collaboration
Yahoo! Research
Raghu Ramakrishnan
Brian Cooper
Utkarsh Srivastava
Adam Silberstein
Nick Puz
Rodrigo Fonseca
CCDI
Chuck Neerdaels
P.P.S. Narayan
Kevin Athey
Toby Negrin
Plus Dev/QA teams

3. SCENARIOS Pie-in-the-sky
Need some slides up front to layout what our definition of a cloud entails (since it’s a rapidly emerging buzzword that has no intrinsic definition. )
SCENARIOS 3

4. Living in the Clouds We want to start a new website, FredsList.com
Our site will provide listings of items for sale, jobs, etc.
As time goes on, we’ll add more features
And illustrate how more cloud capabilities (and corresponding infrastructure components) are used as needed
List of capabilities/components is illustrative, not exhaustive
Our cloud provides a “dataset” abstraction
FredsList doesn’t worry about the underlying components

5. Simple Web Service API’s
Step 1: Listings
FredsList wants to store listings as (key, category, description)
FredsList.com application
DECLARE DATASET Listings AS
( ID String PRIMARY KEY,
Category String,
Description Text )
, transportation, For sale: one bicycle, barely used
, childcare,
Nanny available in San Jose
215534,
wanted,
Looking for issue 1 of Superman comic book
Simple Web Service API’s
Database
Sherpa

6. FredsList.com application
Step 2: Search
FredsList’s customers quickly ask for keyword search
FredsList.com application
ALTER Listings
SET Description SEARCHABLE
“bicycle”
“dvd’s”
“nanny”
Simple Web Service API’s
Database
Search
Sherpa
Vespa
Messaging
YMB

7. FredsList.com application
Step 3: Photos
FredsList decides to add photos to listings
FredsList.com application
ALTER Listings
ADD Photo BLOB
Simple Web Service API’s
Database
Storage
Search
Foreign key
photo → listing
MObStor
Sherpa
Vespa
Messaging
YMB

8. FredsList.com application
Step 4: Data Analysis
FredsList wants to analyze its listings to get statistics about category, do geocoding, etc.
FredsList.com application
ALTER Listings
MAKE ANALYZABLE
Hadoop program to generate fancy pages for listings
Pig query to analyze categories
Hadoop program to geocode data
Simple Web Service API’s
Compute
Database
Storage
Search
Grid
Foreign key
photo → listing
MObStor
Sherpa
Vespa
Batch export
Messaging
YMB

9. Step 5: Performance FredsList wants to reduce its data access latency
FredsList.com application
ALTER Listings
MAKE CACHEABLE
Simple Web Service API’s
Compute
Database
Storage
Search
Caching
Grid
Foreign key
photo → listing
MObStor
Sherpa
Vespa
memcached
Batch export
Messaging
YMB

10. EYES TO THE SKIES Motherhood-and-Apple-Pie
Need some slides up front to layout what our definition of a cloud entails (since it’s a rapidly emerging buzzword that has no intrinsic definition. )
EYES TO THE SKIES 10

11. Why Clouds?
On-demand infrastructure to create a fundamental shift in the OE curve. Let’s us:
Do things we can’t do
Reduce time to market
Build more robustly, more efficiently, more globally, more completely, for a given budget
Cloud services should do heavy lifting of heavy-lifting of scaling & high-availability
Today, this is done at the app-level, which is not productive

12. Requirements for Cloud Services
Multitenant. A cloud service must support multiple, organizationally distant customers.
Elasticity. Tenants should be able to negotiate and receive resources/QoS on-demand.
Resource Sharing. Ideally, spare cloud resources should be transparently applied when a tenant’s negotiated QoS is insufficient, e.g., due to spikes.
Horizontal scaling. It should be possible to add cloud capacity in small increments; this should be transparent to the tenants of the service.
Metering. A cloud service must support accounting that reasonably ascribes operational and capital expenditures to each of the tenants of the service.
Security. A cloud service should be secure in that tenants are not made vulnerable because of loopholes in the cloud.
Availability. A cloud service should be highly available.
Operability. A cloud service should be easy to operate, with few operators. Operating costs should scale linearly or better with the capacity of the service.

13. Types of Cloud Services
Two kinds of cloud services:
Horizontal Cloud Services
Functionality enabling tenants to build applications or new services on top of the cloud
Functional Cloud Services
Functionality that is useful in and of itself to tenants. E.g., various SaaS instances, such as Saleforce.com; Google Analytics and Yahoo!’s IndexTools; Yahoo! properties aimed at end-users and small businesses, e.g., flickr, Groups, Mail, News, Shopping
Could be build on top of horizontal cloud services or from scratch
Yahoo! has been offering these for a long while (e.g., Mail for SMB, Groups, Flickr, BOSS, Ad exchanges)

14. Horizontal Cloud Services
Horizontal cloud services are foundations on which tenants build applications or new services. They should be:
Semantics-free. Must be "generic infrastructure,” and not tied to specific app-logic.
May provide the ability to inject application logic through well-defined APIs
Broadly applicable. Must be broadly applicable (i.e., it can't be intended for just one or two properties).
Fault-tolerant over commodity hardware. Must be built using inexpensive commodity hardware, and should mask component failures.
While each cloud service provides value, the power of the cloud paradigm will depend on a collection of well-chosen, loosely coupled services that collectively make it easy to quickly develop and operate innovative web applications.

15. What’s in the Horizontal Cloud?
Simple Web Service API’s
Horizontal Cloud Services
Provisioning & Virtualization
e.g., EC2
Batch Storage & Processing
e.g., Hadoop
& Pig
Operational Storage
e.g., S3,
MObStor,
Sherpa
Edge Content Services
e.g., YCS, YCPI
Other Services
Messaging, Workflow, virtual DBs & Webserving
ID & Account Management
Metering, Billing, Accounting
Shared
Infrastructure
Security
Monitoring & QoS
Common Approaches to QA, Production Engineering,
Performance Engineering, Datacenter Management, and Optimization 15

16. Yahoo! CCDI Thrust Areas
Fast Provisioning and Machine Virtualization: On demand, deliver a set of hosts imaged with desired software and configured against standard services
Multiple hosts may be multiplexed onto the same physical machine.
Batch Storage and Processing: Scalable data storage optimized for batch processing, together with computational capabilities
Operational Storage: Persistent storage that supports low-latency updates and flexible retrieval
Edge Content Services: Support for dealing with network topology, communication protocols, caching, and BCP
Rest of
today’s talk

17. Hadoop: Batch Storage/Analysis
Why is batch processing important?
Whether it’s
response-prediction for advertising
machine-learned relevance for Search, or
content optimization for audience,
data-intensive computing is increasingly central to everything Yahoo! does
Hadoop is central to addressing this need
Hadoop is a case-study in our cloud vision
Processes enormous amounts of data
Provides horizontal scaling and fault-tolerance for our users
Allows those users to focus on their app logic
[Workflow]
High-level query layer (Pig)
Map-Reduce
HDFS OK

18. SHERPA To Help You Scale Your Mountains of Data
A project in Y!R focused on a long-range problem, origins in earlier work at Wisconsin. Basis for the Goldrush hack, which won the recent Local hack competition, and could contribute to creation/refinement of Y! Local content and Next Gen Search.

19. The Yahoo! Storage Problem
Small records – 100KB or less
Structured records - tens, hundreds or thousands of fields
Extreme data scale - Tens of TB
Extreme request scale - Tens of thousands of requests/sec
Low latency globally datacenters worldwide
High Availability - outages cost $millions
Variable usage patterns - as applications and users change 19

20. The next generation global-scale record store
The Sherpa Solution
The next generation global-scale record store
Record-orientation: Routing, data storage optimized for low-latency record access
Scale out: Add machines to scale throughput (while keeping latency low)
Asynchrony: Pub-sub replication to far-flung datacenters to mask propagation delay
Consistency model: Reduce complexity of asynchrony for the application programmer
Cloud deployment model: Hosted, managed service to reduce app time-to-market and enable on demand scale and elasticity 20

21. What is Sherpa? CREATE TABLE Parts ( ID VARCHAR, StockNumber INT,
Status VARCHAR … )
E C
A E
B W
C W
D E
F E
A E
B W
C W
D E
E C
F E
Structured, flexible schema
E C
A E
B W
C W
D E
F E
Geographic replication
Parallel database
Hosted, managed infrastructure 21

22. What Will Sherpa Become?
A E
B W
C W
D E
F E
E C
A E
B W
C W
D E
F E
Indexes and views
CREATE TABLE Parts (
ID VARCHAR,
StockNumber INT,
Status VARCHAR … )
E C
A E
B W
C W
D E
F E
Geographic replication
Parallel database
Structured, flexible schema
Hosted, managed infrastructure

23. Sherpa Design Goals Scalability Geographic replication
Thousands of machines
Easy to add capacity
Restrict query language to avoid costly queries
Geographic replication
Asynchronous replication around the globe
Low-latency local access
High availability and fault tolerance
Automatically recover from failures
Serve reads and writes despite failures
Consistency
Per-record guarantees
Timeline model
Option to relax if needed
Multiple access paths
Hash table, ordered table
Primary, secondary access
Hosted service
Applications plug and play
Share operational cost 23 23

24. Technology Elements Applications PNUTS API Tabular API PNUTS
Query planning and execution
Index maintenance
Distributed infrastructure for tabular data
Data partitioning
Update consistency
Replication
YCA: Authorization
YDOT FS
Ordered tables
YDHT FS
Hash tables
YMB
Pub/sub messaging
Zookeeper
Consistency service 24 24

25. Data Manipulation Per-record operations Multi-record operations
Get
Set
Delete
Multi-record operations
Multiget
Scan
Getrange
Web service (RESTful) API 25

26. Tablets—Hash Table 0x0000 0x2AF3 0x911F 0xFFFF Name Description Price
Grape
Grapes are good to eat
$12
Lime
Limes are green $9
Apple
Apple is wisdom $1
Strawberry
Strawberry shortcake
$900
0x2AF3
Orange
Arrgh! Don’t get scurvy! $2
Avocado
But at what price? $3
Lemon
How much did you pay for this lemon? $1
Tomato
Is this a vegetable?
$14
0x911F
Banana
The perfect fruit $2
Kiwi
New Zealand $8
0xFFFF 26

27. Tablets—Ordered Table
Name
Description
Price A
Apple
Apple is wisdom $1
Avocado
But at what price? $3
Banana
The perfect fruit $2
Grape
Grapes are good to eat
$12 H
Kiwi
New Zealand $8
Lemon
How much did you pay for this lemon? $1
Lime
Limes are green $9
Orange
Arrgh! Don’t get scurvy! $2 Q
Strawberry
Strawberry shortcake
$900
Tomato
Is this a vegetable?
$14 Z 27

28. Flexible Schema Posted date Listing id Item Price 6/1/07 424252 Couch
$570
763245
Bike
$86
6/3/07
211242
Car
$1123
6/5/07
421133
Lamp
$15
Color
Red
Condition
Good
Fair

29. Detailed Architecture
Remote regions
Local region
Clients
REST API
Routers
YMB
Tablet controller
Storage
units 29

30. Tablet Splitting and Balancing
Each storage unit has many tablets (horizontal partitions of the table)
Storage unit may become a hotspot
Storage unit
Tablet
Overfull tablets split
Tablets may grow over time
Shed load by moving tablets to other servers 30

31. QUERY PROCESSING 31

32. Accessing Data SU SU SU Get key k Get key k Record for key k 1 4 2 332

33. Bulk Read SU SU SU {k1, k2, … kn} Get k1 Get k2 Get k3 1 2 33 Scatter/
gather server SU SU SU 33

34. Range Queries in YDOT Clustered, ordered retrieval of records
Storage unit 1
Canteloupe
Storage unit 3
Lime
Storage unit 2
Strawberry
Router
Apple
Avocado
Banana
Blueberry
Storage unit 1
Canteloupe
Storage unit 3
Lime
Storage unit 2
Strawberry
Grapefruit…Lime?
Grapefruit…Pear?
Lime…Pear?
Canteloupe
Grape
Kiwi
Lemon
Lime
Mango
Orange
Storage unit 1
Storage unit 2
Storage unit 3
Strawberry
Tomato
Watermelon
Apple
Avocado
Banana
Blueberry
Strawberry
Tomato
Watermelon
Lime
Mango
Orange
Canteloupe
Grape
Kiwi
Lemon

35. Updates SU SU SU Sequence # for key k Write key k Write key k1 8
Sequence # for key k
Write key k
Routers
Message brokers 2
Write key k 3
Write key k 7 4
Sequence # for key k 5
SUCCESS SU SU SU 6
Write key k 35

36. ASYNCHRONOUS REPLICATION AND CONSISTENCY36

37. Asynchronous Replication37

38. Consistency Model
Goal: make it easier for applications to reason about updates and cope with asynchrony
What happens to a record with primary key “Brian”?
Record inserted
Update
Update
Update
Update
Update
Update
Update
Delete
v. 1
v. 2
v. 3
v. 4
v. 5
v. 6
v. 7
v. 8
Time
Time
Generation 1 38

39. Consistency Model Read Stale version Stale version Current version
Time
Generation 1 39

40. Consistency Model Read up-to-date Stale version Stale version
Current version
v. 1
v. 2
v. 3
v. 4
v. 5
v. 6
v. 7
v. 8
Time
Generation 1 40

41. Consistency Model Read ≥ v.6 Stale version Stale version
Current version
v. 1
v. 2
v. 3
v. 4
v. 5
v. 6
v. 7
v. 8
Time
Generation 1 41

42. Consistency Model Write Stale version Stale version Current version
Time
Generation 1 42

43. Consistency Model Write if = v.7 Stale version Stale version
ERROR
Stale version
Stale version
Current version
v. 1
v. 2
v. 3
v. 4
v. 5
v. 6
v. 7
v. 8
Time
Generation 1 43

44. Consistency Model Mechanism: per record mastership Write if = v.7
ERROR
Stale version
Stale version
Current version
v. 1
v. 2
v. 3
v. 4
v. 5
v. 6
v. 7
v. 8
Time
Generation 1 44

45. Consistency Techniques
Per-record mastering
Each record is assigned a “master region”
May differ between records
Updates to the record forwarded to the master region
Ensures consistent ordering of updates
Tablet-level mastering
Each tablet is assigned a “master region”
Inserts and deletes of records forwarded to the master region
Master region decides tablet splits
These details are hidden from the application
Except for the latency impact!

46. Mastering Tablet master 46 A 42342 E B 42521 W C 66354 W D 12352 E
E C
F E
A E
B W
Tablet master
C W
D E
E C
F E
A E
B W
C W
D E
E C
F E 46

47. Bulk Insert/Update/Replace
Client feeds records to bulk manager
Bulk loader transfers records to SU’s in batches
Bypass routers and message brokers
Efficient import into storage unit
Client
Bulk manager
Source Data

48. Bulk Load in YDOT YDOT bulk inserts can cause performance hotspots
Solution: preallocate tablets

49. Index Maintenance
How to have lots of interesting indexes, without killing performance?
Solution: Asynchrony!
Indexes updated asynchronously when base table updated
Planned functionality

50. SHERPA IN CONTEXT 50

51. MObStor
Yahoo!’s next-generation globally replicated, virtualized media object storage service
Better provisioning, easy migration, replication, better BCP, and performance
New features (Evergreen URLs, CDN integration, REST API, …)
The object metadata problem addressed using Sherpa, though MObStor is focused on blob storage. 51 51

52. Storage & Delivery Stack

53. The World Has Changed Web applications need:
Scalability!
Preferably elastic
Geographic distribution
High availability
Reliable storage
Web applications can do without:
Complicated queries
Strong transactions

54. Structured record storage
Web Data Management
CRUD
Point lookups and short scans
Index organized table and random I/Os
$ per latency
Scan oriented workloads
Focus on sequential disk I/O
$ per cpu cycle
Structured record storage
(PNUTS)
Large data analysis
(Hadoop)
Object retrieval and streaming
Scalable file storage
$ per GB
Blob storage
(SAN/NAS)

55. Retrieval from single table of objects/records
Types of Record Stores
Query expressiveness S3
PNUTS
Oracle
Simple
Feature rich
Object retrieval
Retrieval from single table of objects/records
SQL

56. Types of Record Stores Consistency model S3 PNUTS Oracle Best effort
Strong guarantees
Eventual consistency
Timeline consistency
ACID
Program centric consistency
Object-centric consistency

57. Types of Record Stores Elasticity (ability to add resources on demand)
PNUTS S3
Oracle
Not scalable
Elastic
Limited
(via data distribution)
VLSD
(Very Large Scale Distribution /Replication)

58. Data Stores Comparison
User-partitioned SQL stores
Microsoft Azure SDS
Amazon SimpleDB
Multi-tenant application databases
Salesforce.com
Oracle on Demand
Mutable object stores
Amazon S3
Versus PNUTS
More expressive queries
Users must control partitioning
Limited elasticity
Highly optimized for complex workloads
Limited flexibility to evolving applications
Inherit limitations of underlying data management system
Object storage versus record management

59. Application Design Space
Get a few things
Sherpa
MObStor
YMDB
MySQL
Oracle
Filer
BigTable
Scan everything
Everest
Hadoop
Records
Files 59

60. Alternatives Matrix Consistency model Global low Structured SQL/ACID
latency
Structured
access
Consistency model
SQL/ACID
Operability
Availability
Updates
Elastic
Sherpa
Y! UDB
MySQL
Oracle
HDFS
BigTable
Dynamo
Cassandra 60

61. Further Reading
Efficient Bulk Insertion into a Distributed Ordered Table (SIGMOD 2008)
Adam Silberstein, Brian Cooper, Utkarsh Srivastava, Erik Vee,
Ramana Yerneni, Raghu Ramakrishnan
PNUTS: Yahoo!'s Hosted Data Serving Platform (VLDB 2008)
Brian Cooper, Raghu Ramakrishnan, Utkarsh Srivastava,
Adam Silberstein, Phil Bohannon, Hans-Arno Jacobsen,
Nick Puz, Daniel Weaver, Ramana Yerneni

62. Opening Up Yahoo! Search
Phase 1
Phase 2
SearchMonkey Messaging:
• SearchMonkey is another proof point in Yahoo!’s big bets strategy
o Openness – SearchMonkey will create access to the structured data that the web is based on and build a better experience for Yahoo! Search users
o Partner of choice - Yahoo! Search is opening up its platform to any size site owner or developer to play in the search ecosystem with self-service tools
o Insights – site owners will have better insight to search traffic drivers; better able to create targeted results for better engagement
• Yahoo! Search is a leading starting point on the Web for users and the first major search engine to open its search page to site owners and developers and allow them to create visibly differentiated search results.
• By opening up the search platform, Yahoo! Search improves the way search engines display results to the benefit of users, site owner and developers.
Giving site owners and developers control over the appearance of Yahoo! Search results.
BOSS takes Yahoo!’s open strategy to the next level by providing Yahoo! Search infrastructure and technology to developers and companies to help them build their own search experiences. 62

63. Search Results of the Future
yelp.com
Gawker
babycenter
New York Times
epicurious
LinkedIn
answers.com
webmd

64. BOSS Offerings API CUSTOM ACADEMIC
BOSS offers two options for companies and developers and has partnered with top technology universities to drive search experimentation, innovation and research into next generation search.
API
A self-service, web services model for developers and start-ups to quickly build and deploy new search experiences.
CUSTOM
Working with 3rd parties to build a more relevant, brand/site specific web search experience.
This option is jointly built by Yahoo! and select partners.
ACADEMIC
Working with the following universities to allow for wide-scale research in the search field:
University of Illinois Urbana Champaign
Carnegie Mellon University
Stanford University
Purdue University
• MIT
Indian Institute of
Technology Bombay
University of
Massachusetts
(Slide courtesy Prabhakar Raghavan) 64

65. Partner Examples About Medium
Founded in 2006 in Boulder, Colorado, Me.dium is a social browsing software company offering a browser extension that allows people to surf with friends for the first time. By revealing this new Social Exploration Environment TM (SEE), Me.dium graphically connects users with their friends and others enabling users to interact online, similarly to how one interacts with people in the real world. Me.dium’s Social Search Alpha is the first crowd-powered search engine.   Social Search enables people to find relevant information based on the activity of crowds right now.  See an entirely new layer of information over and above what traditional search provides.  You get the hottest, most active content related to your query - as determined by the current activity of the crowds.
Me.dium combined the BOSS API with it’s insight into the real time surfing activity of the crowds to build a unique "Crowd-Powered" social search engine prototype. 65

66. QUESTIONS? 66