The Three E’s of Big Data and What DB People can do About Them UC BERKELEY Michael Franklin – UC Berkeley Beckman Database Get Together October 14, 2013

The Big Data Problem - Nutshelled TimeQualityMoney 2 Massive Diverse and Growing Data Massive Diverse and Growing Data Something’s gotta give :

The 3 E’s of Big Data:

Extreme Elasticity - Machines Option #1 – Build your own Cluster/WSC (US East – Saturday Sept Option #3 – Try your luck on the Spot Market Option #2 – Rent Machines from AWS x Servers needed 46K Servers (2010 estimate)

Extreme Elasticity - Algorithms Agarwal et al., BlinkDB: Queries with Bounded Errors and Bounded Response Times on Very Large Data. ACM EuroSys 2013.

Extreme Elasticity - People 6 Incentives Fatigue, Fraud, & other Failure Modes Latency & Prediction Work Conditions Interface Answer Quality Task Structuring Task Routing

Extreme Elasticity Algorithms Approximate Answers ML Libraries and Ensemble Methods Active Learning Machines Cloud Computing – esp. Spot Instances Multi-tenancy Relaxed (eventual) consistency/ Multi-version methods People Dynamic Task and Microtask Marketplaces Visual analytics Manipulative interfaces and mixed mode operation

The Challenge

The Good News: We already know how to do this (kinda)! SQLResultMQL Model ✦ End Users tell the system what they want, not how to get it

Query Planner / Optimizer Runtime ML Developer API ML Library MQL Parser (Contracts) Release d July 2013 initial release: Spring 2014 MLbase: Progress

For More Information amplab.cs.berkeley. edu du UC BERKELEY