1. Michael J. Carey Information Systems Group UCI CS Department Big Data 2.0

2. Outline Big Data Platform Space – The Big Data Era – Brief History of Data Platforms – Dominant Platforms AsterixDB Project @ UCI / UCR – Our Approach – Feature Overview – Call for Participation Quick Q&A (time permitting) 1

3. It’s the “Big Data” Era Unprecedented growth in data being generated and its potential uses/value – Tweets, social networks (statuses, check-ins, shared content), blogs, click streams, various logs, … – Facebook: ≈ 1.4B monthly active users, > 4.5B likes/day – Twitter: ≈ 288M monthly active users, ≈ 500M tweets/day Everyone is interested (might be why you’re here! ) – Trade press and popular press: “Big Data!” – Enterprises, Web companies, online businesses, governments, public health researchers, social scientists, … – Untapped value and countless new opportunities to understand, optimize, assist, and/or compete 2

4. One Reason Big Data’s Exploding 3

5. Brief History of Big Data Database community – all about “Big Data” since the beginning of time – Born from the relational database movement initiated by Ted Codd in 1970 Distributed systems community – all about “Big Data” since 2000 or so (due to the Web) – Google, Yahoo!, Amazon, Facebook, …, all started fresh, but are now following the DB field’s path Let’s briefly look at the two parallel universes... 4

6. Big Data in the Database World Enterprises needed to store and query their historical data (data warehouses) – 1970’s: Relational databases appeared (with SQL) – 1980’s: Parallel database systems based on “shared- nothing” architectures (Gamma, GRACE, Teradata) – 2000’s: Netezza, Aster Data, DATAllegro, Greenplum, Vertica, ParAccel, … (Serious “Big $” acquisitions!) 5 Also OLTP systems – Run daily businesses – Points of origin for warehoused data SN Scalable OLTP  SN – 1980’s: Tandem NonStop SQL System

7. Big Data in the Systems World Late 1990’s: new scaling demands (Web data) – Google File System (GFS) OS-level byte streams spanning 1000’s of machines 3x replication for fault-tolerance (high availability) – Google’s MapReduce (MR) programming model User writes two simple functions: Map & Reduce “Parallel programming for dummies” – MR runtime does the heavy lifting (via partitioned parallelism) – Yahoo!, Facebook, et al soon cloned Google’s “Big Data” infrastructure from papers Now we have Apache Hadoop (MR) and HDFS 6

8. Soon tired of solving 2-move puzzles, higher- level languages appeared (hiding MR model) – Pig (Yahoo!) & Hive (Facebook) compile to MR jobs – Quickly overran native MR (Pig > 60%, HiveQL > 90%) – Web indexing, click stream analysis, log analysis, information extraction, some machine learning,... Also key-value or “NoSQL” systems (≈ OLTP) – Simple, key-based retrievals and updates (put/get) – Fast, highly scalable, highly available – but simple! – BigTable, Dynamo, HBase, MongoDB, Couchbase,... 7 Systems Big Data World (cont.)

9. No More “One Size Fits All”... (  ) 8

10. Other Up-and-Coming Platforms Bulk Synchronous Programming (BSP) platforms, e.g., Pregel, Giraph, GraphLab,..., for doing Big Graph analysis Spark for in-memory cluster computing – for repetitive data analyses, iterative machine learning tasks,... 9 (“Big” is the platform’s concern) “Think Like a Vertex” – Receive messages – Update state – Send messages iter. 1 iter. 2... Input Distributed memory Input query 1 query 2 query 3... one-time processing iterative processing

11. AsterixDB: Big Data 2.0 10 Semistructured Data Management Parallel Database Systems Wild World of Hadoop BDMS BDMS Desiderata: Flexible data model Efficient runtime Full query language Costs proportional to tasks at hand Built-in support for continuous data ingestion Support today’s “Big Data data types”

12. create dataverse MyNewSocialSite; create type MugshotUserType as { id: int32, alias: string, name: string, user-since: datetime, address: { street: string, city: string, state: string, zip: string, country: string }, friend-ids: {{ int32 }}, employment: [EmploymentType] } User Model (Scale-Independent!) 11 create dataset MugshotUsers(MugshotUserType) primary key id; #AsterixDB Ex: User names and messages sent by users who joined the Mugshot social network within a certain time window: from $user in dataset MugshotUsers where $user.user-since >= datetime('2010-07-22T00:00:00') and $user.user-since <= datetime('2012-07-29T23:59:59') select { "uname" : $user.name, "messages" : from $message in dataset MugshotMessages where $message.author-id = $user.id select $message.message }; Highlights include: JSON++ based data model Rich base types (spatial, temporal, …) Records, lists, bags Open vs. closed types Full query language (AQL) (optional)

13. ASTERIX System Overview 12 (ADM = ASTERIX Data Model; AQL = ASTERIX Query Language)

14. ASTERIX Software Stack 13#AsterixDB

15. Project Status 4-year initial NSF project (now ~300 KLOC) – Code developed jointly by UCI and UCR (team stays 15-20 strong) – Also working with UCSD (queries) and UCLA (use cases) – Hardening/sharing AsterixDB & doing Big Active Data (“BAD”) work AsterixDB BDMS is here! (@ June 6 th, 2013) – Semistructured “NoSQL” style data model – Declarative parallel queries, inserts, deletes,... – Data storage with primary and secondary indexing – Internal and external datasets both supported – Rich set of data types (incl. text, time, and geo location) – Fuzzy and spatial query processing – Data feeds and external indexing as well We would love to work with you, if you have use cases...! – Ex: Social data analytics, education (especially MOOCs), public health, emergency response, power use monitoring,... – Heading to Apache; interested in additional contributors 14#AsterixDB

16. For More Info AsterixDB project page: http://asterixdb.ics.uci.edu http://asterixdb.ics.uci.edu Pregelix project page: http://pregelix.ics.uci.edu http://pregelix.ics.uci.edu Open source code base: ASTERIX: http://code.google.com/p/asterixdb/http://code.google.com/p/asterixdb/ Hyracks: http://code.google.com/p/hyrackshttp://code.google.com/p/hyracks (Pregelix: http://hyracks.org/projects/pregelix/)http://hyracks.org/projects/pregelix/ 15#AsterixDB Questions…?

17. Backup Slides & Figures 16

18. Figure – PDB 17 SQL Compiler Relational Dataflow Runtime Row/Column Storage Manager SQL Compiler Relational Dataflow Runtime Row/Column Storage Manager SQL Compiler Relational Dataflow Runtime Row/Column Storage Manager