CS122B: Projects in Databases and Web Applications Winter 2017 Notes 07: NoSQL and Hadoop Professor Chen Li Department of Computer Science UC Irvine CS122B

NoSQL Very hot these days What does NoSQL mean? Mainly with startups, less with enterprises What does NoSQL mean? “Not Only SQL” or “Not Relational”.

NoSQL: Overview “A NoSQL database provides a mechanism for storage and retrieval of data that is modeled in means other than the tabular relations used in relational databases.” Give up some of the ACID constraints to improve performance Simple interface: Write (=Put): needs to write all replicas Read (=Get): may get only one

NoSQL Key features: Scale horizontally “simple operations” Replicate/distribute data over many servers Simple call level interface (contrast w/ SQL) Weaker concurrency model than ACID Efficient use of distributed indexes and RAM Flexible schema

1. Key-value Stores Think “file system” more than “database” Put/get calls with a key Voldemort, Memcached Only primary index: lookup by key No secondary indexes

2. Document Stores Nested values, extensible records (e.g., XML or JSON). Schema-less MongoDB, CouchDB May have secondary indexes

3. Column stores Data organized as columns, not rows Vertica, HBase, Cassandra

4. Scalable Relational Systems Means RDBMS that offer sharding MySQL Cluster, VoltDB

What is Hadoop? Apache top level project, open-source implementation of frameworks for reliable, scalable, distributed computing and data storage. It is a flexible and highly-available architecture for large scale computation and data processing on a network of commodity hardware.

Google Origins 2003 2004 2006

Word Count over a Given Set of Web Pages see 1 bob 1 throw 1 see 1 spot 1 run 1 bob 1 run 1 see 2 spot 1 throw 1 see bob throw see spot run Can we do word count in parallel?

Word Count over a Given Set of Web Pages

“Big Data” Era