NoSQL Or Peles

What is NoSQL A collection of various technologies meant to work around RDBMS limitations (mostly performance) Not much of a definition...

RDBMS Limitations Hard to scale horizontally (for updates) – Distributed ACID requires 2 phase commit. Schema can be a bitch – Hard to change. – Data normalization can slow down queries.

Web Scale Some numbers: – Youtube serves over 100MM videos a day. – Ebay adds over 10TB of storage every week. – Facebook holds over 80 Billion photos, and serves hundreds of thousands of requests/second.

Ideal System Available – can always read and write. Consistent – Reads always pick up the latest write. Partition tolerant – The system can be split across multiple machines and datacenters.

Starbucks doesn’t use two phase commit A great example presented here.here Asynchronous execution Correlation Exception handling: – Write off – Retry – Compensation 2 phase commit would create a choke point.

CAP Theorem CAP (Eric Brewer, 2000): Simply put, of the following 3 properties: Consistency Availability Partition tolerance Only two can hold at any system.

CAP in practice

CA Two phase commits, works best at a single data center. Scaling issues. CP Sharding. Data may become unavailable if a shard fails. AP May return inaccurate data. DNS is a prime example.

Consistency Types Strict Eventual – Causual – Read your writes – Session – Monotonic read – Monotonic write

Concepts In memory vs. disk based. Shared everything vs. shared nothing. Master slave vs. server symmetry. Elastic scalability. MapReduce.

Sharding Split data across machines (database instances). – Feature based sharding. – Key based sharding. – Lookup table.

NoSQL Categories Key-Value stores Document store Tabular

Lean & Mean The Key-Value In-Memory DBs In memory DBs are simpler and faster than their on-disk counterparts. Key value stores offer a simple interface with no schema. Major limitation – data size is limited to RAM size. Often used as caches for on-disk DB systems.

Open Source In-Memory DBs Memcached/MemchachedDB Redis – Both are key-value stores that rely on hash partitioning – Memcached is an LRU based cache. – Redis is more of a data structure server. – Both use a Shared-Nothing architecture

Memcached Really a giant, distributed hash table. Advantages: – Relatively simple – Practically no server to server talk. – Linear scalability Disadvantages: – Doesn’t understand data – no server side operations. The key and value are always strings. – It’s really meant to only be a cache – no more, no less. – No recovery, limited elasticity.

Redis Like Memcached, it’s a distributed hash in memory. Offers support for lists and sets, as well as strings. Offers limited server side operations. Supports master-slave architecture and data replicas for scalability and high availability. Also supports a persistent mode that writes to disk.

Document Stores As the name implies, these databases store documents. Usually schema-free. The same database can store multiple documents. Allow indexing based on document content. Prominent examples: CouchDB, MongoDB.

Documents A document is just a collection of values, usually serialized in JSON. Many implementations offer nesting of documents Example: { "username" : "bob", "address" : { "street" : "123 Main Street", "city" : "Springfield", "state" : "NY" } }

CouchDB Written in ERLANG. Offers ACID guarantees based on multi-version control. Supports replication, but isn’t a real distributed database.

MongoDB Written in C++. Atomic operations on single documents only. Excellent scalability based on sharding. Support for server side javascript and MapReduce.

Tabular stores The original: Google’s BigTable – Proprietary, not open source. The open source elephant alternative – Hadoop with HBase. A top level Apache Project. Large number of users. Contains a distributed file system, MapReduce, a database server (Hbase), and more. Rack aware.

Hadoop components

Hadoop basic components At it’s core, Hadoop is a framework for running MapReduce operations on large data sets. The data sets are placed as text files on the distributed file system.

Hadoop MapReduce Flow

HBase A database engine built on top of Hadoop distributed file system. Scales up to Billions of rows with Millions of columns. Has a Java interface for queries.

The Tradeoff – SQL vs. NoSQL RDBMS: – Mature. – Standard SQL (but not for DDL, extensions). – Robust tools. NoSQL: – Scale – Schemaless

References Eventual Consistency (Werner Vogels, CTO, Amazon) Eventual Consistency Starbucks doesn’t use two phase commit. Starbucks doesn’t use two phase commit Hadoop the definitive guide (O’Reilly) MongoDB the definitive guide (O’Reilly) Many wiki pages

Questions