1. Google Bigtable
Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, Robert E. Gruber
Google, Inc.
UWCS OS Seminar Discussion
Erik Paulson
2 October 2006
See also the (other)UW presentation by Jeff Dean in September of 2005
(See the link on the seminar page, or just google for “google bigtable”)

2. Before we begin… Intersection of databases and distributed systems
Will try to explain (or at least warn) when we hit a patch of database
Remember this is a discussion!

3. Google Scale Lots of data Many incoming requests
Copies of the web, satellite data, user data, and USENET, Subversion backing store
Many incoming requests
No commercial system big enough
Couldn’t afford it if there was one
Might not have made appropriate design choices
Firm believers in the End-to-End argument
450,000 machines (NYTimes estimate, June 14th 2006

4. Building Blocks Scheduler (Google WorkQueue) Google Filesystem
Chubby Lock service
Two other pieces helpful but not required
Sawzall
MapReduce (despite what the Internet says)
BigTable: build a more application-friendly storage service using these parts

5. Google File System Large-scale distributed “filesystem”
Master: responsible for metadata
Chunk servers: responsible for reading and writing large chunks of data
Chunks replicated on 3 machines, master responsible for ensuring replicas exist
OSDI ’04 Paper

6. Chubby {lock/file/name} service
Coarse-grained locks, can store small amount of data in a lock
5 replicas, need a majority vote to be active
Also an OSDI ’06 Paper

7. Data model: a big map
<Row, Column, Timestamp> triple for key - lookup, insert, and delete API
Arbitrary “columns” on a row-by-row basis
Column family:qualifier. Family is heavyweight, qualifier lightweight
Column-oriented physical store- rows are sparse!
Does not support a relational model
No table-wide integrity constraints
No multirow transactions

8. SSTable Immutable, sorted file of key-value pairs
Chunks of data plus an index
Index is of block ranges, not values
SSTable
64K block
64K block
64K block
Index

9. Tablet Contains some range of rows of the table
Built out of multiple SSTables
Tablet
Start:aardvark
End:apple
SSTable
SSTable
64K block
64K block
64K block
64K block
64K block
64K block
Index
Index

10. Table Multiple tablets make up the table SSTables can be shared
Tablets do not overlap, SSTables can overlap
Tablet
Tablet
aardvark
apple
apple_two_E
boat
SSTable
SSTable
SSTable
SSTable

11. Finding a tablet

12. Servers
Tablet servers manage tablets, multiple tablets per server. Each tablet is megs
Each tablet lives at only one server
Tablet server splits tablets that get too big
Master responsible for load balancing and fault tolerance
Use Chubby to monitor health of tablet servers, restart failed servers
GFS replicates data. Prefer to start tablet server on same machine that the data is already at

13. Editing a table
Mutations are logged, then applied to an in-memory version
Logfile stored in GFS
Tablet
Insert
Memtable
Insert
Delete
apple_two_E
boat
Insert
Delete
Insert
SSTable
SSTable

14. Compactions Minor compaction – convert the memtable into an SSTable
Reduce memory usage
Reduce log traffic on restart
Merging compaction
Reduce number of SSTables
Good place to apply policy “keep only N versions”
Major compaction
Merging compaction that results in only one SSTable
No deletion records, only live data

15. Locality Groups Group column families together into an SSTable
Avoid mingling data, ie page contents and page metadata
Can keep some groups all in memory
Can compress locality groups
Bloom Filters on locality groups – avoid searching SSTable

16. Microbenchmarks

18. Application at Google

19. Lessons learned
Interesting point- only implement some of the requirements, since the last is probably not needed
Many types of failure possible
Big systems need proper systems-level monitoring
Value simple designs