Bigtable: A Distributed Storage System for Structured Data Google Inc. OSDI 2006

Content Introduction Data Model API and DEMO Building Blocks Implementation Conclusion

Introduction Bigtable is a distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of data across thousands of commodity servers. goals: – wide applicability – Scalability – high performance – high availability

Introduction

Bigtable resembles a database, but isn’t Bigtable does not support a full relational data model Bigtable treats data as uninterpreted strings Bigtable schema parameters let clients dynamically control whether to serve data out of memory or from disk.

Data Model A Bigtable is a sparse, distributed, persistent multi- dimensional sorted map. The map is indexed by a row key, column key, and a timestamp; each value in the map is an uninterpreted array of bytes. (row:string, column:string, time:int64) -> string

Data Model

Rows – a tablet is a row range: a the unit of distribution and load balancing Column Families – Column Families mean the sets Column keys are grouped into. – syntax: family:qualier. Column family names must be printable,but qualiers may be arbitrary strings Timestamp – 64-bit integers

API and DEMO

Building Blocks Google File System (GFS) – store log and data les – Run in a shared pool of machines Google SSTable: File format – Key to Value – Index map Chubby: a highly-available and persistent distributed lock service

Implementation Client Master Server Tablet Server

Implementation Master Server: – assigning tablets to tablet servers – detecting the addition and expiration of tablet servers – balancing tablet-server load – garbage collection of files in GFS. Tablet Server: – manage a set of tablets – read and write requests to the tablets – Splits tablets

Implementation

Others Refinement – Locality groups – Compression – Caching for read performance – Bloom filters – Commit-log implementation Performance Evaluation Lesson

THANK YOU