Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition, Lecture 24: GFS

17.2 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Google File System Why yet another file system? Who are the “clients”? Design objectives Architecture Cache consistency Location transparent/independent? Stateful or stateless?

17.3 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Google File System Key background: New workload => new filesystem (why?) Extreme scale: 100 TB over 1000s of disks on >1000 machines New API Architecture? Who are the clients? Four problems to solve: Fault tolerance: this many components ensures regular failures. Must have automatic recovery. Huge files (are they?) Most common operation is append, not random writes  Most files are write once, and read-only after that  web snapshots, intermediate files in a pipeline, archival files  implies that streaming is much more important than block caching (and LRU would be a bad choice) Customize the API to enable optimization and flexibility

17.4 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition GFS: New Workload Few million large files, rather than billions of small files Large streaming reads, random reads Large streaming writes, very rare random writes Concurrency for append is critical also producer/consumer concurrency Focus on throughput (read/write bandwidth) not latency Why?

17.5 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition GFS Architecture

17.6 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition GFS Architecture single master, multiple chunkservers, multiple clients fixed-size chunks (giant blocks) (64MB) 64-bit ids for each chunk clients read/write chunks directly from chunkservers unit of replication? master maintains all metadata namespace and access control map from filenames to chunk ids current locations for each chunk no caching for chunks metadata is cached at clients

17.7 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition GFS Master Single master: Claim: simple, but good enough Enables good chuck placement (centralized decision) Scalability is a concern. What is the GFS solution? Clients cache (file name -> chunk id, replica locations), this expires eventually Large chunk size reduces master RPC interaction and space overhead for metadata All metadata is in memory (why?) metadata is 64B per chunk

17.8 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Question: Does GFS implement Location Transparency? What about Location Independence?

17.9 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Cache Consistency in GFS Caching? Of data? Of metadata? Metadata changes? Namespace changes are atomic and serializable (easy since they go through one place: the master) Replicas: “defined” if it reflects a mutation and “consistent” “consistent”: all replicas have the same value Concurrent writes will leave region consistent, but not necessarily defined; some updates may be lost “region”? A failed write leaves the region inconsistent To ensure definition, GFS must apply writes in the same order at all replicas: How?

17.10 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Writes and replica updates in GFS

17.11 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition Consistency in GFS (cont) consistency ensured via version numbers stale versions not used and GC’d client chunk caching may contain stale data! this is OK for append only files, but not for random writes. primary replica decides the write order (it has lease for this right from the master) lease extensions piggybacked on the heartbeat messages master can revoke a lease but only with the replicas agreement; otherwise it has to wait for expiration client pushes data out to all replicas using a data flow tree after all data received, client notifies the primary to issue the write, which then gets ordered and can now be executed client retries if the write fails writes across chunks: transactional? Consequences?

17.12 Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition GFS: statefull or stateless?