U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Emery Berger University of Massachusetts Amherst Operating Systems CMPSCI 377 Lecture 23: Advanced File Systems

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 2 Advanced File Systems Motivation & Brief review Journaling Log-structured file system

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 3 New Challenges Servers have special needs! Large hard-disk partitions Quick crash recovery High-performance I/O Storing thousands of files, TB of data None supported well by ext2

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 4 Review: Blocks & Fragmentation Logical block – smallest unit of storage that can be allocated by file system Internal fragmentation – occurs when file does not fill block completely Example: file = 10K, block = 8K: wastes 6K External fragmentation – occurs when logical blocks that make up file are scattered over the disk Causes poor performance

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 5 Review: Organization Extent – contiguous blocks: Triple: (file offset, starting block number, length) file offset – offset of extent's first block from file start starting block number – first block in extent length –number of blocks in extent Meta data – file system's internal data structures Time stamps, ownership, size & locations on disk, etc.

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 6 Extents Triple: (offset, start, length) Contiguous blocks

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 7 Review: Inodes inode – stores information about file e.g., permissions, types, # of links to file pointers to file data blocks (direct pointers) pointers to direct pointers (indirect pointers) each has unique inode number directory = special file: contains pointers to other files

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 8

9 Metadata Updates Lots of meta data being updated Scattered on disk = slow, and non-atomic Example: Creating a new file Modifies inodes, free lists, directory entries What happens if interrupted? e.g., power outage, crash Interrupted meta-data write ) file system in inconsistent state

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 10 Repairing File System Inconsistency UNIX solution: fsck = “file-system check” Analogous utilities for Windows, etc. Detect and repair structural integrity problems Scan file system’s entire meta data Looks for inconsistencies & fixes them Problems? S…l…o…w… Might not succeed

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 11 Journaling File Systems Solve problem by maintaining journal Log all transactions to disk Updates to the disk committed atomically Example: Creating a new file Log modifications to inodes Log changes to free lists, directory entries Now begin update When finished, write “committed” into journal

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 12 Recovery in JFS Power outage: Some updates fully committed to file system No problem Not yet fully committed File system reads journal, replays transaction Advantages over fsck: Much quicker than scan of disk Guarantees file system always consistent But: does not guarantee zero data loss

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 13 Advanced File Systems Brief review Journaling Log-structured file system

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 14 LFS: Beyond JFS Insight: While logging updates, why not log the data too? “Log-structured file system” Preserves data integrity Provides improved performance, too!

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 15 Sprite LFS [Rosenblum & Ousterhout] Outperforms then-current Unix file systems by an order of magnitude for small-file writes Matches or exceeds Unix performance for reads and large writes Even with overhead Can use 70% of disk bandwidth for writing Unix file systems typically can use only 5-10%

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 16 Technology Trends Disk technology improving rapidly But: capacity rather than performance Increasing RAM ) cache more effective Disk traffic dominated by writes Logs: write new information sequentially Increase write performance, eliminating seeks

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 17 Existing File Systems Exercise worst-case given trends! Information spread around disk Many small accesses Synchronous writes Application performance can be bottlenecked by disk performance

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 18 Log-Structured File Systems Approach: improve write performance by combining into single sequential writes Converts small random writes into large sequential transfers Use file cache as write buffer But where does meta data go?

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 19 Structure of LFS (1/3) Some index structures used to retrieve information Inodes not fixed: inode map used to maintain location of each inode Fixed checkpoint region on each disk: identifies locations of inode map blocks

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 20 Structure of LFS (2/3) Disk layout (compared with Unix FFS) Disk Log InodeDirectory Data Inode map Sprite LFS Unix FFS dir1 dir2 file1file2 dir1dir2 file1file2

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 21 Structure of LFS (3/3) Segments Important to maintain large free extents for writing new data Divide disk into large fixed-size extents called segments Segments always written sequentially Before segment is rewritten, all live data must be copied out (“cleaned”)

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 22 Crash Recovery (1/2) Checkpoints Position in log at which all of file system structures are consistent and complete Checkpoint region written on special fixed position: Periodic intervals When the file system unmounted or shut down

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 23 Crash Recovery (2/2) Roll-forward After crashes, LFS scans through log segments that were written after last checkpoint Use information in segment summary blocks When summary block indicates presence of new inode, LFS updates inode map

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 24 LFS Wrap-Up Extends journaling to data Sequential writes in segments Fast! Periodic clean-up Periodic checkpoints

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 25 Summary Traditional filesystems – integrity problems Solved by journaling Log-structured further improves speed by optimizing write performance