1. Chapter 14: File System Implementation

2. Chapter 14: File System Implementation
File-System Structure
File-System Operations
Directory Implementation
Allocation Methods
Free-Space Management
Efficiency and Performance
Recovery
Example: WAFL File System

3. Objectives
To describe the details of implementing local file systems and directory structures
To describe the implementation of remote file systems
To discuss block allocation and free-block algorithms and trade-offs

4. File-System Structure
File structure
Logical storage unit
Collection of related information
File system resides on secondary storage (disks)
Two characteristics of disks make them convenient for file system:
A disk can be rewritten in place; it is possible to read a block from the disk, modify the block, and write it back into the same block.
A disk can access directly any block of information it contains.
Thus, it is simple to access any file either sequentially or randomly, and switching from one file to another requires the drive moving the read–write heads and waiting for the media to rotate.
Nonvolatile memory (NVM) devices are increasingly used for file storage and thus as a location for file systems.
They differ from hard disks in that they cannot be rewritten in place and they have different performance characteristics.

5. File-System Structure
Disk provides in-place rewrite and random access
I/O transfers performed in blocks of sectors (usually 512 or 4096 bytes)
NVM devices usually have blocks of 4,096 bytes
File systems
Provides efficient and convenient access to disk by allowing data to be stored, located, and retrieved easily
Interface to user involves defining a file and its attributes, the operations allowed on a file, and the directory structure for organizing files.
Algorithms and data structures to map the logical file system to physical secondary storage devices
File control block – storage structure consisting of information about a file
File system organized into layers
Each level in the design uses the features of lower levels to create new features for use by higher levels.
Device driver controls the physical device

6. Layered File System
The I/O Control level consists of device drivers and interrupt handlers to transfer information between the main memory and the disk system.
A device driver can be thought of as a translator.
Its input consists of high-level commands, such as “retrieve block 123” , “read drive1, cylinder 72, track 2, sector 10, into memory location 1060”
Its output consists of low-level, hardware-specific instructions that are used by the hardware controller, which interfaces the I/O device to the rest of the system.
The device driver usually writes specific bit patterns to special locations in the I/O controller's memory to tell the controller which device location to act on and what actions to take.

7. File System Layers
Basic file system (called the “block I/O subsystem” in Linux) needs only to issue generic commands to the appropriate device driver to read and write blocks on the storage device.
It is also concerned with I/O request scheduling.
Also manages memory buffers and caches (allocation, freeing, replacement)
Buffers hold data in transit
Caches hold frequently used file-system metadata to improve performance
File organization module understands files, logical address, and physical blocks
Translates logical block # to physical block #
Manages free space, disk allocation

8. File System Layers (Cont.)
Logical file system manages metadata information
Metadata includes all of the file-system structure except the actual data (or contents of the files).
Translates file name into file number, file handle, location by maintaining file control blocks (inodes in UNIX)
A file-control block (FCB) (an inode in UNIX file systems) contains information about the file, including ownership, permissions, and location of the file contents.
Directory management
Protection
Layering useful for reducing complexity, redundancy, and duplication of code, but adds overhead and can decrease performance
Logical layers can be implemented by any coding method according to OS designer

9. File System Layers (Cont.)
Many file systems, sometimes many within an operating system
Each with its own format
CD-ROM is ISO 9660;
Unix has UNIX File System (UFS), based on the Berkley Fast File System (FFS);
Windows has FAT, FAT32, NTFS (or WindowsNT File System), as well as CD-ROM, DVD and Blu-ray file-system formats
Linux supports more than 130 types, the standard Linux file system is known as the extended file system, with the most common versions being ext3 and ext4;
There are also distributed file systems in which a file system on a server is mounted by one or more client computers across a network.
New ones still arriving – ZFS, GoogleFS, Oracle ASM, FUSE

10. File-System Operations
We have system calls at the API level, but how do we implement their functions?
On-storage and in-memory structures
Boot control block (per volume) contains info needed by system to boot OS from that volume
Needed if volume contains OS, usually first block of volume
In UFS, it is called the boot block. In NTFS, it is the partition boot sector.
Volume control block (per volume) contains volume details
Total # of blocks, # of free blocks, block size, free block pointers or array, and a free-FCB count and FCB pointers.
In UFS, this is called a superblock. In NTFS, it is stored in the master file table.
A directory structure (per file system) is used to organize the files.
In UFS, this includes file names and associated inode numbers.
In NTFS, it is stored in the master file table.

11. File-System Operation (Cont.)
A per-file File Control Block (FCB) contains many details about the file
inode number, permissions, size, dates
NFTS stores into in master file table using relational DB structures
To create a new file, a process calls the logical file system.
The logical file system knows the format of the directory structures.
It allocates a new FCB.
The system then reads the appropriate directory into memory, updates it with the new file name and FCB, and writes it back to the file system.
File Control Block (FCB)

12. In-Memory File System Structures
The in-memory information is used for both file-system management and performance improvement via caching.
The data are loaded at mount time, updated during file-system operations, and discarded at dismount.
An in-memory mount table contains information about each mounted volume.
An in-memory directory-structure cache holds the directory information of recently accessed directories.
The system-wide open-file table contains a copy of the FCB of each open file, as well as other information.
The per-process open-file table contains pointers to the appropriate entries in the system-wide open-file table, as well as other information, for all files the process has open.
Buffers hold file-system blocks when they are being read from or written to a file system.

13. In-Memory File System Structures
UNIX treats a directory exactly the same as a file, one with a “type” field indicating that it is a directory.
Windows implement separate system calls for files and directories and treat directories as entities separate from files.
The logical file system can call the file-organization module to map the directory I/O into storage block locations, which are passed on to the basic file system and I/O control system.
The open() call passes a file name to the logical file system.
Search the system-wide open-file table to see if the file is already in use
If it is, a per-process open-file table entry is created pointing to the existing system-wide open-file table.
If the file is not already open, the directory structure is searched for the given file name.
FCB of the file is copied into a system-wide open-file table in memory, which also tracks the number of processes that have the file open.
Parts of the directory structure are usually cached in memory

14. In-Memory File System Structures
Next, an entry is made in the per-process open-file table, with a pointer to the entry in the system-wide open-file table and some other fields.
The open() call returns a pointer to the appropriate entry in the per-process file-system table.
UNIX systems refer to the entry as a file descriptor; Windows refers to it as a file handle.
When a process closes the file, the per-process table entry is removed, and the system-wide entry's open count is decremented.
When all users that have opened the file close it, any updated metadata are copied back to the disk-based directory structure, and the system-wide open-file table entry is removed.

15. In-Memory File System Structures
In-memory file-system structures. (a) File open. (b) File read.

16. Directory Implementation
Linear list of file names with pointer to the data blocks
Simple to program
Time-consuming to execute
Linear search time
Could keep ordered alphabetically via linked list or use B+ tree
create a new file - search the directory, then add a new entry at the end of the directory.
delete a file - search the directory for the named file and then release the space allocated to it.
To reuse the directory entry:
mark the entry as unused, by assigning it a special name, such as an all-blank name, assigning it an invalid inode number (such as 0), or by including a used–unused bit in each entry
attach it to a list of free directory entries.
copy the last entry in the directory into the freed location and to decrease the length of the directory.

17. Directory Implementation
Hash Table – linear list with hash data structure
Decreases directory search time
Collisions – situations where two file names hash to the same location
Only good if entries are fixed size.
For larger hash table, a new hash function required to map file names to the larger range, and reorganize the existing directory entries to reflect their new hash-function values.
Chained-overflow method, where each hash entry can be a linked list instead of an individual value

18. Allocation Methods - Contiguous
An allocation method refers to how disk blocks are allocated for files:
Contiguous allocation – each file occupies set of contiguous blocks
Best performance in most cases
The number of disk seeks and seek time is minimal
Simple – only starting location (block #) and length (number of blocks) are required
Fast sequential access, easy direct (random) access
Problems include finding space for file, knowing file size, external fragmentation, need for compaction off-line (downtime) or on-line (performance penalty)
How much file size – difficult to estimate, if file size known, pre-allocation is inefficient as it may lead to internal fragmentation
Modified scheme – initial allocation, then add extents
file a (base=1,len=3) file b (base=5,len=2)
what happens if
file c needs 2
sectors???

19. Contiguous Allocation
Mapping from logical to physical
LA/512 Q R
Block to be accessed = Q + starting address
Displacement into block = R

20. Extent-Based Systems
Many newer file systems (i.e., Veritas File System) use a modified contiguous allocation scheme
Extent-based file systems allocate disk blocks in extents
An extent is a contiguous block of disks
Extents are allocated for file allocation
A file consists of one or more extents

21. Allocation Methods - Linked
Linked allocation – each file a linked list of blocks
File ends at nil pointer
No external fragmentation
Each block contains pointer to next block
No compaction, external fragmentation
Free space management system called when new block needed
Improve efficiency by clustering blocks into groups but increases internal fragmentation
Reliability can be a problem, lose block, lose rest of the file
Locating a block can take many I/Os and disk seeks
Can grow files dynamically and free list is managed same as file
Sequential access: seek between each block; Random access: horrible
file a (base=1) file b (base=5)
how do you find
the last block in a?

22. Example: DOS FS (simplified)
Used linked list; but instead of embedding links in pages, they used a separate structure called the File Allocation Table (FAT).
The FAT has an entry for each block on the disk and the entries corresponding to the blocks of a particular file are linked up.
FAT (16-bit entries)
free
file a
Directory (5)
eof 1 1 2
a: 6
b: 2
eof
file b 3 3 4
eof 5 6 4
...

23. FAT discussion FAT (opt) FAT root dir …
Space overhead of FAT is trivial:
2 bytes / 512 byte block = ~.4% (Compare to Unix)
Reliability: how to protect against errors?
Create duplicate copies of FAT on disk.
State duplication a very common theme in reliability
Bootstrapping: where is root directory?
Fixed location on disk:
FAT (opt) FAT root dir …
FAT (MS-DOS)

24. Linked Allocation
Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk
pointer
block =
Mapping
LA/511 Q R
Block to be accessed is the Qth block in the linked chain of blocks representing the file.
Displacement into block = R + 1

25. Linked Allocation

26. File-Allocation Table