Operating Systems File Systems CNS 3060

Topics Files Directories File system implementation Operating Systems Topics Files Directories File system implementation Example file systems

Long-term Information Storage Operating Systems Long-term Information Storage Must store large amounts of data Information stored must survive the termination of the process using it Multiple processes must be able to access the information concurrently

File Structure Three kinds of files Operating Systems Three kinds of files byte sequence (O/S doesn’t care what’s inside) record sequence (O/S understands record structure) tree (Database systems)

Sequential Access o read all bytes/records from the beginning Operating Systems Sequential Access o read all bytes/records from the beginning o cannot jump around, could rewind or back up. o convenient when medium was mag tape

Random Access o bytes/records read in any order – disk drives Operating Systems Random Access o bytes/records read in any order – disk drives o essential for data base systems o read can be … - move file marker (seek), then read or … - read and then move file marker

Typical File Attributes Operating Systems

1. Create 2. Delete 3. Open 4. Close 5. Read 6. Write Operating Systems File Operations 1. Create 2. Delete 3. Open 4. Close 5. Read 6. Write Append Seek Get attributes Set Attributes Rename

Operating Systems Memory Mapped Files In some cases, it is convenient to map a file into the address space of a running process. File access is then done by normal reads and writes of memory. The result is much faster and to some, much easier than actual writing to the file. Memory mapping is done by changing the system’s internal tables so that the file becomes backing store (ala paging) for the memory region into which the file is mapped.

Hierarchical Directory Systems Operating Systems Root A B C A B B C C B B C C C C

Path Names Operating Systems A UNIX directory tree

1. Create 2. Delete 3. Opendir 4. Closedir Operating Systems Directory Operations 1. Create 2. Delete 3. Opendir 4. Closedir 5. Readdir 6. Rename 7. Link 8. Unlink

A File System Implementation Operating Systems sector 0 1 partition is marked as active OS Loader A possible file system layout When the system is started, the BIOS reads in and executes the Master Boot Record (MBR). The MBR locates the active partition and finds it’s boot block. The boot block loads the O/S.

The superblock contains information about the file system itself, for example, how big is the file system super block i-node table data

This is an array of i-node structures. An i-node is identified by it’s position in the array. super block i-node table data

I-Nodes Operating Systems I-nodes are fixed in size. When a file is opened, it’s i-node is loaded from disk into memory. Thus only a small amount of memory is required, and only while the file is opened.

All directories and files are stored in the data area of the file system. Everything in the file system is stored in “blocks ”. Blocks are fixed in size and represent the smallest unit of storage in the file system. super block i-node table data

Creating a new file involves the following operations: 1. The O/S finds an unused i-node 47 super block i-node table data

Creating a new file involves the following operations: 1. The kernel finds an unused i-node 2. The kernel stores file attributes in the i-node 47 super block i-node table data attributes

Creating a new file involves the following operations: 1. The kernel finds an unused i-node 2. The kernel stores file attributes in the i-node 3. The kernel find free blocks and stores the file data 47 super block i-node table data ||||||||| ||||||||| ||||||||| 200 635 821 attributes

Creating a new file involves the following operations: 1. The kernel finds an unused i-node 2. The kernel stores file attributes in the i-node 3. The kernel find free blocks and stores the file data 4. The kernel stores the block number in the i-node 47 super block i-node table data ||||||||| ||||||||| ||||||||| 200 635 821 attributes 200 635 821

Creating a new file involves the following operations: 1. The kernel finds an unused i-node 2. The kernel stores file attributes in the i-node 3. The kernel find free blocks and stores the file data 4. The kernel stores the block number in the i-node 5. The kernel adds an entry to the directory 47 super block i-node table data ||||||||| ||||||||| ||||||||| 200 635 821 attributes 200 635 47 myFile.txt 821

Terminology Operating Systems Sector Block or cluster Track Internal Fragmentation: Occurs when all of a block is not used by a file. A block is the minimum unit of storage for data. Block sizes are defined by the O/S and the file system. External Fragmentation: Occurs when blocks used to store a file are not contiguous. Read/write head

Problem Operating Systems Given that you need n blocks on the disk Sector Operating Systems Block or cluster Track 1 4 A block is the minimum unit of storage for data. Block sizes are defined by the O/S and the file system. 2 3 Read/write head Given that you need n blocks on the disk to hold the contents of a file, how do you allocate those blocks to the application?

Contiguous File Allocation Operating Systems The simplest file allocation scheme is to take blocks sequentially from the disk, as they are needed for each file. This has two major advantages: o It is simple to implement. You only need to keep track of the starting block and the number of blocks in the file. o It is very efficient. Only one seek is required to read in the entire file. (a seek is the operation that moves the read/write head over the correct track.)

Problem Operating Systems Sector Block or cluster Track Read/write 1 2 4 Read/write head

Contiguous File Allocation Operating Systems Create a file of 3 blocks

Contiguous File Allocation Operating Systems Create a file of 3 blocks Create a file of 5 blocks

Contiguous File Allocation Operating Systems Create a file of 3 blocks Create a file of 5 blocks Create a file of 4 blocks

Contiguous File Allocation Operating Systems Create a file of 3 blocks Create a file of 5 blocks Create a file of 4 blocks Create a file of 6 blocks

Contiguous File Allocation Operating Systems What’s the problem with this design? External Disk Fragmentation. you have to have a file that is 5 blocks or less to fit here! Create a file of 3 blocks Create a file of 5 blocks Create a file of 4 blocks Create a file of 6 blocks Now . . . Delete the 2nd file

Linked List Allocation Operating Systems + Every block on disk can be used. Disk blocks can be anywhere. + The directory only need store the address of the first block of the file. - Each block sacrifices the space required to store the pointer - Random access of blocks in the file is slow.

what if you want to randomly Random Access Operating Systems 1 4 2 3 what if you want to randomly access this block?

The directory only contains the address of the first block. Operating So ... you have to access this block, because it contains the pointer to the next block. The directory only contains the address of the first block. Operating Systems 1 4 2 3

Operating Systems Now you need to access this block 1 4 2 3 Now you need to access this block to get the location of the 3rd block . . . but this will more than likely involve a disk seek, i.e. move the disk head

A File Allocation Table (fat) Operating Systems The fat table usually resides in a fixed location at the beginning of on the disk. No space is taken up in the file for pointers.

Why cache the fat? Operating Systems move the disk head to 1 4 fat move the disk head to read the first entry in the fat 2 3

Why cache the fat? Operating Systems now move the disk head to 1 4 fat 2 3 now move the disk head to read in the first block of the file.

to read the next entry in Why cache the fat? Operating Systems 1 4 fat move the disk head back to read the next entry in the fat 2 3

Why cache the fat? Operating Systems Move the disk head to 1 4 fat Move the disk head to read in the next block in the file 2 3

When the fat is in cache Operating Systems + Random access is easier – the chain is entirely in memory - The biggest disadvantage is that the FAT resides in memory assume a 20GB disk with 1024KB block-size. The FAT needs 20 million entries (60-80MB)

Unix uses i-nodes!

Directory Implementations Operating Systems mail attributes disk address games attributes disk address homework attributes disk address music attributes disk address photos attributes disk address A simple Directory * File attributes stored in the directory * Disk address stored in the directory (first block) * Fixed size entries (so fixed length file names) (MS/DOS & Windows3.x)

Directory Implementations Operating Systems mail address of i-node games address of i-node homework address of i-node music address of i-node photos address of i-node Each directory entry points to an i-node. File attributes are stored in the i-node. (Unix)

Handling Long File Names Operating Systems Fragmentation Issues (take a directory entry out) Page Faults may occur (directory spans multiple pages)

Handling Long File Names Operating Systems

Unix allows different processes to share files … File Sharing Unix allows different processes to share files … The Process Table: every process has an entry in the process table includes all open file descriptors owned by the process - file descriptor flags - a pointer into the file table process table entry fd flags ptr fd 0: fd 1: fd 2: ...

The File Table (per process) table of all open files - status flags for the file (read, write, append, etc) - the current file offset - a pointer to the i-node for this file file table entry process table entry file status flags current file offset i-node pointer fd flags ptr fd 0: fd 1: fd 2: ...

read from disk when the file is opened The i-node Table one for each open file read from disk when the file is opened includes a pointer to the file’s i-node - file permissions - file owner - file size - device file is physically located on - pointers to the actual file blocks on disk - etc file table entry process table entry i-node file status flags current file offset i-node pointer permissions user & group ids File size Time stamps . . . Pointer to first disk block fd flags ptr fd 0: fd 1: fd 2: ...

A Single Process With Two Open Files i-node permissions user & group ids File size Time stamps . . . Pointer to first disk block process table entry file table file status flags current file offset i-node pointer fd flags ptr fd 0: fd 1: fd 2: ... file status flags current file offset i-node pointer i-node permissions user & group ids File size Time stamps . . . Pointer to first disk block

Two Processes Sharing the Same File process table entry i-node file table file status flags current file offset v-node pointer permissions user & group ids File size Time stamps . . . Pointer to first disk block fd flags ptr fd 0: fd 1: fd 2: ... file table process table entry file status flags current file offset v-node pointer fd flags ptr fd 0: fd 1: fd 2: ... Note that as each process writes to the file, the file offset is updated in the file table for that process, to reflect the number of bytes written. If this causes the file offset to exceed the file size, the file size is updated in the files i-node.

Sharing Files Operating Systems In the simple directory case, both directories must contain the block addresses for the file. If user B adds to the file, the appended disk blocks will not show up in C’s directory. This file appears in User B’s directory as well as in User C’s directory. This problem is solved using i-nodes, since each directory entry only need point to the i-node.

Operating Systems i-node file data B’s directory C ’s directory

Deleting a Shared File User C creates a file Operating Systems User C’s directory User C creates a file i-node owner = C count = 1 i-node

Deleting a Shared File User B links to the shared file. Operating Systems User C’s directory User B’s directory User B links to the shared file. i-node What do you do when User C deletes the file? You can’t delete the file, Because the B’s Directory will point to an invalid i-node. owner = C count = 2 i-node

Deleting a Shared File User B links to the shared file. Operating Systems User C’s directory User B’s directory User B links to the shared file. i-node What do you do when user C deletes the file? You can’t delete the file, because the B’s Directory will point to an invalid i-node. owner = C count = 1 i-node All you can do is delete C’s directory Entry and leave the file. This works, but may cause accounting problems.

Using Symbolic Links Operating Systems User C’s User B’s directory Using what is called a symbolic link, B links to one of C’s files by creating a new file type called a Link. The Link file just contains the path name of the file it links to. type = file i-node type = link i-node file data path file

Using Symbolic Links Operating Systems User C’s User B’s directory Now, when C deletes the file, the i-node pointed to by B’s directory is valid. However, attempts to access the file will fail. type = link i-node ??? path file

Disk Space Management Block Size Operating Systems As we have seen, disk space is usually managed in fixed size blocks. The question arises, how big should the blocks be? If the block is too large, then a lot of space is wasted. If the block size is too small, then performance suffers because reading each block requires a rotational delay and a seek.

Determining Optimal Block Size for 2KB files data rate is almost completely dominated by seek time and rotational delay of the disk. wasted space Block Size Dark line (left hand scale) gives the data rate of a disk Dotted line (right hand scale) gives disk space efficiency Note that disk space utilization and data rate are in direct conflict!

Managing Free Disk Blocks Operating Systems bit map Linked List n bytes to hold a disk block number 1 bit per block Fixed size

The MS-DOS File System was patterned after Operating Systems The MS-DOS File System was patterned after the CP/M File System Directory entry – fixed 32 byte length bytes 8 3 1 10 2 2 2 4 File Name Ext Reserved Time Date Size First block number in FAT Attribute read-only archive hidden system file Pinned to 1980 Accurate to within 2 seconds

FAT-12 Operating Systems MS-DOS defines a block (called a cluster by Microsoft) as some multiple of 512 bytes. FAT-12 blocks were 512 bytes. Each cluster is represented in the File Allocation table by a 12 bit number. Thus, the maximum partition size was about 2MB and the File location Table contained 4096 2-byte entries.

Fat-16 Operating Systems As drives got bigger, the File Allocation Table structure had to change to accommodate the bigger sizes. In a FAT-16 table, each cluster was now represented by a full 16 bit entry in the file allocation table. Additional block sizes of 8 KB, 16KB, and 32 KB supported 2 GB partitions. But what’s the problem with bigger block sizes?

Fat-32 FAT-32 was introduced in the second release of Windows 98. FAT-32 represents each cluster in the file table with a 28-bit entry. The maximum partition size for a FAT-32 system is 2 Terabytes. A big advantage of FAT-32 is that for an equivalent partition size, FAT-32 blocks can be much smaller than FAT-16 blocks. Thus, the file system is more efficient. The downside is that the File Allocation Table is much bigger. For a 4KB block and a 2GB partition, the FAT takes up 2 MB of RAM.

The Windows 98 File System NT bit (for compatibility) Adds time accuracy to creation date/time - within 10ms Last access time bytes 8 3 1 10 2 2 2 4 File Name Ext Time Date Size Lower 16 bits of first block number Attribute read-only archive hidden system file Last write Upper 16 bits of first block number Creation date/time

Long File Names Windows 98 provides 2 file names for each file. Operating Systems Windows 98 provides 2 file names for each file. * The standard DOS 8 + 3 file name * A long file name The algorithm for creating an 8 + 3 file name is to truncate the file name to 6 characters and add ~1 to the name ( or ~2 if the name already exists).

Operating Systems Each long file name is stored in the directory, in front of the normal directory entry for the file. Up to 13 Unicode characters are stored in the following format. Multiple entries are used to Provide storage for the entire long file name. 5 characters 6 characters 2 chars Attribute byte 0x0F Sequence checksum

. . . The Big Long Name Operating Systems 65 a m e A C 1 T h e B A C C 1 T h e B A C i g L o n g N Creation Time T h e B I g ~ 1 A NT S . . . low The Big Long Name

Windows NT File System

NTFS does not use sectors. Instead it uses a cluster, which is some number (power of 2 ) sectors. This makes the file system independent of sector size.

free space bitmap boot sector system files master file table data MFT Records Record Header attribute-value pairs Attribute header data

Attribute header data one of: standard info file name attribute list (location of additional mft records, if needed) volume name data . . .