Files CS Spring 2002

Overview Example: FAT File System File Organization File System Organization –File Directories and File Sharing –Record Blocking –Disk space allocation Example: Unix File Systems

Fat Disk Structure Boot Sector Other Reserved File Allocation Table (FAT) File Allocation Table Copy Root Directory File and Directory Area Reserved Sectors Allocation chains and Free List Redundant Copy Fixed size Array of root directory entries All other directories and files are located here

FAT Boot Sector Location: First sector of the drive Contains: –Jump to first level boot code (byte offset 0) –Disk Parameter Block (byte offset 11) which describes the drive –Other identifying information like the volume label (byte offset 43) –First level boot code (typically at offset 0x40)

Disk Parameter Block Geometry –BytesPerSector[2] –SectorsPerTrack[2] –Heads[2] –Sectors[2] or LargeSectors[4] –Media –HiddenSectors[4] Structure Sizes –ReservedSectors[2] –NumberOfFats –SectorsPerFat[2] –NbrOfRootEntries[2] –SectorsPerCluster Field sizes in bytes are indicated in square brackets

FAT Directory Entries Typedef struct _DIRENT { UCHAR FileName[11]; UCHAR Attributes; UCHAR NtByte; UCHAR CreationCSec; FAT_TIME_STAMP CreationTime; FAT_DATE LastAccessDate; USHORT ExtendedAttributes; FAT_TIME_STAMP LastWriteTime; USHORT FirstClusterOfFile; ULONG FileSize; } DIRENT;

File Allocation Table xff7 1a 0 0xfff 14 8dot3 Name... First Cluster (15) File Size (5a0) 0xfff Directory Entry aFree Cluster Bad Cluster FAT12 FOO BAR... 1b 32f Dir Entry

How to translate FAT tables entries to sector offsets in the disk partition RootDirSize = NbrRootEntries * sizeof(DIRENT) / SectorSize FileAreaOffset = NbrReservedSectors + NbrFat*SectorsPerFat + RootDirSize SectorOffset = FileAreaOffset + (FAT_Offset - 2) * SectorsPerCluster Fat_Offset to SectorOffset

Long Filename Directory Entry typedef struct _LFN_DIRENT { UCHAR Ordinal; UCHAR Name1[10]; // 5 Unicode chars UCHAR Attributes; // Should be 0x0f UCHAR Type; UCHAR CheckSum; // of 8dot3 name WCHAR Name2[6]; USHORT MustBeZero; WCHAR Name3[2]; } LFN_DIRENT;

DIRENT with LFN 0x44 Characters 26 to 383 Characters 13 to 252 First 13 Unicode Chars1 Up to 13 Unicode Chars 8dot3 Name Ordinal All have same Checksum Field Calculate rotated checksum from this

File Structure CS Spring 2002 Segment II

File Organization Vocabulary –Field - basic element of data –Record - collection of related fields –File - collection of similar records File Organization Choice Criteria –Rapid access –Ease of updates –Economy of storage –Reliability

Some File Types Pile Sequential Indexed Sequential Indexed Direct or Hashed

Pile Variable length records with variable fields Fields must be self describing No order to records - exhaustive search F1F2F5 F1F3 Record 1 Record 2 F4 DataF4 Hdr F3

Sequential File Fixed length records of fixed format fixed length fields. Distinguished key field. Records sorted in key field order Search is typically sequential Insertions are not easy - so usually accumulated in separate overflow file with periodic update of the master file 17F2F3F4F5 24F2F3F4F5 26F2F3F4F5 14F2F3F4F5 Record 1 Record 2 Record 3 Record 4 Key

Indexed Sequential Files Sequential file with index file and overflow file for updates Records still in order of key index Overflow file also accessible through index - Records have Next_Record field which can point into overflow file. Multiple level so indexing can speed lookups

Indexed Sequential File Index Overflow File Master File Key

Indexed Files Index for each searchable field, but each record must have at least one searchable field Records not sorted, no overflow needed Allows for variable length fields and records with missing fields Exhaustive index - one entry per record, versus Partial index - on entry per record containing the indexed field.

Indexed File Partial Index Exhaustive Index Indexed File

Hashed Files Hashed access with overflow tables to handle collisions Usually used with fixed length records Very fast access No way to sequentially access all records

Hashed File Overflow Table Hashed File Pointers calculated via hashing function

File Systems CS Spring 2002 Segment III

Directories Contents –Name –File type and organization –Address: Size, location –Access control information –Usage Information - times, current usage Structure - e.g. directory tree Operations: Search, Create file, Delete file, and List Directory

File Sharing Access Rights - Allow various users or groups to share files Simultaneous Access control - Allow multiple threads to share the same file at the same time

Record Blocking Three approaches –Fixed Blocking –Variable length records with records restricted to a single block - no spanning –Variable length records which can overlap (span) two blocks First two may have internal fragmentation Third is hard to update

Record Blocking Methods Rec 1Rec 2Rec 3 Rec 4Rec 5Rec 6 Rec 1Rec 2Rec 3Rec 4 Rec 5Rec 6 R7 Rec 3Rec 4Rec 5 Rec 1Rec 2 Rec 4 Block 1 Block 2 Block 1 Block 2 Block 1 Block 2 Spanning Records Fragment

Disk Space Allocation (1 of 2) Recorded in File Allocation Table Pre-allocation (at file creation) versus dynamic allocation of space Allocation size - variable or fixed, large or small, contiguous or not Allocation Methods –Contiguous - just need address and total size –Chained - pointer to next block –Indexed - index refers to file portions

Allocation Methods 0UU Indexed Chained Contiguous

Disk Space Allocation (2 of 2) Free Space Management (Disk Allocation Table) –Bit maps of Free Blocks –Chained Free Portions –Index of Free Portions Reliability Problems –Caused by improper shutdown when File and Disk allocation tables kept in memory –Locking is often too expensive

Free List Management Indexed Chained Bit Map FreeAllocated Free

Unix File System CS Spring 2002 Segment IV

Classic Unix File System Superblock Inode Table Files and Directories Superblock Free Inode List Free Block List Inode Table Buffer Cache Main Memory Disk Drive File Table

V6 Unix Inode Entries i_mode - type (dir, char or block special, or ordinary) and file permissions i_nlink - number of names (hard links) i_uid, i_gid - user and group owners i_size0, i_size1 - size of file in bytes i_ addr[8] - location of file on disk i_atime, i_mtime - access and modification times

Classic V7 Unix Inode Rest of inode 256 More Block Nbrs 10 Direct Block Nbrs 256 More Indirect Block Nbrs 256 Block Nbrs 255 More of these Block number of Triple Indirect Block Inode File location part of inode Double Indirect Block Single Indirect Block

Unix File Allocation Limits Level Nbr of Blks Nbr of Bytes Direct10 10 KB Single Indirect KB Double Indirect256 x MB = 64 K Triple Indirect256 x 64 K 16 GB = 16 M

Classic Unix Free Lists Superblock Free Block List 255 Free Block Nbrs Superblock Free Disk Inode List One per disk Partition One per disk Partition

Summary File Organization File System Organization –File Directories –File Sharing –Record Blocking –Disk space allocation Examples: Unix and FAT File Systems