1. Files CS 470 - Spring 2002

2. Overview Example: FAT File System File Organization File System Organization –File Directories and File Sharing –Record Blocking –Disk space allocation Example: Unix File Systems

3. Fat Disk Structure Boot Sector Other Reserved File Allocation Table (FAT) File Allocation Table Copy Root Directory File and Directory Area Reserved Sectors Allocation chains and Free List Redundant Copy Fixed size Array of root directory entries All other directories and files are located here

4. FAT Boot Sector Location: First sector of the drive Contains: –Jump to first level boot code (byte offset 0) –Disk Parameter Block (byte offset 11) which describes the drive –Other identifying information like the volume label (byte offset 43) –First level boot code (typically at offset 0x40)

5. Disk Parameter Block Geometry –BytesPerSector[2] –SectorsPerTrack[2] –Heads[2] –Sectors[2] or LargeSectors[4] –Media –HiddenSectors[4] Structure Sizes –ReservedSectors[2] –NumberOfFats –SectorsPerFat[2] –NbrOfRootEntries[2] –SectorsPerCluster Field sizes in bytes are indicated in square brackets

6. FAT Directory Entries Typedef struct _DIRENT { UCHAR FileName[11]; UCHAR Attributes; UCHAR NtByte; UCHAR CreationCSec; FAT_TIME_STAMP CreationTime; FAT_DATE LastAccessDate; USHORT ExtendedAttributes; FAT_TIME_STAMP LastWriteTime; USHORT FirstClusterOfFile; ULONG FileSize; } DIRENT;

7. File Allocation Table 18 0 0xff7 1a 0 0xfff 14 8dot3 Name... First Cluster (15) File Size (5a0) 0xfff Directory Entry 14 16 17 18 19 1aFree Cluster Bad Cluster FAT12 FOO BAR... 1b 32f Dir Entry

8. How to translate FAT tables entries to sector offsets in the disk partition RootDirSize = NbrRootEntries * sizeof(DIRENT) / SectorSize FileAreaOffset = NbrReservedSectors + NbrFat*SectorsPerFat + RootDirSize SectorOffset = FileAreaOffset + (FAT_Offset - 2) * SectorsPerCluster Fat_Offset to SectorOffset

9. Long Filename Directory Entry typedef struct _LFN_DIRENT { UCHAR Ordinal; UCHAR Name1[10]; // 5 Unicode chars UCHAR Attributes; // Should be 0x0f UCHAR Type; UCHAR CheckSum; // of 8dot3 name WCHAR Name2[6]; USHORT MustBeZero; WCHAR Name3[2]; } LFN_DIRENT;

10. DIRENT with LFN 0x44 Characters 26 to 383 Characters 13 to 252 First 13 Unicode Chars1 Up to 13 Unicode Chars 8dot3 Name Ordinal All have same Checksum Field Calculate rotated checksum from this

11. File Structure CS 470 - Spring 2002 Segment II

12. File Organization Vocabulary –Field - basic element of data –Record - collection of related fields –File - collection of similar records File Organization Choice Criteria –Rapid access –Ease of updates –Economy of storage –Reliability

13. Some File Types Pile Sequential Indexed Sequential Indexed Direct or Hashed

14. Pile Variable length records with variable fields Fields must be self describing No order to records - exhaustive search F1F2F5 F1F3 Record 1 Record 2 F4 DataF4 Hdr F3

15. Sequential File Fixed length records of fixed format fixed length fields. Distinguished key field. Records sorted in key field order Search is typically sequential Insertions are not easy - so usually accumulated in separate overflow file with periodic update of the master file 17F2F3F4F5 24F2F3F4F5 26F2F3F4F5 14F2F3F4F5 Record 1 Record 2 Record 3 Record 4 Key

16. Indexed Sequential Files Sequential file with index file and overflow file for updates Records still in order of key index Overflow file also accessible through index - Records have Next_Record field which can point into overflow file. Multiple level so indexing can speed lookups

17. Indexed Sequential File 5 42 35 18 9 11 18 35 42 13 38 15 5 Index Overflow File Master File Key

18. Indexed Files Index for each searchable field, but each record must have at least one searchable field Records not sorted, no overflow needed Allows for variable length fields and records with missing fields Exhaustive index - one entry per record, versus Partial index - on entry per record containing the indexed field.

19. Indexed File Partial Index Exhaustive Index Indexed File

20. Hashed Files Hashed access with overflow tables to handle collisions Usually used with fixed length records Very fast access No way to sequentially access all records

21. Hashed File 8 37 91 231 19 54 14 83 24 87 23 43 Overflow Table 37 87 23 91 24 14 Hashed File Pointers calculated via hashing function

22. File Systems CS 470 - Spring 2002 Segment III

23. Directories Contents –Name –File type and organization –Address: Size, location –Access control information –Usage Information - times, current usage Structure - e.g. directory tree Operations: Search, Create file, Delete file, and List Directory

24. File Sharing Access Rights - Allow various users or groups to share files Simultaneous Access control - Allow multiple threads to share the same file at the same time

25. Record Blocking Three approaches –Fixed Blocking –Variable length records with records restricted to a single block - no spanning –Variable length records which can overlap (span) two blocks First two may have internal fragmentation Third is hard to update

26. Record Blocking Methods Rec 1Rec 2Rec 3 Rec 4Rec 5Rec 6 Rec 1Rec 2Rec 3Rec 4 Rec 5Rec 6 R7 Rec 3Rec 4Rec 5 Rec 1Rec 2 Rec 4 Block 1 Block 2 Block 1 Block 2 Block 1 Block 2 Spanning Records Fragment

27. Disk Space Allocation (1 of 2) Recorded in File Allocation Table Pre-allocation (at file creation) versus dynamic allocation of space Allocation size - variable or fixed, large or small, contiguous or not Allocation Methods –Contiguous - just need address and total size –Chained - pointer to next block –Indexed - index refers to file portions

28. Allocation Methods 0UU213 021 3543 21 012 5 Indexed Chained Contiguous

29. Disk Space Allocation (2 of 2) Free Space Management (Disk Allocation Table) –Bit maps of Free Blocks –Chained Free Portions –Index of Free Portions Reliability Problems –Caused by improper shutdown when File and Disk allocation tables kept in memory –Locking is often too expensive

30. Free List Management 0 1442205 4324344333 321 51522 1 Indexed Chained Bit Map 1100101111 FreeAllocated Free

31. Unix File System CS 470 - Spring 2002 Segment IV

32. Classic Unix File System Superblock Inode Table Files and Directories Superblock Free Inode List Free Block List Inode Table Buffer Cache Main Memory Disk Drive File Table

33. V6 Unix Inode Entries i_mode - type (dir, char or block special, or ordinary) and file permissions i_nlink - number of names (hard links) i_uid, i_gid - user and group owners i_size0, i_size1 - size of file in bytes i_ addr[8] - location of file on disk i_atime, i_mtime - access and modification times

34. Classic V7 Unix Inode Rest of inode 256 More Block Nbrs 10 Direct Block Nbrs 256 More Indirect Block Nbrs 256 Block Nbrs 255 More of these Block number of Triple Indirect Block Inode File location part of inode Double Indirect Block Single Indirect Block

35. Unix File Allocation Limits Level Nbr of Blks Nbr of Bytes Direct10 10 KB Single Indirect256 256 KB Double Indirect256 x 256 64 MB = 64 K Triple Indirect256 x 64 K 16 GB = 16 M

36. Classic Unix Free Lists Superblock Free Block List 255 Free Block Nbrs Superblock Free Disk Inode List One per disk Partition One per disk Partition

37. Summary File Organization File System Organization –File Directories –File Sharing –Record Blocking –Disk space allocation Examples: Unix and FAT File Systems