File Systems in Real-Time Embedded Applications March 5th Eric Julien Understanding How the File Allocation Table (FAT) Operates 1

Timeline Summary Microsoft worked actively on FAT for 20 years: Original 8-bit in 1977 FAT12 in 1980: 32 MB FAT16 – first version in 1984: 2 GB FAT32 – introduced in 1996: 2 TB Today: still the most common format for removable media (usb drives, sd cards, etc.) 2

FAT characteristics FAT is A disk file system, Designed to support sector-based devices, Very light-weight, Popular for removable media, Supported by major desktop OSes. 3

Master Boot Record (MBR) 4

Partitioning 5 *Each partition can become a volume

Volume Layout 6 *2 nd FAT optional Cluster Sec 1Sec 2Sec 3…Sec N

Reserved Area Boot Sector VBR (Volume Boot Record) VBR contains the BPB (BIOS Parameter Block) FS Info Sector (FAT32) Additional Reserved Sectors (optional) 7

Bios Parameter Block (BPB) 8 Bytes per sector (512, 1024, 2048 and 4096 are valid values) … Number of sectors in File System Number of sectors per FAT Number of sectors in File System (< 16 bits) Number of files in root directory. 512 for FAT16 Number FATs Number of sectors of Reserved Area Number of sectors per cluster. Power of 2, Max 32KB Signature value (0xAA55) 0 Offsets in bytes.

Data Area The data area: Is divided in clusters of equal size Contains file data Contains file metadata (name, size, etc.) Contains directory organization info Contains the Root Directory (FAT32 only) 9

File Allocation Table (FAT) The FAT is a cluster « map »: Each cluster is represented by an entry in the FAT Entry size depends on which FAT: 12/16/ … … Data area Data Cluster 38 Data Cluster 39 Data Cluster 40 Data Cluster 41 Data Cluster 42 Sector index: FAT Table

FAT Each entry indicates either: the next cluster number in a chain the EOC (end of chain) the cluster is bad the cluster is not used 11

Directory entries Directory entries are used to store metadata For a file For a directory For LFN (Long file names) 12

Directory entries The directory entries contain information about files and directories : File name and extension Timestamps First cluster number Size 13

An entry example 14 5,923

Files and directories The data contained in files is located in the data area. Data storage is allocated on a per-cluster basis. Each files consumes an integer number of clusters. 15

Files and directories A directory table (folder) is just a special type of file. Its data is a list of directory entries (that can point to other files and directory tables). 16 5,923

LFN (VFAT) The original FAT specification supported only 8.3 naming To support up to 255 UTF-16 characters, VFAT introduced the Long File Names (LFN) VFAT uses « special » directory entries (up to 20) that contain the LFN. 17

LFN (VFAT) 18 Normal Directory Entry … LFN Directory Entry Sequence Number File Name [1-5] [6-11] [12-13] Attributes ( 0x0f ) Checksum Reserved 031

File search With LFN enabled, a file name can be contained in up to 21 directory entries, each of them comprised of 32 bytes. Hence, a single file/folder can use up to 672 bytes in a directory table. This means it can span across multiple sectors. 19

File search If your file system suite was implemented to limit the usage of RAM, it is likely that internal buffers (used for file search) are equal to a sector size (512 bytes typically). 20

File search Example: Searching for a file in a folder containing 50 files. Worst case : 21 entries * 32 bytes * 50 files = bytes = 66 sectors to read = 66 read operations on file system 21

File system transactions Transactions can trigger up to 3 different operations Allocation/deallocation of clusters (FAT region) Creation/deletion/update of the directory entry (directory table, part of the data region or root directory) Update of the data (data region) 22

File system transactions These transactions are NOT atomic. An unexpected interruption (such as a power failure, or a « hot » media removal) can corrupt the file system AND the data. 23

File system and data corruption Incomplete transactions can create errors: Orphaned cluster chains Orphaned directory entries Chain length mismatch Unbounded cluster chains Cross-Linked files 24

Licensing Microsoft owns a patent for the support of long file names (LFN). Microsoft recently won a few times in court even if the validity has been disputed several times. The cost: $0.25 per unit, up to $250,

Future FAT32 is limited to 2 TB in volume size. File size is limited to 4GB without FAT+. Microsoft proposed exFAT (covered by patents). Not supported outside Windows and OS X. Recommended by the SDXC specification. 26