Lecture 11: The FAT, VFAT, and NTFS Filesystems 6/19/2003 CSCE 590 Summer 2003

NTFS Basics The Windows NT file system is NTFS NTFS is designed for high performance on very large hard disks (read, write and file-system recovery) Formatting a volume with the NTFS file system results in: –Master Files Table (MFT) –System Files Formatted NTFS Volume Partition Boot Sector Master File Table System Files File Area

NTFS Includes: –Security features –Data access control – permissions on files and folders –Folders can be shared –Individual compression of files and folders New features –Encryption –Disk quotas –Link tracking – tracks broken and moved shortcut links –Alternate Data Streams used more heavily

NTFS New Features (cont.) –Sparse files – files with large sections of empty data – compresses it –Reparse points – allows a drive to be mounted as a folder on another drive (NTFS version 5) –NTFS junctions – folders on one drive appear as folders on another –Hierarchical Storage Management – infrequently accessed files stored in offline storage, but still appears part of file system

All the World’s a File Basically everything on the volume is a file –Even file system metadata (info about file system itself) –Can see info about hidden files on NTFS (not NTFS5) dir /ah $MFT Everything in a file is an attribute: –Data attribute –Security attribute –File name attribute

NTFS Partition Boot Sector BIOS Parameter Block –Information on volume layout similar to FAT –File system structures Code –How to find code that loads operating system –On NT points to and loads NTLDR Duplicate boot sector located in the logical middle of the volume

NTFS System Files First 16 records in MFT contain metadata files: –0: Master File Table ($MFT) – index of files –1: Master File Table2 – a mirror of first 4 records ($MFTMIRR) –2: Log File – list of transaction steps for NTFS recovery ($LOGFILE) –3: Volume – volume name, NTFS version, serial number, dirty flag ($VOLUME) –4: Attribute Definition Table – table of attribute names, numbers, and descriptions ($ATTRDEF)

NTFS System Files (2) –5: Root Filename Index (.) –6: Cluster Bitmap – bitmap of clusters in use ($BITMAP) –7: Partition Boot Sector – if bootable, bootstrap code ($BOOT) –8: Bad Cluster File – locations of bad clusters ($BADCLUS) –9: Security File – security descriptors for all files in volume ($SECURE) Used to be $QUOTA in NT4

NTFS System Files (3) –10: Upcase Table – lowercase to Unicode uppercase ($UPCASE) –11: Extension File – optional extensions ($EXTEND) Quotas ($QUOTA) Reparse point data ($Reparse) Object identifiers ($ObjId) –12-15: are reserved for future use

$BITMAP File Maps clusters in use and free clusters One bit in bitmap for each cluster in filesystem 1 = in use 0 = available Kind of like a FAT

NTFS Master File Table MFT A database that contains info on every file in an NTFS volume Small files are stored entirely in MFT (< 1500 bytes) Large files are referred to with pointers Small directories are stored in the MFT Large directories point to clusters where their directory entries are stored

NTFS File Attributes NTFS file is a collection of attribute/value pairs Attributes residing entirely in MFT are said to be resident attributes $FILENAME - File name in Unicode –Short (8.3, case insensitive) –Long (255 Unicode characters) –Preserves case, but only distinguishes based on case for POSIX applications Allocation flag –0 = marked for deletion, unallocated

NTFS File Attributes (2) $STANDARD_INFORMATION –MAC timestamps –Standard attributes (read only, archive, …) –Hard link count –These attributes are always resident Attribute List – location of nonresident attributes stored in cluster elsewhere on the volume –Represented as array of [logical block #, physical block #, # of blocks]

NTFS File Attributes (3) Security Descriptor – owner and access rights $DATA – small files stored here –Has one unnamed data attribute –Can have multiple named data attributes (think ADS) –Too much data for MFT, addresses of clusters where data is actually stored is placed here

File Attribute Definitions Object ID – a volume-unique file identifier for distributed link tracking service Logged Tool Stream – like ADS, but changes are logged to NTFS Log File like metadata changes. (for EFS) Reparse Point – used for volume mount points

Folder/Directory Implementation Index Root – index entries for folder contents Index Allocation – too many index entries in a folder for the MFT, store remainder in index buffers in clusters and record cluster locations here Support for links (a file can be referenced by entries in multiple folders) Stored as binary trees, not lists for performance

Allocating a File 1.Bitmap modified to mark clusters as allocated 2.Create an allocated MFT record 3.Create index entry in parent folder’s MFT record or index buffer 4.For non-resident file: create cluster extent entries in file’s MFT record Extents are contiguous chunks of disk blocks

Deleting a File 1.Cluster references in $BITMAP file zeroed 2.MFT allocation flag zeroed – marked for deletion 3.Index entry deleted, all entries below it are moved up (except if it is the last entry) NTFS overwrites MFT entries marked for deletion before creating new ones Non-resident attributes of a file may still be found, even if MFT entry is deleted

$LOGFILE Keep track of filesystem transactions to enable recovery Transactions: operations that change file system data or the volume’s directory structure Keeps track of completion of steps in file deletion or creation – pick up where you left off, etc

Information in $LOGFILE Index entries from folders (name, MAC times) Copy of MFT record (all begin with word ‘File’ followed by hex character “2A” Link file headers Index buffers (begin with “INDX”) A good way to find file names that no longer exist elsewhere

File and Folder Permissions Special PermissionsFull Control ModifyRead & Execute List Folder Contents ReadWrite Traverse Folder/Execute FileXXXX List Folder/Read DataXXXXX Read AttributesXXXXX Read Extended AttributesXXXXX Create Files/Write DataXXX Create Folders/Append DataXXX Write AttributesXXX Write Extended AttributesXXX Delete Subfolders/FilesX DeleteXX Read PermissionsXXXXXX Change PermissionsX Take OwnershipX SynchronizeXXXXXX

NTFS Compression Individual files, entire folders, or volumes Can be read/written by any Windows application without 3 rd party compression utilities Compression filter driver in NTFS automatically compresses/decompresses in real-time when file is used Data stream contains info on whether any buffer in the stream is compressed by gap of empty space following compressed section

Encrypting File System EFS – symmetric key encryption + public key technology. EFS users are issued a digital certificate with public key/private key pair Uses logged on user’s keys Real-time automatic encryption/decryption when user works with encrypted files

Sparse Files Files with large sections of empty data – compresses it Large sections of zeroes are not allocated space

Recovery Deleted NTFS Files The example uses these parameters: –Total Sectors –Cluster size 512 bytes –One Sector per Cluster –MFT starts from offset 0x4000, non-fragmented –MFT record size 1024 bytes –MFT Size 1968 records

Example Scan all 1968 MFT for flag indicating file marked for deletion MFT record number 57 contains our recently deleted file "My Presentation.ppt“ MFT entries have predefined structure Begins with standard “FILE” record header 8 fields into record, in byte 23 is Flag field 00 = marked deleted 01 = in use

Offset A B C D E F C 45 2A C FILE*...?t! D G O ` DD A3 18 F1 C1 01 H SY?.nA B D8 48 E9 C0 01 C0 BF 20 A0 18 F1 C OHeA.A?.nA DD A3 18 F1 C SY?.nA x A0 5A Z B DD A3 18 F1 C DD A3 18 F1 C1 01 SY?.nA. SY?.nA C DD A3 18 F1 C DD A3 18 F1 C1 01 SY?.nA. SY?.nA D E C 02 4D M.Y.P F E E R.E.S.~.1...P.P F 00 6E T.i.o.n.0...€ h DD A3 18 F1 C SY?.nA DD A3 18 F1 C DD A3 18 F1 C1 01 SY?.nA. SY?.nA DD A3 18 F1 C SY?.nA D M.y..P.r.e.s E F 00 6E 00 e.n.t.a.t.i.o.n E p.p.t.€...H A0 6D B0 00 DC DC U U C0 00 DC E EB C U neA D0 FF FF FF FF yyyy‚yG E F

Example (2) Can see Flag set to deleted (in red) See short and long filenames (in blue) Offset 0x188, non-resident DATA attribute (green), interested in: –Compression unit size (0 = non-compressed) –Allocated size of attribute: (0xDC00 = bytes) –Real size of attribute: (0xDC00 = bytes) –Data Runs

Example – The Data Runs We find the values: 31 6E EB C –0x31 0x3 = 3 bytes allocated to hold the value of the start cluster offset 0x1 = 1 byte allocated to hold the value of the length of the data run = 0x6E 0x6E = 110 clusters 0xEBC404 = start cluster offset = x00 = end of data runs (not fragmented)

Example – Getting the Data Cluster size = 512 bytes 110 clusters * 512 bytes = bytes = Real/Allocated size attributes Since we have 512 byte clusters, offset =512 * = = 0x0989D600 Or use dd # dd if=./ntfs.img bs=51200 count=110 skip= | nc \ > –w 3 forensic.net 31337

References