1. UNIX File Systems
(based on Chap 4. in the book “the design of the UNIX OS”)
Acknowledgement : Soongsil Univ. Presentation Materials
단국대 최종무 교수님 강의노트

2. Hard Disk Drive Structure
Characteristics
Seek time, rotational latency, transmission time
Disk Scheduling?, cylinder group?, Buffering?

3. Flash Memory limitation Seek optimization 이 불필요
Block erasure : block 단위로 지워짐.
Memory wear : # of erase limiation
Read disturb
Seek optimization 이 불필요
Require strong reliability (e.g. power failure recovery)

4. File System
 the system in which files are named and where they are placed logically for storage and retrieval
컴퓨터에서 파일이나 자료를 쉽게 발견 및 접근(read/write)할 수 있도록 보관/조직하는 체제
Properties
Hierarchical structure
Ability to create, read/write, and delete files
Dynamic growth of files
Protection of file data

5. File System 하는일 Support file interfaces to user File management
File interface: open, close, read, write, …
Directory manipulation: mkdir, cd, rmdir,…
File system interfaces: mount, mkfs, fsck, …
File management
i-node, FAT
Access control, attribute control, ..
Storage management
Disk management (Block allocation, I/O handling, …)

6. OS별 File Systems UNIX File System LINUX : EXT2, EXT3, EXT4
Windows : NTFS
OS X (Apple Mac OS) : HFS+
Traditional OS (MS-Windows 9x) : FAT32

7. FAT Size limitation of FAT32 단순/빠르고 호환성이 좋음 (대부분의 OS에서 사용가능)
32GB보다 큰 파티션을 만들수 없다
4GB 초과하는 파일을 만들수 없다
단순/빠르고
호환성이 좋음 (대부분의 OS에서 사용가능)

8. FAT
File Allocation Table

9. FAT

10. NTFS MS-Windows 계열 OS 기본 file system 보안 (사용 권한 및 암호화)
신뢰성 (파일시스템 오류 자동 복구)
Storage growth
Support for Large volume and file sizes
(대용량 파일(16EB)/볼륨 지원)
호환성 떨어짐 (Mac OS에서는 read만 지원, LINUX에서는 배포판/버전에 따라 불분명)

11. UNIX File System Overview
Boot block : 부팅에 필요한 정보 저장
Super block : file system 전체 정보 저장
i-list : 각 file당 하나의 i-node 할당, file 정보 저장
Data blocks : 실제 파일 내용을 블록 단위 저장
Block : 데이터를 한번에 읽고 쓰는 단위 (주로 1KB~4KB)

12. File System Layout
Root directory의 i-node 는 미리 정해져 있음.

13. Sample File System

14. File Sharing
Each user has a “file descriptor table” (or “per-user open file table”)
Each entry in the channel table is a pointer to an entry in the system-wide “open file table”
Each entry in the open file table contains a file offset (file pointer) and a pointer to an entry in the “memory-resident i-node table”
If a process opens an already-open file, a new open file table entry is created (with a new file offset), pointing to the same entry in the memory-resident i-node table
If a process forks, the child gets a copy of the channel table (and thus the same file offset)

15. UNIX File System Overview

16. EXT2 file system LINUX 기본 file system Ext2 특징 Ext3 특징 Ext4 특징
block group 도입 : 같은 파일은 같은 block group (하드 디스크상 같은 cylinder)을 가능한 사용하여 성능 향상(disk seek time, fragmentation )
Ext3 특징
Journaling 기능 도입: Logging and Fast Recovery
Ext4 특징
Large size 파일 지원

17. UNIX File System

18. Table of Contents Inodes Structure of a regular file Directories
Conversion of a path name to an Inode
Super block
Inode assignment to a new file
Allocation of disk blocks
Other file types
Summary

19. Summary
Inode is the data structure that describes the attributes of a file, including the layout of its data on disk.
Two versions of the inode
Disk copy : store the inode information when file is not in use
In-core copy : record the information about active files.
Inode disk blocks
Directories : files that correlate file name components to inode numbers
namei : convert file names to inodes
Inode assignment
Disk block assignment

20. Table of Contents Inodes Structure of a Regular File Directories
Conversion of a Path Name to an Inode
Super Block
Inode Assignment to a New File
Allocation of Disk Blocks
Summary

21. Definition of Inodes Every file has a unique inode
Contain the information necessary for a process to access a file
Exist in a static form on disk
Kernel reads them into an in-core inode to manipulate them.

22. Contents of Disk inodes
File owner identifier (individual/group owner)
File type (regular, directory,..)
File access permission (owner,group,other)
File access time
Number of links to the file
Table of contents for the disk address of data in a file (byte stream vs discontiguous disk blocks)
File size
* inode does not specify the path name that access the file

23. Sample Disk Inode File owner identifier File type
File access permission
File access time
Number of links to the file
Table of contents for the disk address of data in a file
File size
Owner mjb
Group os
Type regular file
Perms rwxr-xr-x
Accessed Oct :45 P.M
Modified Oct :3 A.M
Inode Oct :30 P.M
Size 6030 bytes
Disk addresses

24. Distinction Between Writing inode and File
File change only when writing it.
Inode change when changing the file, or when changing its owner, permisson,or link settings.
Changing a file implies a change to the inode,
But, changing the inode does not imply that the file change.

25. Contents of The In-core copy of The Inode
Fields of the disk inode
Status of the in-core inode, indicating whether
Inode is locked
Process is waiting for the inode to become unlocked
In-core inode differs from the disk copy as a result of a change to the data in the inode
In-core inode differs from the disk copy as a result of a change to the file data
Inode number (linear array on disk, disk inode does not need this field)
Reference count

26. Inode Lock and Reference Count
Kernel manipulates them independently
Inode lock
Set during execution of a system call to prevent other processes from accessing the inode while it is in use.
Kernel releases the lock at the conclusion of the system call
Reference count
Kernel increase/decrease when reference is active/inactive
Prevent the kernel from reallocating an active in-core inode

27. Table of Contents Inodes Structure of a Regular File Directories
Conversion of a Path Name to an Inode
Super Block
Inode Assignment to a New File
Allocation of Disk Blocks
Summary

28. Direct and Indirect Blocks in Inode
single indirect
double indirect
triple indirect
Inode
Data Blocks

29. Byte Capacity of a File System V UNIX. Assume that Run with 13 entries
1 logical block : 1K bytes
Block number address : a 32 bit (4byte) integer
1 block can hold up to 256 block number (1024byte / 4byte)
10 direct blocks with 1K bytes each=10K bytes
1 indirect block with 256 direct blocks= 1K*256=256K bytes
1 double indirect block with 256 indirect blocks=256K*256=64M bytes
1 triple indirect block with 256 double indirect blocks=64M*256=16G
Size of a file : 4G (232), if file size field in inode is 32bits
* Refer to the link on our class webpage for the case of current UNIX file system.

30. Byte Offset and Block Number
Process access data in a file by byte offset.
The file starts at logical block 0 and continues to a logical block number corresponding to the file size
Kernel accesses the inode and converts the logical file block into the appropriate disk block

31. Conversion of Byte Offset to Block Number
Algorithm bmap /* block map of logical file byte offset to file system block */
Input : inode, byte offset
Output: (1)block number in file system, (2)byte offset into block,
(3)bytes of I/O in block, (4)read ahead block number
calculate logical block number in file from byte offset;
calculate start byte in block for I/O; /* output 2 */
calculate number of bytes to copy to user; /* output 3 */
check if read-ahead applicable, mark inode; /* output 4*/
determine level of indirection;
while(not at necessary level of indirection)
calculate index into inode or indirect block from logical block number in file;
get disk block number from inode or indirect block;
release buffer from previous disk read, if any (algorithm brelse);
if(no more levels of indirection) return (block number);
read indirect disk block (algorithm bread);
adjust logical block number in file according to level of indirection;

32. Block Layout of a Sample File and Its inode
4096
228
45423
11111
101
367
428
9156
824
331
3333
Single indirect
Double indirect
Data block 75
Byte 9000 in a file -> 8block 808th byte 8 8
816th byte
(10K+256K)
Byte 350,000 in a file 11

33. Block Entry in the Inode is 0
Logical block entry contain no data.
Process never wrote data into the file at that byte offset
No disk space is wasted
Cause by using the lseek and write system call

34. Table of Contents Inodes Structure of a Regular File Directories
Conversion of a Path Name to an Inode
Super Block
Inode Assignment to a New File
Allocation of Disk Blocks
Summary

35. Directories A directory is a file
Its data is a sequence of entries, each consisting of an inode number and the name of a file contained in the directory
Path name is a null terminated character string divided by “/”
Each component except the last must be the name of a directory, last component may be a non-directory file

36. Directory Layout for /etc
Each entry : inode number and filename
Inode Number
(2 bytes)
File Names 83 2
1798
1276
... 95
188 . ..
init
fsck …
crash
mkfs
inittab

37. Directories : How How do you find files? How do you list files (ls)
Read the directory, search for the name you want (checking for wildcards)
How do you list files (ls)
Read directory contents, print name field
How do you list file attributes (ls -l)
Read directory contents, open inodes, print name + attributes

38. Table of Contents Inodes Structure of a Regular File Directories
Conversion of a Path Name to an Inode
Super Block
Inode Assignment to a New File
Allocation of Disk Blocks
Summary

39. Algorithm for Conversion of a Path Name to an Inode
Algorithm namei /* convert path name to inode */
Input : path name
Output : locked inode {
if(path name starts from root) working inode = root inode (algorithm iget);
else working inode = current directory inode (algorithm iget);
while(there is more path name){
read next path name component from input;
verify that working inode is of directory,access permission OK;
if(working inode is of root and component is “..”)
continue; /* loop back to while */
read directory (working inode) by repeated use of algorithms
bmap,bread and brelse; …

40. Algorithm for Conversion of a Path Name to an Inode
if(component matches an entry in directory (working inode)){
get inode number for matched component;
release working inode (algorithm iput);
working inode=inode of matched component(algorithm iget); } else /* component not in directory
return (no inode); }
return (working inode);

41. Table of Contents Inodes Structure of a Regular File Directories
Conversion of a Path Name to an Inode
Super Block
Inode Assignment to a New File
Allocation of Disk Blocks
Other File Types
Summary

42. Super block File System Super block consists of
the size of the file system
the number of free blocks in the file system
a list of free blocks available on the file system
the index of the next free block in the free block list
the size of the inode list
the number of free inodes in the file system
a list of free inodes in the file system
the index of the next free inode in the free inode list
lock fields for the free block and free inode lists
a flag indicating that the super block has been modified
boot block super block inode list data blocks

43. Table of Contents Inodes Structure of a Regular File Directories
Conversion of a Path Name to an Inode
Super Block
Inode Assignment to a New File
Allocation of Disk Blocks
Summary

44. Inode Assignment to a New File
Super block contains an array to store the index numbers of free inodes in the file system
remembered inode
이 inode보다 작은 번호의 inode는 superblock free list에 없음을 보장한다.
reduce free inode search time

45. Algorithm for Assigning New Inodes
Algorithm ialloc /* allocate inode */
Input : file system
Output : locked inode {
while(not done){
if(super block locked) {
sleep(event super block becomes free); continue; }
if(inode list in super block is empty){
lock super block;
get remembered inode for free inode search;
search disk for free inodes until super block full,
or no more free inodes (bread and brelese);
unlock super block;
wake up (event super block becomes free);
if(no free inodes found on disk) return (no inode);
set remembered inode for next free inode search;

46. Algorithm for Assigning New Inodes
/* there are inodes in super block inode list */
get inode number from super block inode list;
get inode (algorithm iget);
if(inode not free after all) {
write inode to disk;
release inode (algorithm iput);
continue; /* while loop */ }
/* inode is free */
initialize inode;
decrement file system free inode count;
return (inode);
} // end of while

47. Assigning Free Inode from Middle of List
free inodes empty
array1
Super Block Free Inode List
index
free inodes empty
array2

48. Assigning Free Inode – Super Block List Empty
index
empty
array1
Super Block Free Inode List
free inodes
array2
remembered inode

49. Algorithm for Freeing Inode
Algorithm ifree /* inode free */
Input : file system inode number
Output : none {
increment file system free inode count;
if(super block locked) return;
if(inode list full){
if(inode number less than remembered inode for search)
set remembered inode for search = input inode number;
} else
store inode number in inode list;
return; }

50. Placing Free Inode Numbers Into the Super Block
free inodes
remembered inode
Original Super Block List of Free Inodes
index
Free Inode 499
free inodes
remembered inode
index
Free Inode 601
free inodes
remembered inode
index

51. Table of Contents Inodes Structure of a Regular File Directories
Conversion of a Path Name to an Inode
Super Block
Inode Assignment to a New File
Allocation of Disk Blocks
Summary

52. Linked List of Free Disk Block Numbers
…………………………..
……………………
109
…………………… 214
211
……………………
310
Super block list

53. Algorithm for Allocating Disk Block
Algorithm alloc /* file system block allocation */
Input : file system number
Output : buffer for new block {
while(super block locked) sleep (event super block not locked);
remove block from super block free list;
if(removed last block from free list){
lock super block;
read block just taken from free list (algorithm bread);
copy block numbers in block into super block;
release block buffer (algorithm brelse);
unlock super block;
wake up processes (event super block not locked); } …

54. Algorithm for Allocating Disk Block…
get buffer form block removed from super block list (algorithm getblk);
zero buffer contents;
decrement total count of free blocks;
mark super block modified;
return buffer; }

55. Requesting and Freeing Disk Blocks
super block list
109 …………………………………………………………
109
…………………………….. 112
original configuration
…………………………………………………..
109
………………………………. 112
After freeing block number 949

56. Requesting and Freeing Disk Blocks
109 ………………………………………………………..
109
………………………………. 112
After assigning block number(949)
……………………………… 112
211
………………………………. 243
After assigning block number(109)
replenish super block free list