1. CENG 334 – Operating Systems 07- File Systems
Asst. Prof. Yusuf Sahillioğlu
Computer Eng. Dept, , Turkey

2. File Concept Users  Applications/Processes
We think and use files when we want to store something
Can your code write something to the disk without using files? NO
File is not a physical thing; it is an abstract entity
Actual data we’re writing to a file sits in the storage media (disk/CD)
OS provides that abstract entity via its File System component
How to place files on physical media
How to read/write ..

3. File Concept File concept illustration
Don’t deal with the disk directly
Instead deal with the files (logical storage: byte 0 to n)

4. File Concept
OS File System componenet is a software that views the hardware as a sequence of blocks of some size
Those blocks are mapped to sectors of the disk by Disk Driver
Given a block number Disk Driver finds the corresponding sector num

5. File Concept
OS File System componenet views the underneath storage as a sequence of blocks
Applications are viewing the storage as a set of FILES
Hardware has the storage as a set of sectors

6. File Concept
OS File System componenet views the underneath storage as a sequence of blocks
Applications are viewing the storage as a set of FILES
There has to be a mapping from FILES to blocks

7. File Concept
A file is sitting in some blocks (contiguous or non-contiguous)
If a file content needs to occupy 4 blocks, OS File System Component decides which 4 blocks should contain the content

8. File Concept
We will first see File System Interface (which functions provided)
Then we will see how those functions are implemented
File  block mapping
So we will understand OS File System Component thoroughly

9. File Concept
File is just a contiguous logical address space (a storage)
Actual content may be non-contiguous in the disk
OS makes the arrangement so that you view the file as a contiguous logical space/storage
What can we store in a file?
Data
numeric
character
binary
Program User’s (process) view of a file

10. File Structure None (no structure at all): sequence of words, bytes
Unix, Windows
Simple record structure: sequence of records
Lines
Fixed length
Variable length
Complex Structures
Formatted document: understood by Word program, not OS
Executable file: understood by OS
Can simulate last two with first method by inserting appropriate control characters
Who decides:
Operating system
Program

11. File Attributes Name – only information kept in human-readable form
Identifier – unique tag (number) identifies file within file system
Type – needed for systems that support different types
Location – pointer to file location on device
Size – current file size
Protection – controls who can do reading, writing, executing
Time, date, and user identification – data for protection, security, and usage monitoring
Information about files are kept in the directory structure, which is maintained on the disk, not in memory

12. File Attributes
There are 2 basic things stored on disk as part of the area controlled by the file system
Files: storage content
Directory info (can be a tree): info about files, attributes, locations
Organize files into a directory structure for efficient access
One entry per file: filename + pointer to attrbts

13. File Operations
File is an abstract data type: A class with attributes and operations
Create
Write
Read
Reposition within file
Delete
Truncate
These ops are implemented by the File Sys Component of the OS
These ops are used by the application programmer
Open(Fi) – search the directory structure on disk for entry Fi, and move the content of entry to memory
Close (Fi) – move the content of entry Fi in memory to directory structure on disk

14. File Operations Efficient open-file manangement: open-file table
Say a read-operation comes for an open file
Search directory on the disk (slow disk access) to locate entry Fi
Go to that location indicated by the entry

15. File Operations Efficient open-file manangement: open-file table
Say a read-operation comes for an open file
Repeat searching the directory for each read operation: slow 
OS maintains an open-file table for efficiency

16. File Operations Efficient open-file manangement: open-file table
Only the open system call searches the directory, locates Fi, caches it into open-file table, and returns the index of open-file table entry to the process: File Descriptor

17. File Operations Efficient open-file manangement: open-file table
Subsequent operations (read, write, ..) can then use File Descriptor handle without any further disk access

18. File Operations Efficient open-file manangement
File pointer: pointer to last read/write location, per process that has the file open
File-open count: counter of number of times a file is open – to allow removal of data from open-file table when last processes closes it
Disk location of the file: cache of data access information
Access rights: per-process access mode information

19. File Operations Each process has its own file-poisitoin pointer
System-wide open file table: location on disk & open-count same forall

20. File Types

21. File Operations
You can manually start the app that reads the file and load the file in it
Or OS automatically starts the app that reads the file (association)

22. File Access Methods Sequential Access (fscanf) read next write next
reset
no read after last write
(rewrite)
Direct Access
read n
write n
position to n
rewrite n
n = relative block number

23. File Access Methods
Sequential Access

24. File Sharing Sharing of files on multi-user systems is desirable
Sharing may be done through a protection scheme
On distributed systems, files may be shared across a network
Network File System (NFS) is a common distributed file-sharing method

25. File Sharing
User IDs identify users, allowing permissions and protections to be per-user
Group IDs allow users to be in groups, permitting group access rights
Attributes for protection

26. File Sharing Protection is based on the use of UserIDs and GroupIDs
Each file has associated protection bits (permissions) for userID and groupID
userID: read, write, execute?
groupID: read, write, execute?

27. File Sharing Remotely
Idea: open, read, write, .. file as if it is a local file

28. File Sharing Remotely
Uses networking to allow file system access between systems
Manually via programs like FTP
Automatically, seamlessly using distributed file systems
Semi automatically via the world wide web
Client-server model allows clients to mount remote file systems from servers
Server can serve multiple clients
Client and user-on-client identification is insecure or complicated
NFS is standard UNIX client-server file sharing protocol
CIFS is standard Windows protocol
Standard operating system file calls are translated into remot calls
Distributed Information Systems (distributed naming services) such as LDAP, DNS, NIS, Active Directory implement unified access to information needed for remote computing

29. File Protection File owner/creator should be able to control:
what can be done (read, write, execute, ..)
by whom (owner, others, group member, ..)
Types of access
Read
Write
Execute
Append
Delete
List

30. File Protection How it is done in Unix
Mode of access: read, write, execute
Three classes of users
RWX
a) owner access 7  RWX
b) group access 6  1 1 0
c) public access 1  0 0 1
Ask manager to create a group (unique name), say G, and add some users to the group.
For a particular file (say game) or subdirectory, define an appropriate access.

31. File Protection
A sample Unix directory listing

32. File System Implementation
File system design involves
Defining File System Interface (Done!)
How file system looks to the user
What is a file and its attributes
What are the operations
Directory structure to organize files
How that File System can be implemented (Now!)
Design algorithms
Design data structures (in-memo and on-disk structures)
Map logical file system to physical storage device (disk, CD, ..)
Mapping depends on the storage device

33. File System Implementation
File control block: storage structure consisting of info about file (on disk)
Layered file system
file sys
device drivers

34. File System Implementation
Layers

35. File System Implementation
Layers

36. File System Implementation
Block to sector mapping by Disk Driver

37. File System Implementation
Now our problem is: how to map files on disk blocks?
Don’t care about details (blocksector) underneath (driver handles)
Block size is a multiple of sector size
Sector size = 512 bytes; block size can be 1024 or 4096 bytes
Unix: 4KB

38. File System Implementation
Now our problem is: how to map files on disk blocks?
2 files occupying 2 (red) and 3 (blue) blocks

39. File System Implementation
Now our problem is: how to map files on disk blocks?
2 files occupying 2 (red) and 3 (blue) blocks

40. File System Implementation
Now our problem is: how to map files on disk blocks?
2 files occupying 2 (red) and 3 (blue) blocks
Eventually

41. File System Implementation
To implement a File System, file system software maintains major on-disk structures, conisiting of
Boot control block: contains info needed by system to boot OS from that volume (power on, 1st block of disk accessed (loaded into memory), a small program is run which knows where kernel is)
Volume control block: contains vol details (how many blocks, directory location, ..)
Directory structure: organizes files
Per-file file control block (FCB) contains many details about the file

42. File System Implementation
File system gets commands from the processes and issues to the driver
Driver than works on the disk

43. Directory Implementation
When you want to open/read/write/.. a file, you search the directory, for a given file name, to find the corresponding info of the file
Directory on disk; so fast search required
Linear list of file names with pointer to the data blocks
Simple to program
Time-consuming to execute
Hash table: linear list with hash data structure
Decreases directory search time
Collisions: situations where two file names hash to the same location
Only good if entries are fixed size, or use chained-overflow method

44. Allocation MEthods
How can we allocate disk space to files? Important issue!
A disk is seen (to the file sys) as a sequence of blocks
How disk blocks are allocated for files
Contiguous allocation
Linked allocation
Indexed allocation

45. Contiguous Allocation
Each file occupies a set of contiguous blocks on the disk
Simple: only starting location (block#) and length (num of blocks) are required to find out the disk data blocks of file
Random access is fast
Waste of space (dynamic storage-alloc problem; ext. fragmentation)
Files cannot grow (can grow only if block10 is empty)

46. Contiguous Allocation
Given an offset (byte X of file, or equivalenty, LA = X), what is the corresponding disk location?
Assume block size 1024 bytes
Wheich disk block contains the byte 0 of file X (LA=0)? What is the displacement on that block?
Answeer: Disk block=6 & displacement (disk block offset) = 0
Which disk block contains the byte
at LA=2500? Where is LA=2500 mapped
on disk?
Answer: 2500/1024 = 2
2500%1024 = 452
Disk block = startAddr + 2 = = 8 & displacement = 452

47. Contiguous Allocation

48. Extent-Based Systems Problem with contiguous allocation: file growing
A modified contiguous allocation scheme (newer systems)
Initially make a guess: size of my file will be X
When file tries to grow more, alloc another extent and point there
Extent-based file systems allocate disk blocks in extents
An extent is a contiguous block of disks
Extents are allocated for file allocation
A file consists of one or more extents

49. Linked Allocation
Each file is a linked list of disk blocks: blocks may be noncontiguous
Block structure:

50. Linked Allocation File starts at block5
Now you can write some data in block5
If block5 filled up and more to write, find another empty block (3)

51. Linked Allocation

52. Linked Allocation Simple: need only startingAddress
Info to be maintined in the directory entry (or FCB) is just 5 (sli50)
In contiguous alloc, we additionaly maintain num of blocks (slid45)
No waste of space (no external fragmentation problem)
No random access (not easy)

53. Linked Allocation
Given an offset (byte X of file, or equivalenty, LA = X), what is the corresponding disk location?
Logical to physical mapping
dataSize = blockSize – pointerSize
blockNumber = X / dataSize
displacemnt (disk block offset) = X % dataSize
Assume block size = 1024 bytes; pointer size = 4 bytes
Assume file size = 4000 bytes
Find the disk location corresponding to X=LA=2900?
blockNum = 2900 / 1020 = 2 & 2900 % 1020 = = 864

54. File Allocation Table (FAT)
Prev, wasting a (small) space in each data block for the pointer part
Data size in a block is no longer a power of 2
Collect all these pointers into a table: File Allocation Table
# entries in table = # blocks on the disk
Used in MS-DOS
Data size is a power of 2

55. File Allocation Table (FAT)
All pointers collected into FAT
How many blocks allocated to this file? 3

56. File Allocation Table (FAT)
You can load FAT into memory if it is not that big
Then random access to file’ll be quite fast (unlike linked alloc on disk) 3

57. Indexed Allocation
Brings all pointers together into the index block (not linked anymore)
Logical view:
For sequential reading of the file from beginning to end (= access all the data blocks of the file), access index table, go from entry0 to N
Index table size = # blocks in file 3

58. Indexed Allocation
19: address of the disk block that is containing the index table of file 3

59. Indexed Allocation
Random access fast: just go to the 4th entry in table to access block3
No waste of space (no external fragmentation)
Mapping from logical file addresses to physical block numbers
displacement into index table = LA / blockSize
Learn disk block address from that entry
Displacement into disk block = LA % blockSize
Block size 512 bytes; what can be the max size of a file mapped by Index table approach?
Index table sits in one block (called index block): 512 entries in tabl
Each index entry points to a data block of 512 bytes
Using a single table, you can point 512 blocks  maxFileSize=
512x512bytes=256KB 3

60. Indexed Allocation For larger files, we need other index blocks
Instead of pointing to a disk block, outer index table points to another index table, which in turn points to many blocks (instead of just 1) 3

61. Indexed Allocation
Unix uses indexed allocation and a small portion of the index table is kept in a structure called inode
For every file in a unix file system, we have a corresponding inode
A ptr to this inode structure is in directory entry associated w/ fname
indirects point to index blocks,
which includes ptrs to data blcks 3

62. Free Space Management Done: Allocation of disk blocks to files
How to locate the disk blocks allocated to a file
Now: Keep track of free blocks of the disk
Necessary while growing a file (need free blocks)
Bit vector (bitmap) method
Linked list method
Grouping
Counting 3

63. Free Space Management Bit vector method (used in Unix file system)
We have a bit vector (bitmap) where we have one bit per block indicating if the block is used or free
If the block is free, the corresponding bit is 1, else 0 3

64. Free Space Management Bit vector method
Search word by word for efficiency 3

65. Free Space Management Bit vector method
Cons: Bitmap requires extra space (to be stored)
Block size 2^12 bytes
Disk size 2^30 bytes = 1GB
N = 2^30/2^12 = 2^18 blocks exist on disk
Hence we need 2^18 bits in bitmap, which makes 2^18 / 8 / 1024 = 32KB space just to store the bitmap
Pros: Easy to get contiguous files
Pros: Blocks of a file can be kept close to each other (fseek) 3

66. Free Space Management Linked list method
Each free block has pointer to the next free block
We keep a pointer to the 1st free block (in some special loction in disk)
Cons: cannot get contiguous space easily
Pros: no waste of space 3

67. Free Space Management
Linked list method 3

68. Free Space Management Grouping
A free block contains multiple free block pointers/addresses 3

69. Free Space Management Counting
Besides the free block pointer, keep a counter saying how many blocks are free contiguously after that free block 3