1. Project 3: File System Design COS318 Fall 2002

2. Last Time Web Server Extensive use of a file system on server machine without actually worrying about how it was implemented. request response serverclient u/ vivek/ public_html/ index.html cos318_02/

3. Outline Introduction Design requirement Helper functions Implementation of file system calls Hints Reference

4. Introduction Why do we need a file system? –Abstraction –Protected access –Security Simple Unix-like file system –i-node –directory tree –links(symbolic/physical)

5. What does my file server need to accomplish A New Client Program with New Methods: –GET Fetch a file or dir listing and show them –CREATE Create a dir in server disk –DELETE Delete a file or dir in server disk populated directories should not be deleted –PUT Create a new file in server disk, or append to an exsiting file with contents from local file [Notice: name uses absolute path]

6. More on file server functionality Use our client program to create a populated directory tree inside of server disk, and then use your web server to serve this content. More test programs: –Randomly requests –Maybe not just be PUT or GET: after some file deletions, things will become more interesting. –Coming soon…

7. So what do you have A Virtual Disk: a data structure –Disk_name: dev_318 –Disk_pointer: used to access the disk(in fact a file descriptor) –Sector_size: 512 bytes –Capacity: M, G … Rather than write to a real disk, you'll be building a disk inside of a large file All development and debugging are in user space

8. Helper Functions: not tuxedo GetExistingDisk(char *pathname) CreateNewDisk(void) CloseDisk(void) ReadSector(int sectorNum, char *buf) WriteSector(int sectorNum, char *buf) Notice: –You are allowed to access disk only with those function calls;

9. Helper Functions(cont.) GetExistingDisk(char *pathname) –returns a pointer to previously created disk in directory pathname, or exits if no such disk CreateNewDisk(void) –returns a pointer to a new disk CloseDisk(void) –closes the virtual disk(dev_318)

10. Helper Functions(cont.) ReadSector(int sectorNum, char *buf) –reads 512 bytes from disk WriteSector(int sectorNum, char *buf) –writes 512 bytes from disk Notice: –If any other user space function calls can facilitate your work, feel free to use them. Using file system functions to build disk is relatively easy compared to using really disk to build file system

11. What do I need to do You should build on the code you wrote for the previous assignments –Good thing: we have a reference server The first step is to understand the default file system provided by the base code(helper functions) You'll have to understand at least one new header with one of the new methods.

12. What do I need to do(cont.) You'll have to write versions of various system calls that you used in Project 1 –mkfs, open, close, read, and write –Additionally, you'll also have to implement versions of stat, mkdir, and a few others, also function calls to getting file metadata and directory information in order to properly build directory listings No need to handle all of the various flags and parameters, but should implement enough to get the project working correctly

13. One example-open() For example, the open() system call can be used to try to test the existence of a file in addition to creating it for writing O_CREATE: the file creation flag –To properly handle “PUT" method Not always true: need other flags –“GET” a non-existent file

14. mkfs File system creation –Dos: format –Linux: mkfs/mke2fs/mkfs.msdos –Solaris: newfs Structure of a typical file system(unix- like) –Superblock Describes layout of i-Nodes, bitmaps, blocks, root directory, etc. –i-Nodes Describes a file/dir – Link counter, list of blocks, size, etc. (owner, permission, …) –Block Allocation Bitmap –Data blocks

15. mkfs(cont.) Creates the file system –Set up the Superblock –Flag all i-Nodes as type free (f) –Zero out the Alloc Bitmap –Create the root directory Flag an i-Node as type directory (D) Store it’s number in Superblock Alloc one block in the bitmap Initialize the ‘.’ and ‘..’ directory entries and set the others as being INVALID Bootblock Kernel Procs Super Block i-Nodes Block Alloc Bitmap* Data Blocks

16. SuperBlock Describes the layout of the file system –signature – A magic number –size – Total size of fs in blocks –root_inode – The root directory –inode_start – 1 st block for i-Nodes –inode_blocks – Total i-Nodes blocks –bitmap_start – 1 st block of bitmap –bitmap_blocks – Total bitmap blocks –alloc_start – 1 st block for data –num_blocks – Total data blocks –free_blocks – Count of unallocated blocks

17. Bitmap –Indicates which data blocks are free, and which are used –Maps the data blocks only (not i-Nodes)

18. I-node Describes a file/directory –type – File, directory –opens – Count of open file handles –links – Count of links to this i-Node –size – Total byte count –direct[10] – List of data blocks –indirect[3] – List of indirect blocks Advantages –Fast access to small files –Supports very large files –Support sparse files

19. Hints Break this project into two components - expansion of the main server itself, and implementation of the underlying disk Within the file system: –Start with the creation of files and try to get that working properly; –Then build directories on top of that mechanism; –Finally work on directory listing support. For files: direct blocks first

20. More Hints Keep the high level interface as much as possible –By having wrappers that can either call the real call or your replacement, you'll be able to do testing quickly and efficiently Standard interface: –Stat(const char *pathname, struct stat *buf)

21. stat Returns statistics for given a filename –inodeNo – The file’s i-Node number –Type – File or directory –Opens – The open count –Size – The byte count to EOF –numBlocks – Number of allocated blocks

22. Reference Check the manual page of the function calls –What they do –What’s their interface –What’s the most useful parameters

23. Wrap Up Fun again? Probably…. Not too difficult. But a little bit more coding. Questions?