1. CENG334 Introduction to Operating Systems Erol Sahin Dept of Computer Eng. Middle East Technical University Ankara, TURKEY URL: http://kovan.ceng.metu.edu.tr/ceng334 Disks and Filesystems Topics: Disks

2. © 2006 Matt Welsh – Harvard University 2 Today: Disks and Filesystems Physical operation of modern disk drives Operating system access to raw disk and disk I/O scheduling Overview of filesystem design Next lecture: Detailed look at filesystem implementation Adapted from Matt Welsh’s (Harvard University) slides.

3. © 2006 Matt Welsh – Harvard University 3 A Disk Primer Disks consist of one or more platters divided into tracks Each platter may have one or two heads that perform read/write operations Each track consists of multiple sectors The set of sectors across all platters is a cylinder Aperture Platter Heads Track Sector Adapted from Matt Welsh’s (Harvard University) slides.

4. © 2006 Matt Welsh – Harvard University 4 Hard Disk Evolution IBM 305 RAMAC (1956) First commercially produced hard drive 5 Mbyte capacity, 50 platters each 24” in diameter! Adapted from Matt Welsh’s (Harvard University) slides.

5. © 2006 Matt Welsh – Harvard University 5 Hard Disk Evolution Adapted from Matt Welsh’s (Harvard University) slides.

6. © 2006 Matt Welsh – Harvard University 6 Disk access time-1 Command overhead: Time to issue I/O, get the HDD to start responding, select appropriate head Seek time: Time to move disk arm to the appropriate track Depends on how fast you can physically move the disk arm These times are not improving rapidly Settle time: Time for head position to stabilize on the selected track Adapted from Matt Welsh’s (Harvard University) slides.

7. © 2006 Matt Welsh – Harvard University 7 Disk access time-2 Rotational latency: Time for the appropriate sector to move under the disk arm Depends on the rotation speed of the disk (e.g., 7200 RPM) Transfer time Time to transfer a sector to/from the disk controller Depends on density of bits on disk and RPM of disk rotation Faster for tracks near the outer edge of the disk – why? Modern drives have more sectors on the outer tracks! Adapted from Matt Welsh’s (Harvard University) slides.

8. © 2006 Matt Welsh – Harvard University 8 Example disk characteristics IBM Ultrastar 36XP drive form factor: 3.5” capacity: 36.4 GB rotation rate: 7,200 RPM platters: 10 surfaces: 20 sector size: 512-732 bytes cylinders: 11,494 cache: 4MB transfer rate: 17.9 MB/s (inner) – 28.9 MB/s (outer) full seek: 14.5 ms head switch: 0.3 ms Disk interface speeds SCSI: From 5 MB/sec to 320 MB/sec ATA: from 33 MB/sec to 100 MB/sec Serial ATA (single wire): Starting at 150 MB/sec Firewire: 50 MB/sec Adapted from Matt Welsh’s (Harvard University) slides.

9. © 2006 Matt Welsh – Harvard University 9 Disk I/O Scheduling Given multiple outstanding I/O requests, what order to issue them? Why does it matter? Major goals of disk scheduling: Adapted from Matt Welsh’s (Harvard University) slides.

10. © 2006 Matt Welsh – Harvard University 10 Disk I/O Scheduling Given multiple outstanding I/O requests, what order to issue them? Why does it matter? Major goals of disk scheduling: 1) Minimize latency for small transfers Primarily: Avoid long seeks by ordering accesses according to disk head locality 2) Maximize throughput for large transfers Large databases and scientific workloads often involve enormous files and datasets Note that disk block layout also has a large impact on performance Where we place file blocks, directories, file system metadata, etc. This will be covered in future lectures Adapted from Matt Welsh’s (Harvard University) slides.

11. © 2006 Matt Welsh – Harvard University 11 Disk I/O Scheduling Given multiple outstanding I/O requests, what order to issue them? FIFO: Just schedule each I/O in the order it arrives What's wrong with this? Adapted from Matt Welsh’s (Harvard University) slides.

12. © 2006 Matt Welsh – Harvard University 12 Disk I/O Scheduling Given multiple outstanding I/O requests, what order to issue them? FIFO: Just schedule each I/O in the order it arrives What's wrong with this? Potentially lots of seek time! SSTF: Shortest seek time first Issue I/O with the nearest cylinder to the current one Why might this not work so well??? Adapted from Matt Welsh’s (Harvard University) slides.

13. © 2006 Matt Welsh – Harvard University 13 Disk I/O Scheduling Given multiple outstanding I/O requests, what order to issue them? FIFO: Just schedule each I/O in the order it arrives What's wrong with this? Potentially lots of seek time! SSTF: Shortest seek time first Issue I/O with the nearest cylinder to the current one Favors middle tracks: Head rarely moves to edges of disk SCAN (or Elevator) Algorithm: Head has a current direction and current cylinder Sort I/Os according to the track # in the current direction of the head If no more I/Os in the current direction, reverse direction CSCAN Algorithm: Always move in one direction, “wrap around” to beginning of disk when moving off the end Idea: Reduce variance in seek times, avoid discriminating against the highest and lowest tracks Adapted from Matt Welsh’s (Harvard University) slides.

14. © 2006 Matt Welsh – Harvard University 14 SCAN example Current track Direction Adapted from Matt Welsh’s (Harvard University) slides.

15. © 2006 Matt Welsh – Harvard University 15 SCAN example Current track Direction Adapted from Matt Welsh’s (Harvard University) slides.

16. © 2006 Matt Welsh – Harvard University 16 SCAN example Direction Current track Adapted from Matt Welsh’s (Harvard University) slides.

17. © 2006 Matt Welsh – Harvard University 17 SCAN example Direction Current track Adapted from Matt Welsh’s (Harvard University) slides.

18. © 2006 Matt Welsh – Harvard University 18 SCAN example Direction Current track Adapted from Matt Welsh’s (Harvard University) slides.

19. © 2006 Matt Welsh – Harvard University 19 SCAN example Direction Current track Adapted from Matt Welsh’s (Harvard University) slides.

20. © 2006 Matt Welsh – Harvard University 20 SCAN example Direction Current track Adapted from Matt Welsh’s (Harvard University) slides.

21. © 2006 Matt Welsh – Harvard University 21 SCAN example Direction Current track Adapted from Matt Welsh’s (Harvard University) slides.

22. © 2006 Matt Welsh – Harvard University 22 SCAN example Direction Current track Adapted from Matt Welsh’s (Harvard University) slides.

23. © 2006 Matt Welsh – Harvard University 23 SCAN example Direction Current track Adapted from Matt Welsh’s (Harvard University) slides.

24. © 2006 Matt Welsh – Harvard University 24 SCAN example Direction Current track Adapted from Matt Welsh’s (Harvard University) slides.

25. © 2006 Matt Welsh – Harvard University 25 SCAN example Direction Current track What is the overhead of the SCAN algorithm? Count the total amount of seek time to service all I/O requests In this case, 12 tracks in --> direction 15 tracks for long seek back 5 tracks in <-- direction Total: 12+15+5 = 32 tracks Adapted from Matt Welsh’s (Harvard University) slides.

26. © 2006 Matt Welsh – Harvard University 26 Filesystems A filesystem provides a high-level application access to disk As well as CD, DVD, tape, floppy, etc... Masks the details of low-level sector-based I/O operations Provides structured access to data (files and directories) Caches recently-accessed data in memory Hierarchical filesystems Organized as a tree of directories and files Byte-oriented vs. record-oriented files UNIX, Windows, etc. all provide byte-oriented file access May read and write files a byte at a time Many older OS's provided only record-oriented files File composed of a set of records; may only read and write a record at a time Versioning filesystems Keep track of older versions of files e.g., VMS filesystem: Could refer to specific file versions: foo.txt;1, foo.txt;2 Adapted from Matt Welsh’s (Harvard University) slides.

27. © 2006 Matt Welsh – Harvard University 27 Filesystem Operations Filesystems provide a standard interface to files and directories: Create a file or directory Delete a file or directory Open a file or directory – allows subsequent access Read, write, append to file contents Add or remove directory entries Close a file or directory – terminates access What other features do filesystems provide? Accounting and quotas – prevent your classmates from hogging the disks Backup – some filesystems have a “$HOME/.backup” containing automatic snapshots Indexing and search capabilities File versioning Encryption Automatic compression of infrequently-used files Should this functionality be part of the filesystem or built on top? Classic OS community debate: Where is the best place to put functionality? Adapted from Matt Welsh’s (Harvard University) slides.

28. © 2006 Matt Welsh – Harvard University 28 Filesystem Block Layout Filesystem defines a block size (typically 4KB) for all I/O operations Must be at least 1 sector Why ever use more than one sector? Each file on disk has an associated inode On-disk data structure defining location, access rights, etc. for a file A directory is just a list of inodes (one for each file in the directory) The root directory (“/”) is generally inode 0, and stored in a well-known location All other files and directories are accessed through the root directory Simplest allocation policy: contiguous files FS maintains a free block count (e.g., a bitmap), stored on disk and memory Must allocate all blocks of a file in a single contiguous chunk Growing a file may require relocating the whole thing to a new area of disk Simple design but clearly not very efficient! Lots of external fragmentation! Adapted from Matt Welsh’s (Harvard University) slides.

29. © 2006 Matt Welsh – Harvard University 29 Block Layout cont'd Somewhat better policy: Linked blocks Inode points to the first block of the file Each block points to the next block in the file (just a linked list on disk) What are the advantages and disadvantages?? Indexed files Inode contains a list of block numbers containing the file Array is allocated when the file is created What are the advantages and disadvantages?? inode Adapted from Matt Welsh’s (Harvard University) slides.

30. © 2006 Matt Welsh – Harvard University 30 Multilevel Indexed Files Inode contains a list of 10-15 direct blocks First few blocks of file Also contains a pointer to a single indirect, double indirect, and triple indirect blocks Allows file to grow to be incredibly large!!! inode direct blocks single-indirect blocks double-indirect blocks Adapted from Matt Welsh’s (Harvard University) slides.

31. © 2006 Matt Welsh – Harvard University 31 Disks are messy and slow Low-level interface for reading and writing sectors Generally allow OS to read/write an entire sector at a time No notion of “files” or “directories” -- just raw sectors So, what do you do if you need to write a single byte to a file? Disk may have numerous bad blocks – OS may need to mask this from filesystem Access times are still very slow Disk seek times are around 10 ms Although raw throughput has increased dramatically Compare to several nanosec to access main memory Requires careful scheduling of I/O requests Adapted from Matt Welsh’s (Harvard University) slides.

32. © 2006 Matt Welsh – Harvard University 32 ATA and IDE Interfaces IDE stands for Integrated (or “Intelligent”) Drive Electronics Same as “ATA” (Advanced Technology Attachment) Standard interface to hard drives that integrate a drive controller in the drive itself 1 or 2 drives on a chain Enhanced IDE (EIDE) and ATA-2 Faster version of ATA/IDE that supports Direct Memory Access (DMA) transfers Ultra ATA: Speed enhancements to ATA standard Versions running at 33, 66, and 100 Mbytes/sec Serial ATA: Emerging standard using a serial (not parallel) interface Speeds starting at 150 Mbyte/sec Can drive longer cables at much higher clock speeds than parallel cable Serial ATA Rounded parallel ATA Parallel ATA

33. © 2006 Matt Welsh – Harvard University 33 SCSI Interface Standard hardware interface to wide range of I/O devices Disks, CDs, DVDs, tapes, etc. Bus-based design: single shared set of I/O lines that all devices connect to Access model using logical blocks on disk On-disk controller maps logical block # to sector/track/head combination SCSI-1: 8-bit bus, 5 Mhz = 5 Mbytes/sec max speed Supported up to 8 devices on a single bus Lots of problems with termination: required physical connector on end of cable to avoid signal refraction! SCSI-2: The next generation Fast SCSI: 10 Mhz clock speed Wide SCSI: 16 bit bus width Fast wide SCSI: 10 Mhz + 16 bit bus = 20 MB/sec throughput SCSI-3: Ramping up on speed and bus width Highest speed now is “Ultra320 SCSI”: 160 Mhz x 16 bits = 320 MB/sec max speed

34. © 2006 Matt Welsh – Harvard University 34 Relative Interconnect Speeds (from macspeedzone.com)