1. 1 COMP 206: Computer Architecture and Implementation Montek Singh Mon, Nov 21, 2005 Mon, Nov 28, 2005 Topic: Storage Systems (Disk Technology)

2. 2 Disk Systems: Characteristics  Capacity [bytes] Mainframes typically have  3.7 GB of disk storage per MIPS Mainframes typically have  3.7 GB of disk storage per MIPS PCs rend to require  5 MB of disk storage per MIPS PCs rend to require  5 MB of disk storage per MIPS  Bandwidth (throughput) Bytes transferred per unit time Bytes transferred per unit time  Service rate Number of service requests satisfied per unit time Number of service requests satisfied per unit time Supercomputer I/O requests tend to involve large amounts of data; transaction processing systems tend to involve small ones Supercomputer I/O requests tend to involve large amounts of data; transaction processing systems tend to involve small ones  Response time (latency) Time between start and completion of an event Time between start and completion of an event  Cost [$/MB]

3. 3 Example: Parameters of A Single Disk Drive

4. 4 Technical Details (1)  Platters 2-4mm thick, made of an aluminum alloy 2-4mm thick, made of an aluminum alloy Typically 1-6 platters in a hard disk Typically 1-6 platters in a hard disk Both sides of each platter generally used Both sides of each platter generally used One side of one platter dedicated to information to guide the servomechanisms that control speed of rotation and head movement One side of one platter dedicated to information to guide the servomechanisms that control speed of rotation and head movement Each recording surface has its own read-write head Each recording surface has its own read-write head  Only one operational at any time  Data transfer to and from disk is bit-serial  Rotation Speed of rotation carefully controlled by servomechanism Speed of rotation carefully controlled by servomechanism  E.g., 7200 rpm  1% or even  1 rpm All disk drives are synchronized in some disk arrays All disk drives are synchronized in some disk arrays  Angular positions are identical (within tolerance) at any time

5. 5 Technical Details (2)  Flying height Read/write heads do not touch rotating disk surface Read/write heads do not touch rotating disk surface Careful aerodynamic design keeps small constant distance (  0.2  ) Careful aerodynamic design keeps small constant distance (  0.2  ) Disk moves with approximate linear speed of 700 inches/sec Disk moves with approximate linear speed of 700 inches/sec  18m/s, 40 miles/hour Smaller flying height  higher writing density Smaller flying height  higher writing density  Hence, platters and heads sealed hermetically in clean space  Called Winchester drives for historical reasons  “Parking the heads” Before disk is allowed to slow down and stop, heads are moved over an area of the disk not used for recording, where they are allowed to come into contact with the disk surface Before disk is allowed to slow down and stop, heads are moved over an area of the disk not used for recording, where they are allowed to come into contact with the disk surface

6. 6 Why Disks Are Bit-Serial  High TPI value makes parallel access disks unworkable Reading heads move as rigid unit Reading heads move as rigid unit One of them (the one over the servo disk) supposedly defines radial (track) position One of them (the one over the servo disk) supposedly defines radial (track) position  Ensemble is not truly rigid, and various reasons (like thermal dilations) prevent all heads being positioned over same track simultaneously, repeatedly, and reliably  Only one head is positioned accurately at a time: servo guides the assembly to approximate position of requested track, reading head does final positioning using a high frequency signal between data tracks  Sector ID always contains track number for confirmation Portable disks use shock sensors to prevent overwriting of adjacent tracks caused by jarring of R/W head Portable disks use shock sensors to prevent overwriting of adjacent tracks caused by jarring of R/W head

7. 7 Track Densities  Until recently, only innermost track was recorded with maximum density All other tracks contained same number of sectors and bytes All other tracks contained same number of sectors and bytes  Recently, manufacturers are using bands of tracks Total number of tracks (100s-1000s) divided into bands or zones (4, 8, 16, …) each containing the same number of tracks Total number of tracks (100s-1000s) divided into bands or zones (4, 8, 16, …) each containing the same number of tracks Each band has innermost track recorded at maximum density, with other tracks having same capacity Each band has innermost track recorded at maximum density, with other tracks having same capacity Greatest capacity gains occur with a small number of bands Greatest capacity gains occur with a small number of bands  Two scheduling options Constant rotational speed, use buffer for speed matching Constant rotational speed, use buffer for speed matching Constant data rate, head spinning with rotational speed corresponding to recording density of zone Constant data rate, head spinning with rotational speed corresponding to recording density of zone

8. 8 Sector Format  Sector is the smallest unit of data that can be read or written Typically between 32B and 4KB, with 512B being a common size Typically between 32B and 4KB, with 512B being a common size  Format of sector ID: Identifies sector with information such as angular position and track # ID: Identifies sector with information such as angular position and track #  Has its own error correcting code (ECC) GAP: Permits electronics to process ECC information GAP: Permits electronics to process ECC information DATA and its ECC DATA and its ECC IDECC GAP DATAECC GAP Total sector size is measure of formatted capacity of disk as fraction of nominal capacity

9. 9 Disk Miscellanea  Smaller disks tend to be more cost-effective Smaller inertia, lower power consumption per megabyte, shorter seek distances, less vibration, less heat generated Smaller inertia, lower power consumption per megabyte, shorter seek distances, less vibration, less heat generated Heat generation proportional to N  RPM 2.8  D 4.6 Heat generation proportional to N  RPM 2.8  D 4.6  N = number of platters  D = diameter  RPM = rotational speed  Disk access time = seek time + rotational latency + transfer time Typical values: 12-30 ms seek time, 8.3 ms rotational latency Typical values: 12-30 ms seek time, 8.3 ms rotational latency  Disk growth rates Disk areal density doubling every three years Disk areal density doubling every three years Disk transfer rate doubling every five years Disk transfer rate doubling every five years Disk access time halving every ten years Disk access time halving every ten years

10. 10 Ensembles of Disks  Key idea Use collection of disk drives to improve characteristics of disk systems (storage capacity, bandwidth, etc.) Use collection of disk drives to improve characteristics of disk systems (storage capacity, bandwidth, etc.)  Used in mainframes for a long time  RAID Redundant Array of Inexpensive Disks (original 1988 acronym) Redundant Array of Inexpensive Disks (original 1988 acronym) Redundant Array of Independent Disks (redefined in 1992) Redundant Array of Independent Disks (redefined in 1992)

11. 11 Improving Bandwidth with Disk Arrays  Arrays of independent disk drives Similar to high-order interleaving in main memories Similar to high-order interleaving in main memories Each file assigned to different disk drive Each file assigned to different disk drive Simultaneous access to files Simultaneous access to files Load balancing issues Load balancing issues  File striping/disk striping/disk interleaving Single file distributed across array of disks Single file distributed across array of disks Similar to low-order interleaving in main memories Similar to low-order interleaving in main memories Each logical I/O request corresponds to a data stripe Each logical I/O request corresponds to a data stripe  Data stripe divided into number of equal sized stripe units  Stripe units assigned to different disk units Two kinds of striping depending on size of stripe unit Two kinds of striping depending on size of stripe unit  Fine-grained striping: Stripe unit chosen to balance load  Coarse-grained striping: Larger stripe unit

12. 12 Improving Availability with Disk Arrays  MTTF of large-system disks approaches 1,000,000 hours  MTTF of PC-class disks approaches 150,000 hours  However, array of 1,000 PC-class disks has MTTF of 150 hours  All schemes to cope with low MTTF aim to “fail soft” Operation should be able to continue while repair is made Operation should be able to continue while repair is made Always depends on some form of redundancy Always depends on some form of redundancy

13. 13RAID-0 Strip 12 Strip 8 Strip 4 Strip 0 Strip 13 Strip 9 Strip 5 Strip 1 Strip 14 Strip 10 Strip 6 Strip 2 Strip 15 Strip 11 Strip 7 Strip 3  Striped, non-redundant Parallel access to multiple disks Parallel access to multiple disks  Excellent data transfer rate (for small strips)  Excellent I/O request processing rate (for large strips)  Typically used for applications requiring high performance for non-critical data

14. 14RAID-1 Strip 3 Strip 2 Strip 1 Strip 0 Strip 3 Strip 2 Strip 1 Strip 0  Mirrored/replicated (most costly form of redundancy)  I/O request rate: good for reads, fair for writes  Data transfer rate: good for reads; writes slightly slower Read can be serviced by the disk with the shorter seek distance Read can be serviced by the disk with the shorter seek distance Write must be handled by both disks Write must be handled by both disks  Typically used in system drives and critical files Banking, insurance data Banking, insurance data Web (e-commerce) servers Web (e-commerce) servers

15. 15 Combining RAID-0 and RAID-1 Strip 12 Strip 8 Strip 4 Strip 0 Strip 13 Strip 9 Strip 5 Strip 1 Strip 14 Strip 10 Strip 6 Strip 2 Strip 15 Strip 11 Strip 7 Strip 3 Strip 12 Strip 8 Strip 4 Strip 0 Strip 13 Strip 9 Strip 5 Strip 1 Strip 14 Strip 10 Strip 6 Strip 2 Strip 15 Strip 11 Strip 7 Strip 3  Can combine RAID-0 and RAID-1: Mirrored stripes (RAID 0+1, or RAID 01) Mirrored stripes (RAID 0+1, or RAID 01)  Example: picture above Striped Mirrors (RAID 1+0, or RAID 10) Striped Mirrors (RAID 1+0, or RAID 10)  Data transfer rate: good for reads and writes  Reliability: good  Efficiency: poor (100% overhead in terms of disk utilization)

16. 16RAID-3 b0b0 b1b1 b2b2 b3b3 P(b)  Fine-grained (bit) interleaving with parity E.g., parity = sum modulo 2 (XOR) of all bits E.g., parity = sum modulo 2 (XOR) of all bits  Disks are synchronized, parity computed by disk controller  When one disk fails… (how do you know?) Data is recovered by subtracting all data in good disks from parity disk Data is recovered by subtracting all data in good disks from parity disk Recovering from failures takes longer than in mirroring, but failures are rare, so is okay Recovering from failures takes longer than in mirroring, but failures are rare, so is okay Hot spares used to reduce vulnerability in reduced mode Hot spares used to reduce vulnerability in reduced mode  Performance: Poor I/O request rate Poor I/O request rate Excellent data transfer rate Excellent data transfer rate Typically used in large I/O request size applications, such as imaging or CAD Typically used in large I/O request size applications, such as imaging or CAD

17. 17RAID-2 b0b0 b1b1 b2b2 b3b3 f 0 (b)f 1 (b)f 2 (b)  Hamming codes capable of correcting two or more erasures E.g., single error-correcting, double error-detecting (SEC-DED) E.g., single error-correcting, double error-detecting (SEC-DED)  Problem with small writes (similar to DRAM cycle time/access time)  Poor I/O request rate  Excellent data transfer rate

18. 18RAID-4  Coarse-grained striping with parity  Unlike RAID-3, not all disks need to be read on each write New parity computed by computing difference between old and new data New parity computed by computing difference between old and new data  Drawback: Like RAID-3, parity disk involved in every write; serializes small reads Like RAID-3, parity disk involved in every write; serializes small reads  I/O request rate: excellent for reads, fair for writes  Data transfer rate: good for reads, fair for writes Blk 12 Blk 8 Blk 4 Blk 0 Blk 13 Blk 9 Blk 5 Blk 1 Blk 14 Blk 10 Blk 6 Blk 2 Blk 15 Blk 11 Blk 7 Blk 3 P(12-15) P(8-11) P(4-7) P(0-3)

19. 19RAID-5 Blk 12 Blk 8 Blk 4 Blk 0 P(12-15) Blk 9 Blk 5 Blk 1 Blk 13 P(8-11) Blk 6 Blk 2 Blk 14 Blk 10 P(4-7) Blk 3 Blk 15 Blk 11 Blk 7 P(0-3)  Key Idea: reduce load on parity disk Block-interleaved distributed parity Block-interleaved distributed parity Multiple writes can occur simultaneously Multiple writes can occur simultaneously Block 0 can be accessed in parallel with Block 5 Block 0 can be accessed in parallel with Block 5  First needs disks 1 and 5; second needs disks 2 and 4  I/O request rate: excellent for reads, good for writes  Data transfer rate: good for reads, good for writes  Typically used for high request rate, read-intensive data lookup

20. 20 Removable Media  Magnetic Tapes: Have been used in computer systems as long as disks Have been used in computer systems as long as disks Similar magnetic technology as disks Similar magnetic technology as disks  have followed similar density improvements as disks Long strips wound on removable spools Long strips wound on removable spools Benefits: can be essentially of “unlimited” length; removable Benefits: can be essentially of “unlimited” length; removable Drawbacks: only offer sequential access; wear out Drawbacks: only offer sequential access; wear out Used for archival backups; but on the way out Used for archival backups; but on the way out  Optical Disks: CD’s and DVD’s CD-ROM/DVD-ROM: removable, inexpensive, random access CD-ROM/DVD-ROM: removable, inexpensive, random access CD-R/RW, DVD-R/RAM: writable/re-writable CD-R/RW, DVD-R/RAM: writable/re-writable Becoming immensely popular: replacing floppy drives, tape drives Becoming immensely popular: replacing floppy drives, tape drives  Flash: flash cards, memory sticks, digital film  Microdrives: small portable hard disks (up to 1GB)

21. 21 Interfacing I/O Subsystem to CPU Where do we connect I/O devices? Cache? Cache? Memory? Memory?  Low-cost systems: I/O bus = memory bus! –CPU and I/O compete for bus How does CPU address I/O devs.? Memory-mapped I/O Memory-mapped I/O  Portions of address space assigned to I/O devices I/O opcodes (not very popular) I/O opcodes (not very popular) How does CPU synch with I/O? Polling vs. interrupt-driven Polling vs. interrupt-driven Real-time systems: hybrid Real-time systems: hybrid  Clock interrupts periodically  CPU polls during interrupt

22. 22 Bus: Different Options  Bus width High-perf: separate address and data lines High-perf: separate address and data lines Low-cost: multiplex address and data lines Low-cost: multiplex address and data lines  Data width High-perf: wider is faster (e.g., 64 bits) High-perf: wider is faster (e.g., 64 bits) Low-cost: narrower is cheaper (e.g., 8 bits) Low-cost: narrower is cheaper (e.g., 8 bits)  Transfer size High-perf: multiple words have less bus overhead High-perf: multiple words have less bus overhead Low-cost: single-word transfer is simpler Low-cost: single-word transfer is simpler  Bus masters High-perf: multiple (requires arbitration) High-perf: multiple (requires arbitration) Low-cost: single master (simpler) Low-cost: single master (simpler)  Split transaction High-perf: Yes (request and reply in separate “packets”) High-perf: Yes (request and reply in separate “packets”) Low-cost: No (continuous connection is cheaper) Low-cost: No (continuous connection is cheaper)  Clocking High-perf: synchronous (though it is changing) High-perf: synchronous (though it is changing) Low-cost: asynchronous Low-cost: asynchronous

23. 23 Examples of Bus Standards IDE/Ultra ATA SCSIPCIPCI-X Data width 16 bits 8 or 16 bits 32 or 64 bits Clock rate Up to 100 MHz 10-160 MHz 33-66 MHz 66-133 MHz # Bus masters 1MultipleMultipleMultiple Peak Bandwidth 200 MB/sec 320 MB/sec 533 MB/sec 1066 MB/sec ClockingAsyncAsyncSyncSync

24. 24 Does I/O Performance Matter?  The “No” Argument: “There is always another process to run while one process waits for I/O to complete” “There is always another process to run while one process waits for I/O to complete”  Counter-arguments: Above argument applies only if throughput is sole goal Above argument applies only if throughput is sole goal Interactive software and personal computers lay more emphasis on response time (=latency), not throughput Interactive software and personal computers lay more emphasis on response time (=latency), not throughput  Interactive multimedia, transaction processing Process switching actually increases I/O (paging traffic) Process switching actually increases I/O (paging traffic) Desktop/Mobile computing: only one person/computer  fewer processes than in time-sharing Desktop/Mobile computing: only one person/computer  fewer processes than in time-sharing Amdahl’s Law: benefits of making CPU faster taper off if I/O becomes the bottleneck Amdahl’s Law: benefits of making CPU faster taper off if I/O becomes the bottleneck

25. 25 Does CPU Performance Matter? Consequences of Moore’s Law: Large, fast CPU’s Large, fast CPU’s Small, cheap CPU’s Small, cheap CPU’s  Main goal: keeping I/O devices busy, not keeping CPU busy –bulk of the hardware cost is in I/O peripherals, not CPU Shift in focus: Shift in focus:  From computation to communication  Reflected in change of terminology: –1960’s-80’s: Computing Revolution –1990’s-present: Information Age

26. 26 Does Performance Matter? Performance is not the problem it once was! 15 years of doubling CPU speed every 18 months (x1000) 15 years of doubling CPU speed every 18 months (x1000) Disks also have steadily become bigger and faster Disks also have steadily become bigger and faster Most people would prefer more reliability than speed: Today’s speeds come at a cost: frequent crashes Today’s speeds come at a cost: frequent crashes Program crashes  frustration Program crashes  frustration Disk crashes  hysteria Disk crashes  hysteria  Reliability, availability, dependability, etc. are the key terms  Client-server model of computing has made reliability the key criterion for evaluation –E.g., UNC-COMPSCI IMAP server: 300 MHz or something, 99.9… % uptime