1 COMP 206: Computer Architecture and Implementation Montek Singh Mon., Nov. 25, 2002 Topic: Storage Systems (Disk Technology) – contd.

2 Ensembles of Disks Key idea Key idea Use collection of disk drives to improve characteristics of disk systems (storage capacity, bandwidth, etc.) Use collection of disk drives to improve characteristics of disk systems (storage capacity, bandwidth, etc.) Used in mainframes for a long time Used in mainframes for a long time RAID RAID Redundant Array of Inexpensive Disks (original 1988 acronym) Redundant Array of Inexpensive Disks (original 1988 acronym) Redundant Array of Independent Disks (redefined in 1992) Redundant Array of Independent Disks (redefined in 1992)

3 Improving Bandwidth with Disk Arrays Arrays of independent disk drives Arrays of independent disk drives Similar to high-order interleaving in main memories Similar to high-order interleaving in main memories Each file assigned to different disk drive Each file assigned to different disk drive Simultaneous access to files Simultaneous access to files Load balancing issues Load balancing issues File striping/disk striping/disk interleaving File striping/disk striping/disk interleaving Single file distributed across array of disks Single file distributed across array of disks Similar to low-order interleaving in main memories Similar to low-order interleaving in main memories Each logical I/O request corresponds to a data stripe Each logical I/O request corresponds to a data stripe Data stripe divided into number of equal sized stripe units Data stripe divided into number of equal sized stripe units Stripe units assigned to different disk units Stripe units assigned to different disk units Two kinds of striping depending on size of stripe unit Two kinds of striping depending on size of stripe unit Fine-grained striping: Stripe unit chosen to balance load Fine-grained striping: Stripe unit chosen to balance load Coarse-grained striping: Larger stripe unit Coarse-grained striping: Larger stripe unit

4 Improving Availability with Disk Arrays Failure rates of Disk Arrays can be quite high: Failure rates of Disk Arrays can be quite high: MTTF of large-system disks approaches 1,000,000 hours MTTF of large-system disks approaches 1,000,000 hours MTTF of PC-class disks approaches 150,000 hours MTTF of PC-class disks approaches 150,000 hours But, array of 1,000 PC-class disks has MTTF of 150 hours But, array of 1,000 PC-class disks has MTTF of 150 hours All schemes to cope with low MTTF aim to fail soft All schemes to cope with low MTTF aim to fail soft Operation should be able to continue while repair is made Operation should be able to continue while repair is made Always depends on some form of redundancy Always depends on some form of redundancy

5RAID-0 Strip 12 Strip 8 Strip 4 Strip 0 Strip 13 Strip 9 Strip 5 Strip 1 Strip 14 Strip 10 Strip 6 Strip 2 Strip 15 Strip 11 Strip 7 Strip 3 Striped, non-redundant Striped, non-redundant Parallel access to multiple disks Parallel access to multiple disks Excellent data transfer rate (for small strips) Excellent data transfer rate (for small strips) Excellent I/O request processing rate (for large strips) Excellent I/O request processing rate (for large strips) Typically used for applications requiring high performance for non-critical data Typically used for applications requiring high performance for non-critical data

6RAID-1 Strip 3 Strip 2 Strip 1 Strip 0 Strip 3 Strip 2 Strip 1 Strip 0 Mirrored/replicated (most costly form of redundancy) Mirrored/replicated (most costly form of redundancy) I/O request rate: good for reads, fair for writes I/O request rate: good for reads, fair for writes Data transfer rate: good for reads; writes slightly slower Data transfer rate: good for reads; writes slightly slower Read can be serviced by the disk with the shorter seek distance Read can be serviced by the disk with the shorter seek distance Write must be handled by both disks Write must be handled by both disks Typically used in system drives and critical files Typically used in system drives and critical files Banking, insurance data Banking, insurance data Web (e-commerce) servers Web (e-commerce) servers

7 Combining RAID-0 and RAID-1 Strip 12 Strip 8 Strip 4 Strip 0 Strip 13 Strip 9 Strip 5 Strip 1 Strip 14 Strip 10 Strip 6 Strip 2 Strip 15 Strip 11 Strip 7 Strip 3 Strip 12 Strip 8 Strip 4 Strip 0 Strip 13 Strip 9 Strip 5 Strip 1 Strip 14 Strip 10 Strip 6 Strip 2 Strip 15 Strip 11 Strip 7 Strip 3 Can combine RAID-0 and RAID-1: Can combine RAID-0 and RAID-1: Mirrored stripes (RAID 0+1, or RAID 01) Mirrored stripes (RAID 0+1, or RAID 01) Example: picture above Example: picture above Striped Mirrors (RAID 1+0, or RAID 10) Striped Mirrors (RAID 1+0, or RAID 10) Data transfer rate: fair for reads and writes Data transfer rate: fair for reads and writes Reliability: good Reliability: good Efficiency: poor (100% overhead in terms of disk utilization) Efficiency: poor (100% overhead in terms of disk utilization)

8RAID-3 b0b0 b1b1 b2b2 b3b3 P(b) Fine-grained (bit) interleaving with parity Fine-grained (bit) interleaving with parity E.g., parity = sum modulo 2 (XOR) of all bits E.g., parity = sum modulo 2 (XOR) of all bits Disks are synchronized, parity computed by disk controller Disks are synchronized, parity computed by disk controller When one disk fails… (how do you know?) When one disk fails… (how do you know?) Data is recovered by subtracting all data in good disks from parity disk Data is recovered by subtracting all data in good disks from parity disk Recovering from failures takes longer than in mirroring, but failures are rare, so is okay Recovering from failures takes longer than in mirroring, but failures are rare, so is okay Hot spares used to reduce vulnerability in reduced mode Hot spares used to reduce vulnerability in reduced mode Performance: Performance: Poor I/O request rate Poor I/O request rate Excellent data transfer rate Excellent data transfer rate Typically used in large I/O request size applications, such as imaging or CAD Typically used in large I/O request size applications, such as imaging or CAD

9RAID-2 b0b0 b1b1 b2b2 b3b3 f 0 (b)f 1 (b)f 2 (b) Hamming codes capable of correcting two or more erasures Hamming codes capable of correcting two or more erasures E.g., single error-correcting, double error-detecting (SEC-DED) E.g., single error-correcting, double error-detecting (SEC-DED) Problem with small writes (similar to DRAM cycle time/access time) Problem with small writes (similar to DRAM cycle time/access time) Poor I/O request rate Poor I/O request rate Excellent data transfer rate Excellent data transfer rate

10RAID-4 Coarse-grained striping with parity Coarse-grained striping with parity Unlike RAID-3, not all disks need to be read on each read Unlike RAID-3, not all disks need to be read on each read New parity computed by computing difference between old and new data New parity computed by computing difference between old and new data Drawback: Drawback: Like RAID-3, parity disk involved in every write; serializes small reads Like RAID-3, parity disk involved in every write; serializes small reads I/O request rate: excellent for reads, fair for writes I/O request rate: excellent for reads, fair for writes Data transfer rate: fair for reads, fair for writes Data transfer rate: fair for reads, fair for writes Blk 12 Blk 8 Blk 4 Blk 0 Blk 13 Blk 9 Blk 5 Blk 1 Blk 14 Blk 10 Blk 6 Blk 2 Blk 15 Blk 11 Blk 7 Blk 3 P(12-15) P(8-11) P(4-7) P(0-3)

11RAID-5 Blk 12 Blk 8 Blk 4 Blk 0 P(12-15) Blk 9 Blk 5 Blk 1 Blk 13 P(8-11) Blk 6 Blk 2 Blk 14 Blk 10 P(4-7) Blk 3 Blk 15 Blk 11 Blk 7 P(0-3) Key Idea: reduce load on parity disk Key Idea: reduce load on parity disk Block-interleaved distributed parity Block-interleaved distributed parity Multiple writes can occur simultaneously Multiple writes can occur simultaneously Block 0 can be accessed in parallel with Block 5 Block 0 can be accessed in parallel with Block 5 First needs disks 1 and 5; second needs disks 2 and 4 First needs disks 1 and 5; second needs disks 2 and 4 I/O request rate: excellent for reads, good for writes I/O request rate: excellent for reads, good for writes Data transfer rate: good for reads, good for writes Data transfer rate: good for reads, good for writes Typically used for high request rate, read-intensive data lookup Typically used for high request rate, read-intensive data lookup

12 Removable Media Magnetic Tapes: Magnetic Tapes: Have been used in computer systems as long as disks Have been used in computer systems as long as disks Similar magnetic technology as disks Similar magnetic technology as disks have followed similar density improvements as disks have followed similar density improvements as disks Long strips wound on removable spools Long strips wound on removable spools Benefits: can be essentially of unlimited length; removable Benefits: can be essentially of unlimited length; removable Drawbacks: only offer sequential access; wear out Drawbacks: only offer sequential access; wear out Used for archival backups; but on the way out Used for archival backups; but on the way out Optical Disks: CDs and DVDs Optical Disks: CDs and DVDs CD-ROM/DVD-ROM: removable, inexpensive, random access CD-ROM/DVD-ROM: removable, inexpensive, random access CD-R/RW, DVD-R/RAM: writable/re-writable CD-R/RW, DVD-R/RAM: writable/re-writable Becoming immensely popular: replacing floppy drives, tape drives Becoming immensely popular: replacing floppy drives, tape drives Flash: flash cards, memory sticks, digital film Flash: flash cards, memory sticks, digital film Microdrives: small portable hard disks (up to 1GB) Microdrives: small portable hard disks (up to 1GB)

13 Interfacing I/O Subsystem to CPU Where do we connect I/O devices? Cache? Cache? Memory? Memory? Low-cost systems: I/O bus = memory bus! Low-cost systems: I/O bus = memory bus! –CPU and I/O compete for bus How does CPU address I/O devs.? Memory-mapped I/O Memory-mapped I/O Portions of address space assigned to I/O devices Portions of address space assigned to I/O devices I/O opcodes (not very popular) I/O opcodes (not very popular) How does CPU synch with I/O? Polling vs. interrupt-driven Polling vs. interrupt-driven Real-time systems: hybrid Real-time systems: hybrid Clock interrupts periodically Clock interrupts periodically CPU polls during interrupt CPU polls during interrupt

14 Does I/O Performance Matter? The No Argument: The No Argument: There is always another process to run while one process waits for I/O to complete There is always another process to run while one process waits for I/O to complete Counter-arguments: Counter-arguments: Above argument applies only if throughput is sole goal Above argument applies only if throughput is sole goal Interactive software and personal computers lay more emphasis on response time (=latency), not throughput Interactive software and personal computers lay more emphasis on response time (=latency), not throughput Interactive multimedia, transaction processing Interactive multimedia, transaction processing Process switching actually increases I/O (paging traffic) Process switching actually increases I/O (paging traffic) Desktop/Mobile computing: only one person/computer fewer processes than in time-sharing Desktop/Mobile computing: only one person/computer fewer processes than in time-sharing Amdahls Law: benefits of making CPU faster taper off if I/O becomes the bottleneck Amdahls Law: benefits of making CPU faster taper off if I/O becomes the bottleneck

15 Does CPU Performance Matter? Consequences of Moores Law: Large, fast CPUs Large, fast CPUs Small, cheap CPUs Small, cheap CPUs Main goal: keeping I/O devices busy, not keeping CPU busy Main goal: keeping I/O devices busy, not keeping CPU busy –bulk of the hardware cost is in I/O peripherals, not CPU Shift in focus: Shift in focus: From computation to communication From computation to communication Reflected in change of terminology: Reflected in change of terminology: –1960s-80s: Computing Revolution –1990s-present: Information Age

16 Does Performance Matter? Performance is not the problem it once was! 15 years of doubling CPU speed every 18 months (x1000) 15 years of doubling CPU speed every 18 months (x1000) Disks also have steadily become bigger and faster Disks also have steadily become bigger and faster Most people would prefer more reliability than speed: Todays speeds come at a cost: frequent crashes Todays speeds come at a cost: frequent crashes Program crashes frustration Program crashes frustration Disk crashes hysteria Disk crashes hysteria Reliability, availability, dependability, etc. are the key terms Reliability, availability, dependability, etc. are the key terms Client-server model of computing has made reliability the key criterion for evaluation Client-server model of computing has made reliability the key criterion for evaluation –E.g., UNC-COMPSCI IMAP server: 300 MHz or something, 99.9… % uptime