1. RAID Systems Ver.2.0 Jan 09, 2005 Syam

2. RAID Primer Redundant Array of Inexpensive Disks random, real-time, redundant, array, assembly, interconnected, independent, inter-relation, devices, drives, etc. Physical Drive Actual Hard Disks Physical Array One or more physical drive Logical Array Formed by splitting or combining physical arrays Logical Drive One or more logical array

3. Drive Layout Physical Drive 0 Physical Array 0 Physical Array 1 Logical Array 2 Logical Drive Logical Array 0 Logical Array 1

4. Why RAID CPU and Memory Disk I/O Disk Reliability MTBF (Mean Time Between Failures) = MTBF Single Drive / # Drives Fault Tolerance Use multiple small, inexpensive disks into array which yields performance exceeding that of Single Large Expensive Disk

5. RAID Benefits Higher Data Security Fault Tolerance Improved Availability Increased, Integrated Capacity Improved Performance

6. RAID Costs Planning and Design Hardware Software Setup and Training Maintenance

7. RAID Tradeoffs

8. RAID Misconceptions Blanket Statements “Invulnerability Complex” AID 0 not RAID 0

9. Hardware vs. Software RAID Hardware RAID Hardware manages the RAID independently from the host and gives the host a single drive per array Controller operates simultaneously with system Highly fault tolerant Software RAID Software manages RAID Lives in host memory and consumes CPU cycles Array only functional when array software is loaded (array down, array software load?)

10. Hardware vs. Software RAID Hardware RAIDSoftware RAID ImplementationDedicated HardwareSoftware Kernel Automatic Failover Yes How SwapYesNo CostHundreds of DollarsNone CPU ImpactNegligibleTypically 5-15%

11. RAID Concepts Mirroring – One method of data redundancy Data written simultaneously to two hard disks 100% redundancy protects against failure of any of the disks My DATA

12. RAID Concepts Striping – Disks used in parallel Each drive partitioned into stripes from one sector (512 bytes) to several MBs Partition size referred to as Striping Unit Pieces of files are stored on multiple disks Files can be broken up into bytes or blocks Drive 0 Drive 1 Drive 2 Striping Unit

13. Parity – Another method of data redundancy Take N pieces of data and calculate another piece and store the N+1 pieces on separate drives If any one of the N+1 pieces of data is lost, it can be recreated from the other N RAID Concepts

14. Parity Example RAID Concepts D1 = 10100101 D2 = 11110000 D3 = 00111100 D4 = 10111001 Parity = D1 XOR D2 XOR D3 XOR D4 = (((10100101 XOR 11110000) XOR 00111100) XOR 10111001) = 11010000 Now five pieces of data are stored on five separate disks Assume D3 becomes corrupt, can be restored by: D3 = D1 XOR D2 XOR D4 XOR Parity = (((10100101 XOR 11110000) XOR 10111001) XOR 11010000) = 00111100

15. RAID operation in degraded state Two-drive mirrored => performance equals that of a single drive Striped array with parity => regenerating lost information Rebuilding Two-drive mirrored => copy entire good drive to replacement drive Striped array with parity => must determine new parity information RAID Degraded Operation and Rebuilding

16. RAID can continue to operate during rebuild Hardware RAID rebuilds faster than software RAID Automatic rebuild Controller detects failed drive Automatically rebuilds on replacement Manual rebuild Administrator initiates rebuild Can be run in off-peak time RAID Degraded Operation and Rebuilding

17. RAID Reliability Component Reliability System Reliability Function of reliability of components

18. RAID Reliability Example RAID with 4 drives with MTBF 500,000 hours Reliability decreased from 500,000 to 88,235 => decreased 82% RAID reliability referred to RAID with fault tolerance

19. Ability of RAID system to withstand loss of some hardware without loss of data or availability When fault occurs, array enters degraded state Drive must be replaced Array must be rebuilt RAID Fault Tolerance

20. Ability for users to access data Depends on: Hardware Reliability Fault Tolerance Hot Swapping Automatic Rebuilding Service RAID Availability

21. Most RAID levels use striping Possible threats to data integrity Unexpected Hard Disk Failure Failures of Support Hardware Physical Damage Software Problems Viruses Human Error RAID Backups

22. 1988 Paper defined RAID levels 1- 5 Now single RAID levels 0 – 7 Multiple RAID levels RAID Levels

23. JBOD – Just a Bunch Of Disks Spanning multiple physical drives into one logical drive No Fault Tolerance Not a RAID

24. JBOD – Just a Bunch Of Disks

25. RAID 0 - Striping Disk Striping No parity Example: Write essay with 3 hands instead of 1 Increased IO Not a valid RAID implementation due to lack of fault tolerance

26. RAID 0 - Striping

27. Supported by all hardware controllers Supported by most software Minimum of two hard disks Array Capacity = Smallest Drive Size * # of Drives Storage Efficiency = 100% of drive Fault Tolerance: None Availability: Lowest of all RAID levels Failure results in array down until rebuild and restore Degradation and Rebuilding: Not Applicable

28. RAID 0 - Striping Random Read Performance: Very Good, increases with larger stripe size Random Write Performance: Very Good, increases with larger stripe size Sequential Read Performance: Very Good to Great Sequential Write Performance: Very Good Cost: Lowest of all RAID levels Special Considerations: Daily Backups Uses: Non-critical data, Hobbyist, high-end gaming

29. RAID 1 - Mirroring 100% Data Redundancy No IO Speed Increase When a drive fails, the other operates until drive replaced

30. RAID 1 - Mirroring

31. Supported by all hardware controllers Supported by most software Exactly two hard disks Array Capacity = Smallest Drive Size Storage Efficiency = 50% of drives Fault Tolerance: Very Good Availability: Very Good Most allow hot spare and automatic rebuilding Degradation and Rebuilding: Slight degradation of read, write improves Rebuilding is relatively fast

32. RAID 1 - Mirroring Random Read Performance: Good Random Write Performance: Good Sequential Read Performance: Fair Sequential Write Performance: Good Cost: Relatively High Special Considerations: Size Limitation Uses: High Fault Tolerance without high capacity, small databases, accounting and financial data

33. RAID 2 – Memory-Style ECC Introduces Parity Same principle as ECC memory Bit-level with Hamming Code Not used today Cost, Complexity

34. RAID 2 - Memory-Style ECC

35. Special Hardware Controller Required Typically 10 Data Disks and 4 ECC Disks Array Capacity = 10 * Data Disk Size Storage Efficiency = 71% of drives Fault Tolerance: Fair Availability: Very Good “On the fly” error correction Degradation and Rebuilding: In theory little degradation

36. RAID 2 - Memory-Style ECC Random Read Performance: Fair Random Write Performance: Poor Sequential Read Performance: Very Good Sequential Write Performance: Fair to Good Cost: Very Expensive Special Considerations: Not in modern systems Uses: Not Used in Modern Systems

37. RAID 3 – Bit-Interleaved Parity Byte Level striping with dedicated parity disk Read requests hit all data disks Write requests hit all data disks and parity disk Great for high bandwidth but not high I/O rates Parity Disk can be bottleneck

38. RAID 3 - Bit-Interleaved Parity

39. Medium to high-end hardware controller required Minimum of three hard disks Array Capacity = Smallest Drive Size*(# Drives–1) Storage Efficiency = (# Drives-1)/# Drives Fault Tolerance: Good Availability: Very Good Hot swapping and automatic rebuild Degradation and Rebuilding: Relative little degradation and rebuilds can take many hours

40. RAID 3 - Bit-Interleaved Parity Random Read Performance: Good Random Write Performance: Poor Sequential Read Performance: Very Good Sequential Write Performance: Fair to Good Cost: Moderate Special Considerations: Not as popular as other Uses: Large Files with High transfer performance, multimedia, publishing

41. RAID 4 – Block-Interleaved Parity Block-level striping with dedicated parity disk Write requests use read-modify-write i.e. four disks (3 data, 1 parity) Small write request 4 disk IO write the new data to disk 0 (1) read old data from disk 1 & disk 2 (2) Write parity information (1) Parity disk can become bottleneck

42. RAID 4 – Block-Interleaved Parity

43. Medium to high-end hardware controller required Minimum of three hard disks Array Capacity = Smallest Drive Size*(# Drives–1) Storage Efficiency = (# Drives-1)/# Drives Fault Tolerance: Good Availability: Very Good Hot swapping and automatic rebuild Degradation and Rebuilding: Moderate if drive fails and potential lengthy rebuild RAID 4 – Block-Interleaved Parity

44. Random Read Performance: Very Good Random Write Performance: Poor to Fair Sequential Read Performance: Good to Very Good Sequential Write Performance: Fair to Good Cost: Moderate Special Considerations: Performance depends on stripe size Uses: Not as common as level 3 or 5 Large Files with High transfer performance RAID 4 – Block-Interleaved Parity

45. RAID 5 – Block-Interleaved Distributed Parity One of most popular levels Eliminates the parity disk bottleneck by distributing parity information across the array More efficient with small read and large write requests

46. RAID 5 – Block-Interleaved Distributed Parity

47. Moderately high-end hardware controller required Supported by some software solutions Minimum of the hard disks Array Capacity = Smallest Drive Size*(# Drives–1) Storage Efficiency = (# Drives-1)/# Drives Fault Tolerance: Good Availability: Good to Very Good Hot swapping and automatic rebuild Degradation and Rebuilding: Can be substantial due to distributed parity

48. RAID 5 – Block-Interleaved Distributed Parity Random Read Performance: Very Good to Great Random Write Performance: Fair Sequential Read Performance: Good to Very Good Sequential Write Performance: Fair to Good Cost: Moderate Special Considerations: Software RAID can greatly affect performance due to parity calculations Uses: Seen as Middle of RAID tradeoff triangle

49. RAID 6 – P+Q Redundancy Block-level striping with dual distributed parity Adds 2D parity information Can handle up to multiple disk failures High data fault tolerance for mission critical applications

50. RAID 6 – P+Q Redundancy

51. Special Hardware Controller Required Typically Minimum of 4 Hard Disks Array Capacity = Smallest Drive Size * # Drives-2 Storage Efficiency = (#Drives - 2)/#Drives Fault Tolerance: Very Good to Great Availability: Great Degradation and Rebuilding: Can be substantial due to dual distributed parity

52. RAID 6 – P+Q Redundancy Random Read Performance: Very Good to Great Random Write Performance: Poor Sequential Read Performance: Good to Very Good Sequential Write Performance: Fair Cost: High Special Considerations: Tends to be used in proprietary systems Uses: Where RAID 5 plus more fault tolerance

53. Multiple RAID Levels Combine two single levels to obtain improved performance Most common level 01 and 10 RAID level X+Y ≠ Y+X Usually not much impact on capacity More impact on fault tolerance

54. Multiple RAID Levels RAID 01 vs 10 RAID 01 Strip Drives 1,2 RAID 0 for Stripe A Stripe Drives 3,4 RAID 0 for Stripe B Mirror two sets, if Drive 2 fails Stripe A is lost RAID 10 Mirror Drives 1,2 RAID 1 for Mirror A Mirror Drives 3,4 RAID 1 for Mirror B Stripe across A and B, if Drive 2 fails still have Drive 1 maintaining stripe

55. RAID 10 – Mirrored Stripe Mirroring and striping without parity Most Common of Multiple levels Large arrays with high performance and high fault tolerance

56. RAID 10 - Mirrored Stripe

57. Most Hardware Controllers Support Even Number with Minimum of 4 Hard Disks Array Capacity = Smallest Drive Size * # Drives/2 Storage Efficiency = 50% Fault Tolerance: Very Good to Great Availability: Great Degradation and Rebuilding: Relatively Little

58. RAID 10 - Mirrored Stripe Random Read Performance: Very Good to Great Random Write Performance: Good to Very Good Sequential Read Performance: Very Good to Great Sequential Write Performance: Good to Very Good Cost: High Special Considerations: Low storage efficiency Uses: High performance and reliability, enterprise servers

59. RAID Comparison