Disk Failures Eli Alshan

Agenda Articles survey – Failure Trends in a Large Disk Drive Population – Article review – Conclusions – Criticism – Disk failure s in the real world: What does an MTTF of 1,000,000 hours mean to you? – Article review – Conclusions – Criticism Further research suggestion

Definitions Disk failure - drive is considered to have failed if it was replaced as part of a repairs procedure – 15-60% of drives considered to have failed at the user site are found to have no defect by the manufacturers MTTF - Mean Time To Failure AFR - Annual Fail Rate ARR – Annual Replacement Rate

Failure Trends in a Large Disk Drive Population Analysis of drives self monitoring data, collected from large disk drive Attempt to isolate parameters highly correlated with disk failures

Results - Utilization Very young and very old age groups appear to show the expected behavior Possible Explanation -Infant mortality

Results - Temperature Lower temperatures are associated with higher failure rates

Results – SMART Scan Errors – background surface scan errors Reallocation count - count of sector data reallocations triggered by recurring errors caused by the sector Offline Reallocations - reallocation counts in which only reallocated sectors found during background scrubbing Probational Counts - sectors “on probation” until they either fail permanently and are reallocated or continue to work without problems

Results – SMART

Scan errors affect the survival probability of young drives dramatically but after the first month the curve flattens out Older drives, decline steadily in survival probability throughout the 8-month period This behavior could be another manifestation of infant mortality phenomenon

Results – SMART

Conclusions No consistent pattern of higher failure rates for higher temperature drives or for those drives at higher utilization levels was found Few SMART parameters are well-correlated with higher failure probabilities Out of all failed drives, over 36% have no count in any of the SMART signals, temperature or utilization indication before failure

Criticism Attempt to analyze complex, correlated input data one parameter at a time might be misleading Temperature and utilization should be time windowed, so the reading closer to the failure will receive more attention Physical vicinity between tested drives must be taken into account since close drives experience similar environmental conditions

Disk failure s in the real world: What does an MTTF of 1,000,000 hours mean to you? An analysis of seven data sets, with a focus on storage related failures – Disk replacement rates observed in the field and compare our observations with common predictors and models used by vendors – Statistical properties of disk replacement rates

Disk Replacement Rates The measured average ARR was 3.4 times larger than 0.88% given in the datasheet

Disk Replacement Rates Contrary to common and proposed models, hard drive replacement rates do not enter steady state after the first year of operation, but steadily increase over time

Statistical properties of disk failures The hypothesis that time between disks replacements follows an exponential distribution can be rejected with high confidence The distribution of time between disk replacements exhibits decreasing hazard rates. Disk replacements are fit best with gamma and Weibull distributions.

Statistical properties of disk failures The statistical analysis present strong evidence for the existence of correlations between disk replacement intervals. In particular, the empirical data exhibits significant levels of autocorrelation and long-range dependence.

Conclusions The article demonstrates the lack of reliability of data MTTF and AFR provided by disk vendors. Based on the data analysis the papers authors find a significant correlation between disk failures intervals. The paper was able to substantiate with significant statistical confidence the commonly made assumption that exponentially distributed time between failures is not realistic. The article identifies as the key features that distinguish the empirical distribution of time between disk replacements from the exponential distribution, higher levels of variability and decreasing hazard rates.

Criticism Data set size is relatively small which might invalidate it’s thorough statistical analysis performed The statistical model suggested in the article seem to be too simplistic to describe a complex system as a disk in drive population

Further research suggestion State machine disk health model (HMM) State estimation: – Vector of drive health indicators – Current state of the drives physically close to the drive Parameters estimation: – BIC + EM (Baum-Welch Algorithm) …