Coding and Algorithms for Memories Lecture 14 1

Large Scale Storage Systems 2 Big Data Players: Facebook, Amazon, Google, Yahoo,… Cluster of machines running Hadoop at Yahoo! (Source: Yahoo!) Failures are the norm

Problem Setup Disks are stored together in a group (rack) Disk failures should be supported Requirements: – Support as many disk failures as possible – And yet… Optimal and fast recovery Low complexity 3

Reed Solomon Codes Advantages: – Support the maximum number of disk failures – Are very comment in practice and have relatively efficient encoding/decoding schemes Disadvantages – Require to work over large fields Solution: EvenOdd Codes – Need to read all the disks in order to recover even a single disk failure – not efficient rebuild Solution: ZigZag Codes – Solution: Locally Recoverable Codes (LRC) 4

Locally Recoverable Codes (LRC) k k k+1 n n 1 2 r r-1 2 2

Locally Recoverable Codes (LRC) A Locally Recoverable Code (n,k,r) is a code of length n, dimension k, and locality r The problem: Given n,k,r what is the best minimum distance d of the code? A code achieving the maximum d is called an optimal LRC code 6