Presentation is loading. Please wait.

Presentation is loading. Please wait.

22/07/2011 1 The MDS Scaling Problem for Cloud Storage Yu-chong Hu Institute of Network Coding.

Similar presentations


Presentation on theme: "22/07/2011 1 The MDS Scaling Problem for Cloud Storage Yu-chong Hu Institute of Network Coding."— Presentation transcript:

1 22/07/2011 1 The MDS Scaling Problem for Cloud Storage Yu-chong Hu Institute of Network Coding

2 Background 2 A webmaster wants to upload his website for Michael Jackson to Internet

3 Background Dedicated Storage: purchase storage devices (servers) for his demand. A webmaster wants to upload his website for Michael Jackson to Internet

4 Background 4 Dedicated Storage: purchase storage devices (servers) for his demand. Cloud Storage: purchase storage services for his demand. A webmaster wants to upload his website for Michael Jackson to Internet

5 Background  The demand is dynamic Normal case During the 3 months after MJ died (BURST!) $50,000 $100,000 Return to normal 3 months later $100,000

6 Background  The demand is dynamic Normal case During the 3 months after MJ died (BURST!) $50,000 $100,000 Return to normal 3 months later $100,000 waste

7 Background  The demand is dynamic 7 Normal case During the 3 months after MJ died (BURST!) $50,000 Standard : $1 per hour $100,000 High Performance : $2 per hour Return to normal 3 months later $100,000 Standard : $1 per hour waste

8 Background  The demand is dynamic 8 Normal case During the 3 months after MJ died (BURST!) $50,000 Standard : $1 per hour $100,000 High Performance : $2 per hour Return to normal 3 months later $100,000 Standard : $1 per hour waste Always match the demand: No waste

9 Motivation 9 Network Applications Network Applications

10 Motivation 10 Network Applications Network Applications Dynamic Demand Dynamic Demand

11 Motivation 11 Network Applications Network Applications Dynamic Demand Dynamic Demand Cloud Storage Cloud Storage

12 Motivation 12 Network Applications Network Applications Dynamic Demand Dynamic Demand Cloud Storage Cloud Storage Scale the capacity Up Scale the capacity Up Scale the capacity down Scale the capacity down

13 Motivation 13 Network Applications Network Applications Dynamic Demand Dynamic Demand Cloud Storage Cloud Storage How to do the scaling as fast as possible? How to do the scaling as fast as possible? Scale the capacity Up Scale the capacity Up Scale the capacity down Scale the capacity down

14 Motivation 14 Network Applications Network Applications Dynamic Demand Dynamic Demand Cloud Storage Cloud Storage minimizing data migration scaling traffic minimizing data migration scaling traffic How to do the scaling as fast as possible? How to do the scaling as fast as possible? Scale the capacity Up Scale the capacity Up Scale the capacity down Scale the capacity down

15 Motivation  Replication-based data can be scaled very easily

16 Motivation  Replication-based data can be scaled very easily  Erasure coding-based data are more reliable. E.g:

17 Motivation  Replication-based data can be scaled very easily  Erasure coding-based data are more reliable. E.g: vs Replication A A B B (4,2) MDS erasure code A B A+B A+2B Node 1 Node 2 Node 3 Node 4 Node 1 Node 2 Node 3 Node 4 Source A B File Source Node

18 Motivation  Replication-based data can be scaled very easily  Erasure coding-based data are more reliable. E.g: vs Replication A A B B (4,2) MDS erasure code A B A+B A+2B Tolerate one failure Node 1 Node 2 Node 3 Node 4 Node 1 Node 2 Node 3 Node 4 Source A B File Source Node

19 Motivation  Replication-based data can be scaled very easily  Erasure coding-based data are more reliable. E.g: vs Replication A A B B (4,2) MDS erasure code A B A+B A+2B Tolerate one failure Tolerate two failures Node 1 Node 2 Node 3 Node 4 Node 1 Node 2 Node 3 Node 4 Source A B File Source Node

20 MDS Scaling Problem  Problem Statement: How to minimize the scaling traffic in storage system based on MDS codes.

21 MDS Scaling Problem  Problem Statement: How to minimize the scaling traffic in storage system based on MDS codes.  Network Coding + MDS Scaling Problem 21

22 MDS Scaling Problem  Problem Statement: How to minimize the scaling traffic in storage system based on MDS codes.  Network Coding + MDS Scaling Problem Source Node 1 n...... MDS (n,k)

23 MDS Scaling Problem  Problem Statement: How to minimize the scaling traffic in storage system based on MDS codes.  Network Coding + MDS Scaling Problem Source Node 1 n...... 1 n’...... Scaling MDS (n,k)MDS (n’,k’)

24 MDS Scaling Problem  Problem Statement: How to minimize the scaling traffic in storage system based on MDS codes.  Network Coding + MDS Scaling Problem 24 Source Node 1 n...... 1 n’...... Data collector any k’ of n’ nodes can rebuild file Data collector Data collector...... MDS (n,k)MDS (n’,k’) Scaling

25 MDS Scaling Problem  Problem Statement: How to minimize the scaling traffic in storage system based on MDS codes.  Network Coding + MDS Scaling Problem 25 Source Node 1 n...... 1 n’...... Scale traffic Data collector any k’ of n’ nodes can rebuild file Data collector Data collector...... MDS scaling problem = multicasting on information flow graph. MDS scaling problem = multicasting on information flow graph. MDS (n,k)MDS (n’,k’)

26 MDS Scaling Problem  Related work 1.Dress Codes[1]: The scaling problem is considered, but dress codes are especially designed for optimal repairing at first, and the value of k is fixed; MDS scaling problem needs to consider optimal scaling at first, and k can be scaled to k’. 26 1.Dress Codes for the Storage Cloud: Simple Randomized Constructions, Sameer Pawar et al. 2.FastScale: Accelerate RAID Scaling by Minimizing Data Migration, W Zheng et al., FAST 2011’ 3.ALV: A new data redistribution approach to RAID-5 scaling, G Zhang et al. Transactions on computers

27 MDS Scaling Problem  Related work 1.Dress Codes[1]: The scaling problem is considered, but dress codes are especially designed for optimal repairing at first, and the value of k is fixed; MDS scaling problem needs to consider optimal scaling at first, and k can be scaled to k’. 2.RAID Scaling [2, 3]: The authors use a technique “sliding window” to reduce the moved packets for scaling of RAID codes, but they do not give a theoretical bound for scaling traffic. 27 1.Dress Codes for the Storage Cloud: Simple Randomized Constructions, Sameer Pawar et al. 2.FastScale: Accelerate RAID Scaling by Minimizing Data Migration, W Zheng et al., FAST 2011’ 3.ALV: A new data redistribution approach to RAID-5 scaling, G Zhang et al. Transactions on computers

28 MDS Scaling Problem  There are very few theoretical results about the scaling problem.  No bounds obtained  No optimal code and scaling schemes presented  Outline of my talk:  (n,k)MDS → (n+m,k) MDS  Example 1: (3,2) MDS → (4,2) MDS  Example 2: (3,2) MDS → (4,3) MDS  (n,k)MDS → (n+m,k+m) MDS  Example 3: (4,3) MDS → (5,4) MDS 28

29 Example 1: (3,2)MDS → (4,2)MDS 29 (3,2)MDS (4,2)MDS Data collector any 2 of 3 nodes can rebuild file The scaling (n,k)MDS → (n+m,k) MDS increases more data reliability because more failures can be tolerated. Data collector any 2 of 4 nodes can rebuild file Tolerate one failureTolerate two failures

30 Example 1: (3,2)MDS → (4,2)MDS 30 Data collector Source M: any 2 of 3 nodes can rebuild file (3,2)MDS Source Node A B C D A⊕CA⊕C B⊕DB⊕D A B C D

31 Example 1: (3,2)MDS → (4,2)MDS 31 Data collector Source M: any 2 of 3 nodes can rebuild file Source Node A B C D A⊕CA⊕C B⊕DB⊕D A B C D A (3,2)MDS

32 Example 1: (3,2)MDS → (4,2)MDS 32 Data collector Source M: any 2 of 3 nodes can rebuild file Source Node A B C D A⊕CA⊕C B⊕DB⊕D A B C D A ⊕ (3,2)MDS

33 Example 1: (3,2)MDS → (4,2)MDS 33 Data collector Source M: any 2 of 3 nodes can rebuild file Source Node A B C D A⊕CA⊕C B⊕DB⊕D A B C D A C⊕DC⊕D ⊕ (3,2)MDS

34 Example 1: (3,2)MDS → (4,2)MDS 34 Data collector Source M: any 2 of 3 nodes can rebuild file Source Node A B C D A⊕CA⊕C B⊕DB⊕D A B C D A C⊕DC⊕D B⊕DB⊕D ⊕ (3,2)MDS

35 Example 1: (3,2)MDS → (4,2)MDS 35 Data collector Source M: any 2 of 3 nodes can rebuild file Source Node A B C D A⊕CA⊕C B⊕DB⊕D A B C D A C⊕DC⊕D B⊕DB⊕D ⊕ (3,2)MDS

36 Example 1: (3,2)MDS → (4,2)MDS 36 Data collector Source M: any 2 of 3 nodes can rebuild file Source Node A B C D A⊕CA⊕C B⊕DB⊕D A B C D A C⊕DC⊕D B⊕DB⊕D B⊕CB⊕C ⊕ (3,2)MDS

37 Example 1: (3,2)MDS → (4,2)MDS 37 Data collector Source M: any 2 of 3 nodes can rebuild file Source Node A B C D A⊕CA⊕C B⊕DB⊕D A B C D A C⊕DC⊕D B⊕DB⊕D B⊕CB⊕C ⊕ (3,2)MDS

38 Example 1: (3,2)MDS → (4,2)MDS 38 Source M: Source Node A B C D A⊕CA⊕C B⊕DB⊕D A B C D B⊕CB⊕C A⊕B⊕DA⊕B⊕D A C⊕DC⊕D B⊕DB⊕D ⊕ (4,2)MDS

39 Example 1: (3,2)MDS → (4,2)MDS 39 Data collector Source M: any 2 of 4 nodes can rebuild file Source Node A B C D A⊕CA⊕C B⊕DB⊕D A B C D B⊕CB⊕C A⊕B⊕DA⊕B⊕D A C⊕DC⊕D B⊕DB⊕D ⊕ (4,2)MDS

40 Flow graph: 1-node Scaling = 1-loss Repairing 40 Data collector Source M: any 2 of 4 nodes can rebuild file Source Node A B C D A⊕CA⊕C B⊕DB⊕D A B C D 1 loss repair of (4,2) RC code B⊕CB⊕C A⊕B⊕DA⊕B⊕D B⊕CB⊕C A⊕B⊕DA⊕B⊕D A C⊕DC⊕D B⊕DB⊕D ⊕

41 Flow graph: 1-node Scaling = 1-loss Repairing 41 (3,2)MDS → (4,2)MDS 1-loss repair of (4,2) RC 2 2

42 Results  Conclusion 1: The optimal scaling problem from (n,k)MDS to (n+1,k)MDS is equivalent to the 1-loss repair problem based on (n+1,k)RC. 42

43 Results  Conclusion 1: The optimal scaling problem from (n,k)MDS to (n+1,k)MDS is equivalent to the 1-loss repair problem based on (n+1,k)RC.  Conclusion 2: The opitmal scaling problem from (n,k)MDS to (n+r,k)MDS is equivalent to the r-loss repair problem based on (n+r,k)RC for multiple failures. 43

44 Open Problems How to design the optimal scaling algorithms in which (n,k)MDS can grow to different MDS? 44

45 Open Problems How to design the optimal scaling algorithms in which (n,k)MDS can grow to different MDS? Motivation: In cloud storage, a consumer, at first, may select a default level of storage service with availability of 99.99%. Later, the consumer wants to increase the availability, so the cloud storage service should provide different higher level of storage redundancy for user’s selection. 45

46 Open Problems How to design the optimal scaling algorithms in which (n,k)MDS can grow to different MDS? Motivation: In cloud storage, a consumer, at first, may select a default level of storage service with availability of 99.99%. Later, the consumer wants to increase the availability, so the cloud storage service should provide different higher level of storage redundancy for user’s selection. Example: (3,2)MDS:99.99% 46 (4,2)MDS:99.9999% (5,2)MDS:99.999999%

47 Open Problems How to design the optimal scaling algorithms in which (n,k)MDS can grow to different MDS? Motivation: In cloud storage, a consumer, at first, may select a default level of storage service with availability of 99.99%. Later, the consumer wants to increase the availability, so the cloud storage service should provide different higher level of storage redundancy for user’s selection. Example: (3,2)MDS:99.99% Difficulty: This optimal scaling problem may not be equivalent to the optimal repair problem, because there exist at least two optional scaling size. 47 (4,2)MDS:99.9999% (5,2)MDS:99.999999%

48 Example 2: (3,2)MDS → (4,3)MDS 48 (3,2)MDS (4,3)MDS Data collector any 2 of 3 nodes can rebuild file (n,k)MDS → (n+m,k+m) MDS scaling increases higher I/O performance because more I/O bandwidth for data collector can be obtained. Data collector any 3 of 4 nodes can rebuild file 10MB/s

49 Example 2: (3,2)MDS → (4,3)MDS 49 Data collector Source M: any 2 of 3 nodes can rebuild file A B Source Node C D A B C E F D E F A⊕DA⊕D B⊕EB⊕E C⊕FC⊕F (3,2)MDS

50 Example 2: (3,2)MDS → (4,3)MDS 50 Source M: A B (4,3)MDS Source Node C D A B C E F D E F A⊕DA⊕D B⊕EB⊕E C⊕FC⊕F

51 Example 2: (3,2)MDS → (4,3)MDS 51 Source M: A B (4,3)MDS Source Node C D A B C E F D E F A⊕DA⊕D B⊕EB⊕E C⊕FC⊕F C

52 Example 2: (3,2)MDS → (4,3)MDS 52 Source M: A B (4,3)MDS Source Node C D A B E F D E F A⊕DA⊕D B⊕EB⊕E C⊕FC⊕F C

53 Example 2: (3,2)MDS → (4,3)MDS 53 Source M: A B (4,3)MDS Source Node C D A B E F D E F A⊕DA⊕D B⊕EB⊕E C⊕FC⊕F C D

54 Example 2: (3,2)MDS → (4,3)MDS 54 Source M: A B (4,3)MDS Source Node C D A B E F E F A⊕DA⊕D B⊕EB⊕E C⊕FC⊕F C D

55 Example 2: (3,2)MDS → (4,3)MDS 55 (4,3)MDS A⊕DA⊕D B⊕EB⊕E C⊕FC⊕F

56 Example 2: (3,2)MDS → (4,3)MDS 56 (4,3)MDS A⊕DA⊕D B⊕EB⊕E C⊕FC⊕F A⊕D⊕B⊕EA⊕D⊕B⊕E B⊕E⊕C⊕FB⊕E⊕C⊕F

57 Example 2: (3,2)MDS → (4,3)MDS 57 Source M: A B (4,3)MDS Source Node C D A B E F E F A⊕D⊕B⊕EA⊕D⊕B⊕E B⊕E⊕C⊕FB⊕E⊕C⊕F (3,2)MDS C D

58 Example 2: (3,2)MDS → (4,3)MDS 58 Data collector Source M: any 3 of 4 nodes can rebuild file A B (4,3)MDS Source Node C D A B E F E F A⊕D⊕B⊕EA⊕D⊕B⊕E B⊕E⊕C⊕FB⊕E⊕C⊕F (3,2)MDS C D

59 Example 2: (3,2)MDS → (4,3)MDS 59 Data collector Source M: any 3 of 4 nodes can rebuild file A B (4,3)MDS Source Node C D A B E F E F A⊕D⊕B⊕EA⊕D⊕B⊕E B⊕E⊕C⊕FB⊕E⊕C⊕F (3,2)MDS C D Scaling traffic = M/3

60 Example 2: (3,2)MDS → (4,3)MDS 60 Data collector Source M: any 3 of 4 nodes can rebuild file A B (4,3)MDS Source Node C D A B E F E F A⊕D⊕B⊕EA⊕D⊕B⊕E B⊕E⊕C⊕FB⊕E⊕C⊕F (3,2)MDS C D Scaling traffic = M/3 The size of new node is M/3, so the scaling traffic is minimal

61 Example 3: (4,3)MDS → (5,4)MDS (n+m,k+m) MDS→(n+m+l,k+m+l)MDS scaling (iterative scaling) increases higher and higher I/O performance. (4,3)MDS (5,4)MDS Data collector any 3 of 4 nodes can rebuild file 10MB/s Data collector any 4 of 5 nodes can rebuild file 10MB/s

62 Example 3: (4,3)MDS → (5,4)MDS In real storage systems, a data object is divided into a block stream for some practical reasons: A1 B1 C1 D1 E1 F1 A2 B2 C2 D2 E2 F2 …… Data Object:

63 Example 3: (4,3)MDS → (5,4)MDS (4,3)MDS C1 D1 A1 B1 E1 F1 A1 ⊕ D1 ⊕ B1 ⊕ E1 B1 ⊕ E1 ⊕ C1 ⊕ F1 A2 ⊕ D2 ⊕ B2 ⊕ E2 B2 ⊕ E2 ⊕ C2 ⊕ F2 A2 B2 E2 F2 C2 D2 Segment 1Segment 2 In real storage systems, a data object is divided into a block stream for some practical reasons: A1 B1 C1 D1 E1 F1 A2 B2 C2 D2 E2 F2 …… Data Object:

64 Example 3: (4,3)MDS → (5,4)MDS C1 D1 A1 B1 E1 F1 A1 ⊕ D1 ⊕ B1 ⊕ E1 B1 ⊕ E1 ⊕ C1 ⊕ F1 A2 ⊕ D2 ⊕ B2 ⊕ E2 B2 ⊕ E2 ⊕ C2 ⊕ F2 A2 B2 E2 F2 C2 D2 Segment 1Segment 2

65 Example 3: (4,3)MDS → (5,4)MDS C1 D1 A1 B1 E1 F1 A1 ⊕ D1 ⊕ B1 ⊕ E1 B1 ⊕ E1 ⊕ C1 ⊕ F1 A2 ⊕ D2 ⊕ B2 ⊕ E2 B2 ⊕ E2 ⊕ C2 ⊕ F2 A2 B2 E2 F2 C2 D2 Segment 1Segment 2 Consider two adjacent segments

66 Example 3: (4,3)MDS → (5,4)MDS 66 (4,3)MDS C1 D1 E1 F1 A1 ⊕ D1 ⊕ B1 ⊕ E1 B1 ⊕ E1 ⊕ C1 ⊕ F1 A2 ⊕ D2 ⊕ B2 ⊕ E2 B2 ⊕ E2 ⊕ C2 ⊕ F2 E2 F2 C2 D2 A1 B1 A2 B2

67 Example 3: (4,3)MDS → (5,4)MDS 67 C1 D1 E1 F1 A1 ⊕ D1 ⊕ B1 ⊕ E1 B1 ⊕ E1 ⊕ C1 ⊕ F1 A2 ⊕ D2 ⊕ B2 ⊕ E2 B2 ⊕ E2 ⊕ C2 ⊕ F2 E2 F2 C2 D2 (5,4)MDS A1 B1 A2 B2

68 Example 3: (4,3)MDS → (5,4)MDS 68 C1 D1 E1 F1 A1 ⊕ D1 ⊕ B1 ⊕ E1 B1 ⊕ E1 ⊕ C1 ⊕ F1 A2 ⊕ D2 ⊕ B2 ⊕ E2 B2 ⊕ E2 ⊕ C2 ⊕ F2 E2 F2 C2 D2 (5,4)MDS A1 B1 A2 B2

69 Example 3: (4,3)MDS → (5,4)MDS 69 C1 D1 E1 F1 A1 ⊕ D1 ⊕ B1 ⊕ E1 B1 ⊕ E1 ⊕ C1 ⊕ F1 A2 ⊕ D2 ⊕ B2 ⊕ E2 B2 ⊕ E2 ⊕ C2 ⊕ F2 E2 F2 C2 D2 (5,4)MDS A1 B1 A2 B2 E1

70 Example 3: (4,3)MDS → (5,4)MDS 70 C1 D1 F1 A1 ⊕ D1 ⊕ B1 ⊕ E1 B1 ⊕ E1 ⊕ C1 ⊕ F1 A2 ⊕ D2 ⊕ B2 ⊕ E2 B2 ⊕ E2 ⊕ C2 ⊕ F2 E2 F2 C2 D2 (5,4)MDS A1 B1 A2 B2 E1

71 Example 3: (4,3)MDS → (5,4)MDS 71 C1 D1 F1 A1 ⊕ D1 ⊕ B1 ⊕ E1 B1 ⊕ E1 ⊕ C1 ⊕ F1 A2 ⊕ D2 ⊕ B2 ⊕ E2 B2 ⊕ E2 ⊕ C2 ⊕ F2 E2 F2 C2 D2 (5,4)MDS A1 B1 A2 B2 E1 D1

72 Example 3: (4,3)MDS → (5,4)MDS 72 C1 F1 A1 ⊕ D1 ⊕ B1 ⊕ E1 B1 ⊕ E1 ⊕ C1 ⊕ F1 A2 ⊕ D2 ⊕ B2 ⊕ E2 B2 ⊕ E2 ⊕ C2 ⊕ F2 E2 F2 C2 D2 (5,4)MDS A1 B1 A2 B2 E1 D1

73 Example 3: (4,3)MDS → (5,4)MDS 73 A1 ⊕ D1 ⊕ B1 ⊕ E1 B1 ⊕ E1 ⊕ C1 ⊕ F1 A2 ⊕ D2 ⊕ B2 ⊕ E2 B2 ⊕ E2 ⊕ C2 ⊕ F2 (5,4)MDS

74 Example 3: (4,3)MDS → (5,4)MDS 74 A1 ⊕ D1 ⊕ B1 ⊕ E1 B1 ⊕ E1 ⊕ C1 ⊕ F1 A2 ⊕ D2 ⊕ B2 ⊕ E2 B2 ⊕ E2 ⊕ C2 ⊕ F2 (5,4)MDS A1 ⊕ D1 ⊕ B1 ⊕ E1B1 ⊕ E1 ⊕ C1 ⊕ F1 ⊕ A2 ⊕ D2 ⊕ B2 ⊕ E2 B1 ⊕ E1 ⊕ C1 ⊕ F1 B2 ⊕ E2 ⊕ C2 ⊕ F2 ⊕

75 Example 3: (4,3)MDS → (5,4)MDS 75 C1 F1 E2 F2 C2 D2 (5,4)MDS A1 B1 A2 B2 E1 D1 A1 ⊕ D1 ⊕ C1 ⊕ F1 B1 ⊕ E1 ⊕ C1 ⊕ F1 ⊕ B2 ⊕ E2 ⊕ C2 ⊕ F2 A2 ⊕ D2 ⊕ B2 ⊕ E2

76 Example 3: (4,3)MDS → (5,4)MDS 76 C1 F1 E2 F2 C2 D2 (5,4)MDS A1 B1 A2 B2 E1 D1 A1 ⊕ D1 ⊕ C1 ⊕ F1 B1 ⊕ E1 ⊕ C1 ⊕ F1 ⊕ B2 ⊕ E2 ⊕ C2 ⊕ F2 A2 ⊕ D2 ⊕ B2 ⊕ E2 Data collector any 4 of 5 nodes can rebuild file

77 Example 3: (4,3)MDS → (5,4)MDS 77 C1 F1 E2 F2 C2 D2 (5,4)MDS A1 B1 A2 B2 E1 D1 A1 ⊕ D1 ⊕ C1 ⊕ F1 B1 ⊕ E1 ⊕ C1 ⊕ F1 ⊕ B2 ⊕ E2 ⊕ C2 ⊕ F2 A2 ⊕ D2 ⊕ B2 ⊕ E2 Data collector any 4 of 5 nodes can rebuild file Scaling traffic = 3 blocks

78 Example 3: (4,3)MDS → (5,4)MDS 78 C1 F1 E2 F2 C2 D2 (5,4)MDS A1 B1 A2 B2 E1 D1 A1 ⊕ D1 ⊕ C1 ⊕ F1 B1 ⊕ E1 ⊕ C1 ⊕ F1 ⊕ B2 ⊕ E2 ⊕ C2 ⊕ F2 A2 ⊕ D2 ⊕ B2 ⊕ E2 Data collector any 4 of 5 nodes can rebuild file Scaling traffic = 3 blocks

79 Example 3: (4,3)MDS → (5,4)MDS 79 C1 F1 E2 F2 C2 D2 (5,4)MDS A1 B1 A2 B2 E1 D1 A1 ⊕ D1 ⊕ C1 ⊕ F1 B1 ⊕ E1 ⊕ C1 ⊕ F1 ⊕ B2 ⊕ E2 ⊕ C2 ⊕ F2 A2 ⊕ D2 ⊕ B2 ⊕ E2 Data collector any 4 of 5 nodes can rebuild file Scaling traffic = 3 blocks Optimal code

80 Open Problems 1.What is the theoretical bound of scaling traffic from (n,k)MDS to (n+m,k+m)MDS? 2.How to design the generalized optimal scaling algorithms to match the bound? 3.How to deal with more complicated cases, such as iterative scaling? 4.What about (n,k)MDS → (n-m,k-m)MDS? 80 Generalize form: How to minimize the scaling traffic in storage system from (n,k) to (n’,k’) MDS codes. Generalize form: How to minimize the scaling traffic in storage system from (n,k) to (n’,k’) MDS codes.

81 Fin 81


Download ppt "22/07/2011 1 The MDS Scaling Problem for Cloud Storage Yu-chong Hu Institute of Network Coding."

Similar presentations


Ads by Google