Presentation is loading. Please wait.

Presentation is loading. Please wait.

Mark Feltner.  2.5 Petabytes/day: Wal-Mart's transaction database  40 Terabytes/second: CERN  1 Terabyte/day: NYSE Trading data  10 billion: Facebook.

Similar presentations


Presentation on theme: "Mark Feltner.  2.5 Petabytes/day: Wal-Mart's transaction database  40 Terabytes/second: CERN  1 Terabyte/day: NYSE Trading data  10 billion: Facebook."— Presentation transcript:

1 Mark Feltner

2

3  2.5 Petabytes/day: Wal-Mart's transaction database  40 Terabytes/second: CERN  1 Terabyte/day: NYSE Trading data  10 billion: Facebook photos

4  Theory  Algorithms  Implementations & Technology

5

6

7  All-or-nothing

8  Data is always in a valid state

9  Serially executed transactions result in same state as concurrent transactions

10  COMMIT means transaction is permanent across all clients

11

12

13

14

15

16 1. The network is reliable. 2. Latency is zero. 3. Bandwidth is infinite. 4. The network is secure. 5. Topology doesn't change. 6. There is one administrator. 7. Transport cost is zero. 8. The network is homogeneous.

17

18  Eventual consistency “…there must exist a total order on all operations such that each operation looks as if it were completed at a single instant. This is equivalent to requiring requests of the distributed shared memory to act as if they were executing on a single node, responding to operations one at a time.” (Gilbert, Lynch)

19 “For a distributed system to be continuously available, every request received by a non-failing node in the system must result in a response” (Gilbert, Lynch)

20 “In order to model partition tolerance, the network will be allowed to lose arbitrarily many messages sent from one node to another. When a network is partitioned, all messages sent from nodes in one component of the partition to nodes in another component are lost”(Gilbert, Lynch)

21

22

23 TitleArtistAlbumYear Breaking the LawJudas PriestBritish Steel1980 Aces HighIron MaidenPowerslave1984 Kickstart My HeatMotley CrueDr. Feelgood1989 Raining BloodSlayerReign in Blood1986 I Wanna Be SomebodyW.A.S.P. 1984

24 Data Storage Model: Breaking the Law Judas Priest British Steel 1980 Aces High Iron Maiden Powerslave 1984 Kickstart My heart Motley Crue Dr. Feelgood 1989 Raining Blood Slayer Reign in Blood 1986 I Wanna Be Somebody W.A.S.P. 1984

25 Data Storage Model: Breaking the Law Aces High Kickstart My Heart Raining Blood I Wanna Be Somebody Judas Priest Iron Madien Motley Crue Slayer W.A.S.P. British Steel Powerslave Dr. Feelgood Reign in Blood W.A.S.P. 1980 1984 1989 1986 1984

26  CREATE  SELECT  MAX, MIN, SUM, AVG, …

27

28

29

30  High performance  MapReduce  Powers: Google Reader, Maps, Book Search, YouTube, Gmail, …

31  MapReduce  Yahoo!  World Record Holder!

32  Key-value  MapReduce  Facebook  Eventual consistency  Scalable, fault-tolerant

33  Relational  LAMP

34  Key-value  What is lacks in durability, it makes up for in speed / simplicity.

35  MapReduce  Hadoop + HDFS  Java and REST API  Column-oriented  Excellent fault-tolerance  Replication  Streaming

36  Graph Database

37  Document-oriented

38  Pick the right tool for the job.

39


Download ppt "Mark Feltner.  2.5 Petabytes/day: Wal-Mart's transaction database  40 Terabytes/second: CERN  1 Terabyte/day: NYSE Trading data  10 billion: Facebook."

Similar presentations


Ads by Google