1. A Simple Optimistic skip-list Algorithm Maurice Herlihy Brown University & Sun Microsystems Laboratories Yossi Lev Brown University & Sun Microsystems Laboratories Victor Luchangco Sun Microsystems Laboratories Nir Shavit Tel-Aviv University & Sun Microsystems Laboratories Assaf Shemesh

2. Talk outline  Introduction.  What is skip-list?  The Algorithm.  Add operation.  Remove operation.  Contains operation.  Correctness.  Performance.  Conclusions.

3. Why use skip-list on Concurrent Programming? Why not use trees? like Red-Black trees, AVL trees, etc… Because trees need to be rebalanced after adding and removing items. This operations are GLOBAL.

4. Introduction Important data structure for storing and retrieving ordered in-memory data. This algorithm VS Implementation by Pugh and by Doug Lea. Main keys of this algorithm: Optimistic - The methods traverse the list without acquiring locks. In addition Only when a method discovers the items it is seeking, does it lock the item and its predecessors. Lazy - removing an item involves logically deleting it by marking it before it is physically removed. A typical pattern is 90% search operations, 9% inserts, and only 1% deletes Works as well as the Lea’s algorithm under this common usage pattern, and much better only under uncommon conditions of extreme contention in multi programmed environments. This algorithm provide a viable alternative to Lea algorithm.

5. What is skip-list? Linked list sorted by keys. Each node is assigned a random height, up to maximum for some fixed probability P. The number of nodes at any height decreases exponentially with that height. Each node has one successor at each level of height. INVARIENT: Each list is a sub list of the list at the layer beneath it. Usually there is a left sentinel and right sentinel at the beginning and the end of the lists with maximum height.

6. What is skip-list? Operations on skip-list   S1S1 S2S2 S3S3  31  64  3134  23 56 6478  313444  122326 S0S0  Searching – search 78 in the set O(logn)

7. The algorithm Class Node{ int key ; int topLayer ; Node** nexts ; bool marked ; bool fullyLinked; Lock lock ; }; Lazy list at each layer. The key of each node is strictly greater than the key of its predecessor. The value of the height of the node. Array in size of top layer. For each layer the node contain his successor. Boolean value which is used to make remove operations appear atomic. Boolean value which is set to true after a node has been linked in at all its layers The value of the node.

8. The algorithm A key is in the abstract set if and only if there is an unmarked, fully linked node with that key in the list. changes are made to the list structure only when locks are acquired for all nodes that need to be modified. Use LSentinel and RSentinel with the values MININT-1 and MAXINT+1. MININT - The lowest value that can enter the set. MAXINT - The highest value that can enter the set.

9. The algorithm Searching the set us accomplished by the findNode helper function. findNode – Input: A key v and two arrays of node pointers, “preds” and “succs”. Output: Returns the height if the node if it was found, otherwise, it returns -1. sets “preds” and “succs” accordantly. findNode searches exactly as in a sequential skip-list. findNode does not acquire any locks. findNode don’t retry in case of conflicting access with some other thread. “findNode” helper function

10. The algorithm int findNode ( int v,Node* preds [ ],Node* succs [ ] ) { int lFound = -1; Node* pred = &LSentinel ; for (int layer = MaxHeight -1;layer > 0 ;layer--) { Node* curr = pred->nexts [ layer ] ; while ( v > curr->key ) { pred = curr ; curr = pred->nexts [ layer ] ; } if ( lFound == -1 && v == curr->key ) { lFound = layer ; } preds [ layer ] = pred ; succs [ layer ] = curr ; } return lFound ; } “findNode” helper function

11. The algorithm - add add(v) adds v to the set and returns true iff v was not already in the set. Calls findNode to determine whether a node with the key is already in the list. General Description int topLayer = randomLevel ( MaxHeight ) ; Node* preds [ MaxHeight ], succs [MaxHeight ] ; while ( true ) { int lFound = findNode ( v, preds, succs ) ; if ( lFound != -1) { Node* nodeFound = succs [ lFound ] ; if ( ! nodeFound->marked ) { while ( ! nodeFound->fullyLinked ) { ; } return false ; } continue ; } STEP 1

12. 2 2 5 5 8 8 7 7 9 9 0 0 0 0 6 6 The algorithm - add Example – add 6 to the set - Marked flag - Fullylinked flag

13. 6 2 2 5 5 8 8 7 7 9 9 0 0 0 0 0 6 6 The algorithm - add Example – add 6 to the set - Marked flag - Fullylinked flag STEP 1

14. The algorithm - add add(v) adds v to the set and returns true iff v was not already in the set. locks and validates all the predecessors returned by findNode up to the height of the new node. General Description Int highestLocked = -1; try { Node *pred, *succ, *prevPred = null ; bool valid = true ; for ( int layer = 0 ; valid && ( layer <= topLayer ) ;layer++) { pred = preds [ layer ] ; succ = succs [ layer ] ; if ( pred != prevPred ) { pred->lock. lock ( ) ; highestLocked = layer ; prevPred = pred ; } valid = ! Pred->marked && ! Succ->marked && pred¡>nexts [ layer]==succ ; } If ( ! valid ) continue; STEP 2

15. 6 2 2 5 5 8 8 7 7 9 9 0 0 0 0 0 0 6 6 The algorithm - add Example – add 6 to the set - Marked flag - Fullylinked flag STEP 2

16. The algorithm - add add(v) adds v to the set and returns true iff v was not already in the set. allocates a new node with the appropriate key and height and links it in. General Description Node* newNode = new Node ( v, topLayer ) ; for ( int layer = 0 ;layer <= topLayer ;layer++) { newNode->nexts [ layer ] = succs [ layer ] ; preds [ layer]->nexts [ layer ] = newNode ; } newNode->fullyLinked = true; return true ; }//end of try finally { unlock ( preds, highestLocked ) ; } }//end of function STEP 3

17. 8 8 7 7 9 9 2 2 5 5 0 0 0 0 0 0 The algorithm - add Example – add 6 to the set - Marked flag - Fullylinked flag STEP 3 6 6 6 Fullylinked = true

18. 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 The algorithm - add Example – add 6 to the set - Marked flag - Fullylinked flag Success!

19. The algorithm - remove removes v from the set and returns true iff v was in the set. okToDelete function General Description bool okToDelete ( Node¤ candidate, int lFound ) { return ( candidate->fullyLinked && candidate->topLayer==lFound && !Candidate->marked ) ; }

20. The algorithm - remove removes v from the set and returns true iff v was in the set. determine whether a node with the appropriate key is in the list General Description Node* nodeToDelete = null ; bool isMarked = false ; int topLayer = -1; Node* preds [ MaxHeight ], succs [ MaxHeight ] ; while ( true ) { int lFound = findNode ( v, preds, succs ) ; if ( isMarked || ( lFound != -1 && okToDelete ( succs [ lFound ], lFound ) ) ) { if ( ! isMarked ) { nodeToDelete = succs [ lFound ] ; topLayer = nodeToDelete->topLayer ; nodeToDelete->lock. lock ( ) ; if ( nodeToDelete¡>marked ) { nodeToDelete->lock. unlock ( ) ; return false ; } nodeToDelete->marked = true ; isMarked = true ; } STEP 1

21. The algorithm - remove Example – remove 6 from the list 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 0 - Marked flag - Fullylinked flag

22. 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 The algorithm - remove Example – remove 6 from the list - Marked flag - Fullylinked flag STEP 1

23. The algorithm - remove removes v from the set and returns true iff v was in the set. locking its predecessors at all layers up to the height of the deleted node General Description int highestLocked = -1; try { Node *pred, *succ, *prevPred = null ; bool valid = true ; for ( int layer = 0 ;valid && ( layer <= topLayer ) ;layer++) { pred = preds [ layer ] ; succ = succs [ layer ] ; if ( pred != prevPred ) { pred->lock. lock ( ) ; highestLocked = layer ; prevPred = pred ; } valid = ! Pred->marked && pred->nexts [ layer]==succ ; } if ( ! valid ) continue ; STEP 2

24. 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 1 0 0 0 The algorithm - remove Example – remove 6 from the list - Marked flag - Fullylinked flag

25. The algorithm - remove removes v from the set and returns true iff v was in the set. splicing the node out one layer at a time General Description for (int layer = topLayer ; layer >=0 ; layer--) { preds [ layer]->nexts [ layer ] = nodeToDelete->nexts [ layer ] ; } nodeToDelete->lock. unlock ( ) ; return true ; }//end of try finally { unlock ( preds, highestLocked ) ; } }//end of if(true) else return false ; }//end of while }//end of function STEP 3

26. 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 1 0 0 0 The algorithm - remove Example – remove 6 from the list - Marked flag - Fullylinked flag STEP 3

27. 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 1 0 0 0 The algorithm - remove Example – remove 6 from the list - Marked flag - Fullylinked flag STEP 3

28. 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 1 0 0 0 The algorithm - remove Example – remove 6 from the list - Marked flag - Fullylinked flag STEP 3

29. 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 1 0 0 0 The algorithm - remove Example – remove 6 from the list - Marked flag - Fullylinked flag STEP 3

30. 8 8 7 7 9 9 2 2 5 5 0 0 0 0 0 0 0 The algorithm - remove Example – remove 6 from the list - Marked flag - Fullylinked flag STEP 3

31. The algorithm - contains returns true iff v is in the set General Description calls findNode and returns true if and only if it finds a unmarked, fully linked node with the appropriate key.

32. Correctness Linearzability Linearizability: (an intuitive definition) Can find a point within the time-interval of each operation, where the operation took place, such that the operations order is legal. an operation (or set of operations) is linearizable if it appears to the rest of the system to occur instantaneously. time q.enq(x) q.enq(y)q.deq(x) q.deq(y) q.enq(x) q.enq(y)q.deq(x) q.deq(y) time

33. Correctness Linearzability point of REMOVE is when MARKED bit is set. Linearzability point of ADD is when FULLYLINKED bit is set. Linearzability time Add(v) Remove(v) time newNode->fullyLinked = true ; nodeToDelete->marked = true ;

34. Correctness Maintaining the skip-list invariant skip-list invariant - the list at each layer is a sub list of the lists at lower layers. Add operation - linking new nodes into the skip list always proceeds from bottom to top. Remove operation - the higher layers are unlinked before the lower layers.

35. Correctness Dead lock freedom a thread always acquires locks on nodes with larger keys first. if a thread holds a lock on a node with key v then it will not attempt to acquire a lock on a node with key greater than or equal to v. acquiring locks on the predecessor nodes from the bottom layer up Contains operation is: a.Wait-freedom b.Lock-freedom c.Obstruction-freedom (if every operation has a bound on the number of steps the algorithm will take before the operation completes) (if it satisfies that when the program threads are run sufficiently long at least one of the threads make progress) (if at any point, a single thread executed in isolation)

36. Performance This algorithm VS Concurrent skip-list written by Doug Lea Throughput in operations per millisecond of 1,000,000 operations 90% search operations, 9% inserts, and only 1% deletes. Different versions of Experiments

37. Conclusions A scalable highly concurrent skip-list using a simple algorithm. Support operations - Add, Remove, Contains. Simple correctness prove For most uses, it offers an interesting viable alternative to the Concurrent Skip List by Doug Lea.