1. Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit Concurrent Skip Lists

2. 2 Set Object Interface Collection of elements No duplicates Methods –add() a new element –remove() an element –contains() if element is present Art of Multiprocessor Programming

3. 3 Many are Cold but Few are Frozen Typically high % of contains() calls Many fewer add() calls And even fewer remove() calls –90% contains() –9% add() –1% remove() Folklore? –Yes but probably mostly true Art of Multiprocessor Programming

4. 4 Concurrent Sets Balanced Trees? –Red-Black trees, AVL trees, … Problem: no one does this well … … because rebalancing after add() or remove() is a global operation Art of Multiprocessor Programming

5. 5 Skip Lists 2 2 5 5 8 8 7 7 9 9 0 0 Probabilistic Data Structure No global rebalancing Logarithmic-time search Art of Multiprocessor Programming

6. 6 Skip List Property 9 9 0 0 Each layer is sub-list of lower levels Art of Multiprocessor Programming

7. 7 Skip List Property 7 7 9 9 0 0 Each layer is sub-list of lower-levels Art of Multiprocessor Programming

8. 8 Skip List Property 5 5 7 7 9 9 0 0 Each layer is sub-list of lower levels Art of Multiprocessor Programming

9. 9 Skip List Property 5 5 8 8 7 7 9 9 0 0 Each layer is sub-list of lower levels Art of Multiprocessor Programming

10. 10 Skip List Property 2 2 5 5 8 8 7 7 9 9 0 0 Each layer is sub-list of lower levels Lowest level is entire list Art of Multiprocessor Programming

11. 11 Skip List Property 2 2 5 5 8 8 7 7 9 9 0 0 Each layer is sub-list of lower levels Not easy to preserve in concurrent implementations … Art of Multiprocessor Programming

12. 12 Search 2 2 5 5 8 8 7 7 9 9 0 0 contains(8) Too far Art of Multiprocessor Programming

13. 13 Search 2 2 5 5 8 8 7 7 9 9 0 0 contains(8) OK Art of Multiprocessor Programming

14. 14 Search 2 2 5 5 8 8 7 7 9 9 0 0 contains(8) Too far Art of Multiprocessor Programming

15. 15 Search 2 2 5 5 8 8 7 7 9 9 0 0 contains(8) Too far Art of Multiprocessor Programming

16. 16 Search 2 2 5 5 8 8 7 7 9 9 0 0 contains(8) Yes! Art of Multiprocessor Programming

17. 17 7 7 Search 8 0 0 2 2 5 5 9 9 contains(8) Art of Multiprocessor Programming

18. 18 7 7 Logarithmic 8 0 0 2 2 5 5 9 9 contains(8) Log N Art of Multiprocessor Programming

19. 19 Why Logarthimic 2 2 5 5 8 8 7 7 9 9 0 0 Property: Each pointer at layer i jumps over roughly 2 i nodes Pick node heights randomly so property guaranteed probabilistically 2i2i 2i2i Art of Multiprocessor Programming

20. 20 Sequential Find int find(T x, Node [] preds, Node [] succs) { … } Art of Multiprocessor Programming

21. 21 Sequential Find int find(T x, Node [] preds, Node [] succs) { … } object height (-1 if not there) Art of Multiprocessor Programming

22. 22 Sequential Find int find(T x, Node [] preds, Node [] succs) { … } Object sought object height (-1 if not there) Art of Multiprocessor Programming

23. 23 Sequential Find int find(T x, Node [] preds, Node [] succs) { … } object sought return predecessors Object height (-1 if not there) Art of Multiprocessor Programming

24. 24 Sequential Find int find(T x, Node [] preds, Node [] succs) { … } object sought return predecessors Object height (-1 if not there) Art of Multiprocessor Programming return successors

25. 25 Successful Search 2 2 5 5 8 8 7 7 9 9 0 0 find(7, …) preds 0 1 2 3 4 Art of Multiprocessor Programming

26. 26 Successful Search 2 2 5 5 8 8 7 7 9 9 0 0 find(7, …) succs 0 1 2 3 4 preds 0 1 2 3 4 Art of Multiprocessor Programming

27. 27 Unsuccessful Search 2 2 5 5 8 8 7 7 9 9 0 0 find(6, …) preds 0 1 2 3 4 Art of Multiprocessor Programming

28. 28 Unsuccessful Search 2 2 5 5 8 8 7 7 9 9 0 0 find(6, …) succs 0 1 2 3 4 preds 0 1 2 3 4 Art of Multiprocessor Programming

29. 29 Lazy Skip List Mix blocking and non-blocking techniques: –Use optimistic-lazy locking for add() and remove() –Wait-free contains() Remember: typically lots of contains() calls but few add() and remove() Art of Multiprocessor Programming

30. 30 Review: Lazy List Remove aa b c d Art of Multiprocessor Programming

31. 31 Review: Lazy List Remove aa b c d Present in list Art of Multiprocessor Programming

32. 32 Review: Lazy List Remove aa b c d Logically deleted Art of Multiprocessor Programming

33. 33 Review: Lazy List Remove aa b c d Physically deleted Art of Multiprocessor Programming

34. 34 Lazy Skip Lists 2 2 5 5 8 8 7 7 Use a mark bit for logical deletion 9 9 0 0 0 0 0 0 0 0 Art of Multiprocessor Programming

35. 35 add(6) Create node of (random) height 4 2 2 5 5 8 8 7 7 9 9 0 0 0 0 6 6 Art of Multiprocessor Programming

36. 36 add(6) find() predecessors 6 2 2 5 5 8 8 7 7 9 9 0 0 0 0 0 6 6 Art of Multiprocessor Programming

37. 37 add(6) find() predecessors Lock them 6 2 2 5 5 8 8 7 7 9 9 0 0 0 0 0 0 6 6 Art of Multiprocessor Programming

38. 38 add(6) find() predecessors Lock them Validate 6 2 2 5 5 8 8 7 7 9 9 0 0 0 0 0 0 6 6 Optimistic approach Art of Multiprocessor Programming

39. 39 add(6) 8 8 7 7 9 9 2 2 5 5 0 0 find() predecessors Lock them Validate Splice 0 6 6 0 0 0 Art of Multiprocessor Programming

40. 40 add(6) find() predecessors Lock them Validate Splice Unlock 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 Art of Multiprocessor Programming

41. 41 remove(6) 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 0 Art of Multiprocessor Programming

42. 42 remove(6) find() predecessors 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 0 Art of Multiprocessor Programming

43. 43 remove(6) find() predecessors Lock victim 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 0 Art of Multiprocessor Programming

44. 44 8 8 7 7 9 9 2 2 5 5 0 0 6 6 remove(6) find() predecessors Lock victim Set mark (if not already set) 0 0 0 0 0 Logical remove… Art of Multiprocessor Programming

45. 45 8 8 7 7 9 9 2 2 5 5 0 0 6 6 remove(6) find() predecessors Lock victim Set mark (if not already set) Lock predecessors (ascending order) & validate 0 0 1 0 0 0 Art of Multiprocessor Programming

46. 46 remove(6) find() predecessors Lock victim Set mark (if not already set) Lock predecessors (ascending order) & validate Physically remove 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 1 0 0 0 Art of Multiprocessor Programming

47. 47 remove(6) find() predecessors Lock victim Set mark (if not already set) Lock predecessors (ascending order) & validate Physically remove 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 1 0 0 0 Art of Multiprocessor Programming

48. 48 remove(6) find() predecessors Lock victim Set mark (if not already set) Lock predecessors (ascending order) & validate Physically remove 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 1 0 0 0 Art of Multiprocessor Programming

49. 49 remove(6) find() predecessors Lock victim Set mark (if not already set) Lock predecessors (ascending order) & validate Physically remove 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 1 0 0 0 Art of Multiprocessor Programming

50. 50 remove(6) find() predecessors Lock victim Set mark (if not already set) Lock predecessors (ascending order) & validate Physically remove 8 8 7 7 9 9 2 2 5 5 0 0 0 0 0 0 0 Art of Multiprocessor Programming

51. 51 contains(8) find() & not marked 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 0 Art of Multiprocessor Programming

52. 52 contains(8) 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 0 Node 6 removed while traversed Art of Multiprocessor Programming

53. 53 contains(8) 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 Node removed while being traversed Art of Multiprocessor Programming

54. 54 contains(8) 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 Prove: an unmarked node (like 8) remains reachable Art of Multiprocessor Programming

55. 55 8 8 7 7 9 9 2 2 5 5 0 0 6 6 remove(6): Linearization Successful remove happens when bit is set 0 0 0 0 0 Logical remove… Art of Multiprocessor Programming

56. 56 Add: Linearization 8 8 7 7 9 9 2 2 5 5 0 0 Successful add() at point when fully linked Add fullyLinked bit to indicate this Bit tested by contains() 0 0 6 6 0 0 0 Art of Multiprocessor Programming

57. 57 contains(7): Linearization 8 8 7 7 9 9 2 2 5 5 0 0 6 6 0 0 0 0 0 When fully-linked unmarked node found Pause while fullyLinked bit unset Art of Multiprocessor Programming

58. 58 contains(7): Linearization 8 8 6 6 9 9 2 2 5 5 0 0 7 7 0 0 1 0 0 When do we linearize unsuccessful Search? 1 So far OK… Art of Multiprocessor Programming

59. 59 contains(7): Linearization 8 8 6 6 9 9 2 2 5 5 0 0 7 7 0 0 0 0 When do we linearize unsuccessful Search? 7 7 But what if a new 7 added concurrently? Art of Multiprocessor Programming

60. 60 contains(7): Linearization 8 8 6 6 9 9 2 2 5 5 0 0 7 7 0 0 0 0 When do we linearize unsuccessful Search? 1 7 7 Prove: at some point 7 was not in the skip list Art of Multiprocessor Programming

61. 61 A Simple Experiment Each thread runs 1 million iterations, each either: –add() –remove() –contains() Item and method chosen in random from some distribution Art of Multiprocessor Programming

62. 62 Lazy Skip List: Performance Multiprogramming Art of Multiprocessor Programming

63. 63 Lazy Skip List: Performance Multiprogramming Higher contention Art of Multiprocessor Programming

64. 64 Lazy Skip List: Performance Multiprogramming Unrealistic Contention Art of Multiprocessor Programming

65. 65 Summary Lazy Skip List –Optimistic fine-grained Locking Performs as well as the lock-free solution in “common” cases Simple Art of Multiprocessor Programming

66. 66 This work is licensed under a Creative Commons Attribution-ShareAlike 2.5 License.Creative Commons Attribution-ShareAlike 2.5 License You are free: –to Share — to copy, distribute and transmit the work –to Remix — to adapt the work Under the following conditions: –Attribution. You must attribute the work to “The Art of Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work). –Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to –http://creativecommons.org/licenses/by-sa/3.0/. Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights. Art of Multiprocessor Programming