Presentation is loading. Please wait.

Presentation is loading. Please wait.

Linked Lists: Locking, Lock-Free, and Beyond … The Art of Multiprocessor Programming Spring 2007.

Similar presentations


Presentation on theme: "Linked Lists: Locking, Lock-Free, and Beyond … The Art of Multiprocessor Programming Spring 2007."— Presentation transcript:

1 Linked Lists: Locking, Lock-Free, and Beyond … The Art of Multiprocessor Programming Spring 2007

2 © Herlihy-Shavit Last Lecture: Spin-Locks CS Resets lock upon exit spin lock critical section...

3 © Herlihy-Shavit Today: Concurrent Objects AddingThreads Should not lower throughput –Contention effects –Mostly fixed by Queue locks Should increase throughput –Not possible if inherently sequential –Surprising things are parallelizable

4 © Herlihy-Shavit Coarse-Grained Synchronization Each method locks the object –Avoid contention using queue locks –Easy to reason about In simple cases –Standard Java model Synchronized blocks and methods So, are we done?

5 © Herlihy-Shavit Coarse-Grained Synchronization Sequential bottleneck –All threads “stand in line” Adding more threads –Does not improve throughput –Struggle to keep it from getting worse So why even use a multiprocessor? –Well, some apps inherently parallel …

6 © Herlihy-Shavit This Lecture Introduce four “patterns” –Bag of tricks … –Methods that work more than once … For highly-concurrent objects Goal: –Concurrent access –More threads, more throughput

7 © Herlihy-Shavit First: Fine-Grained Synchronization Instead of using a single lock.. Split object into –Independently-synchronized components Methods conflict when they access –The same component … –At the same time

8 © Herlihy-Shavit Second: Optimistic Synchronization Object = linked set of components Search without locking … If you find it, lock and check … –OK, we are done –Oops, try again Evaluation –cheaper than locking –mistakes are expensive

9 © Herlihy-Shavit Third: Lazy Synchronization Postpone hard work Removing components is tricky –Logical removal Mark component to be deleted –Physical removal Do what needs to be done

10 © Herlihy-Shavit Fourth: Lock-Free Synchronization Don’t use locks at all –Use compareAndSet() & relatives … Advantages –Robust against asynchrony Disadvantages –Complex –Sometimes high overhead

11 © Herlihy-Shavit Linked List Illustrate these patterns … Using a list-based Set –Common application –Building block for other apps

12 © Herlihy-Shavit The Evolution of Concurrent List-based Sets Course Grained Fine Lock- Coupling Course Optimistic Fine Optim- istic Lock- free Lazy Fine Optim- istic

13 © Herlihy-Shavit Set Interface Unordered collection of objects No duplicates Methods –Add() a new object –Remove() an object –Test if set Contains() object

14 © Herlihy-Shavit List-Based Sets public interface Set { public boolean add(Object x); public boolean remove(Object x); public boolean contains(Object x); }

15 © Herlihy-Shavit List-Based Sets public interface Set { public boolean add(Object x); public boolean remove(Object x); public boolean contains(Object x); } Add object to set

16 © Herlihy-Shavit List-Based Sets public interface Set { public boolean add(Object x); public boolean remove(Object x); public boolean contains(Object x); } Remove object from set

17 © Herlihy-Shavit List-Based Sets public interface Set { public boolean add(Object x); public boolean remove(Object x); public boolean contains(Object x); } Is object in set?

18 © Herlihy-Shavit List Node public class Node { public Object object; public int key; public Node next; }

19 © Herlihy-Shavit List Node public class Node { public Object object; public int key; public Node next; } Object of interest

20 © Herlihy-Shavit List Node public class Node { public Object object; public int key; public Node next; } Usually hash code

21 © Herlihy-Shavit List Node public class Node { public Object object; public int key; public Node next; } Reference to next node

22 © Herlihy-Shavit The List-Based Set abc Ordered + Sentinel nodes (min & max possible keys)

23 © Herlihy-Shavit Reasoning about Concurrent Objects Invariant –Property that always holds Established by –True when object is created –Truth preserved by each method Each step of each method

24 © Herlihy-Shavit Specifically … Invariants preserved by –add() –remove() –contains() Most steps are trivial –Usually one step tricky –Often linearization point

25 © Herlihy-Shavit Interference Proof that invariants preserved works only if –methods considered –are the only modifiers Language encapsulation helps –List nodes not visible outside class

26 © Herlihy-Shavit Interference Freedom from interference neeeded even for removed nodes –Some algorithms traverse removed nodes –Careful with malloc() & free() ! Garbage-collection helps here

27 © Herlihy-Shavit Abstract Data Types Concrete representation Abstract Type –{a, b} ab

28 © Herlihy-Shavit Abstract Data Types Meaning of rep given by abstraction map –S( ) = {a,b} a b

29 © Herlihy-Shavit Rep Invariant Which concrete values are meaningful? –Sorted? Duplicates? Rep invariant –Characterizes legal concrete reps –Preserved by methods –Relied on by methods

30 © Herlihy-Shavit Blame Game Rep invariant is a contract Suppose –add() leaves behind 2 copies of x –remove() removes only 1 Which one is incorrect?

31 © Herlihy-Shavit Blame Game Suppose –add() leaves behind 2 copies of x –remove() removes only 1 Which one is incorrect? –If rep invariant says no duplicates add() is incorrect –Otherwise remove() is incorrect

32 © Herlihy-Shavit Rep Invariant (partly) Sentinel nodes –tail reachable from head Sorted, no duplicates

33 © Herlihy-Shavit Abstraction Map S(head) = –{ x | there exists a such that A reachable from head and a.object = x –}

34 © Herlihy-Shavit Sequential List Based Set a c d b a b c Add() Remove()

35 © Herlihy-Shavit Course Grained Locking a b d c Simple but hotspot + bottleneck

36 © Herlihy-Shavit Coarse-Grained Locking Easy, same as synchronized methods –“One lock to rule them all …” Simple, clearly correct –Deserves respect! Works poorly with contention –Queue locks help –But bottleneck still an issue

37 © Herlihy-Shavit Fine-grained Locking Requires careful thought –“Do not meddle in the affairs of wizards, for they are subtle and quick to anger” Split object into pieces –Each piece has own lock –Methods that work on disjoint pieces need not exclude each other

38 © Herlihy-Shavit Hand-over-Hand locking abc

39 © Herlihy-Shavit Hand-over-Hand locking abc

40 © Herlihy-Shavit Hand-over-Hand locking abc

41 © Herlihy-Shavit Hand-over-Hand locking abc

42 © Herlihy-Shavit Hand-over-Hand locking abc

43 © Herlihy-Shavit Removing a Node abcd remov e b

44 © Herlihy-Shavit Removing a Node abcd remov e b

45 © Herlihy-Shavit Removing a Node abcd remov e b

46 © Herlihy-Shavit Removing a Node abcd remov e b

47 © Herlihy-Shavit Removing a Node acd remov e b

48 © Herlihy-Shavit Removing a Node abcd remov e b remov e c

49 © Herlihy-Shavit Removing a Node abcd remov e b remov e c

50 © Herlihy-Shavit Removing a Node abcd remov e b remov e c

51 © Herlihy-Shavit Removing a Node abcd remov e b remov e c

52 © Herlihy-Shavit Removing a Node abcd remov e b remov e c

53 © Herlihy-Shavit Removing a Node abcd remov e b remov e c

54 © Herlihy-Shavit Removing a Node abcd remov e b remov e c

55 © Herlihy-Shavit Removing a Node abcd remov e b remov e c

56 © Herlihy-Shavit Removing a Node abcd remov e b remov e c

57 © Herlihy-Shavit Uh, Oh acd remov e b remov e c Still in list

58 © Herlihy-Shavit Problem To delete node b –Swing node a’s next field to c Problem is, –Someone could delete c concurrently b acbac

59 © Herlihy-Shavit Insight If a node is locked –No one can delete node’s successor If a thread locks –Node to be deleted –And its predecessor –Then it works

60 © Herlihy-Shavit Hand-Over-Hand Again abcd remov e b

61 © Herlihy-Shavit Hand-Over-Hand Again abcd remov e b

62 © Herlihy-Shavit Hand-Over-Hand Again abcd remov e b

63 © Herlihy-Shavit Hand-Over-Hand Again abcd remov e b Found it!

64 © Herlihy-Shavit Hand-Over-Hand Again abcd remov e b Found it!

65 © Herlihy-Shavit Hand-Over-Hand Again acd remov e b Found it!

66 © Herlihy-Shavit Removing a Node abcd remov e b remov e c

67 © Herlihy-Shavit Removing a Node abcd remov e b remov e c

68 © Herlihy-Shavit Removing a Node abcd remov e b remov e c

69 © Herlihy-Shavit Removing a Node abcd remov e b remov e c

70 © Herlihy-Shavit Removing a Node abcd remov e b remov e c

71 © Herlihy-Shavit Removing a Node abcd remov e b remov e c

72 © Herlihy-Shavit Removing a Node abcd remov e b remov e c

73 © Herlihy-Shavit Removing a Node abd remov e b remov e c

74 © Herlihy-Shavit Removing a Node abd remov e b remov e c

75 © Herlihy-Shavit Removing a Node ad remov e b remov e c

76 © Herlihy-Shavit Remove method public boolean remove(Object object) { int key = object.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }}

77 © Herlihy-Shavit Remove method public boolean remove(Object object) { int key = object.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }} Key used to order node

78 © Herlihy-Shavit Remove method public boolean remove(Object object) { int key = object.hashCode(); Node pred, curr; try { … } finally { currNode.unlock(); predNode.unlock(); }} Predecessor and current nodes

79 © Herlihy-Shavit Remove method public boolean remove(Object object) { int key = object.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }} Make sure locks released

80 © Herlihy-Shavit Remove method public boolean remove(Object object) { int key = object.hashCode(); Node pred, curr; try { … } finally { curr.unlock(); pred.unlock(); }} Everything else

81 © Herlihy-Shavit Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … }

82 © Herlihy-Shavit Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } lock previous

83 © Herlihy-Shavit Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } Lock current

84 © Herlihy-Shavit Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); … } finally { … } Traversing list

85 © Herlihy-Shavit Remove: searching while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false;

86 © Herlihy-Shavit Remove: searching while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Search key range

87 © Herlihy-Shavit Remove: searching while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; At start of each loop: curr and pred locked

88 © Herlihy-Shavit Remove: searching while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; If node found, remove it

89 © Herlihy-Shavit Remove: searching while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; If node found, remove it

90 © Herlihy-Shavit Remove: searching while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Unlock predecessor

91 © Herlihy-Shavit Remove: searching while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Only one node locked!

92 © Herlihy-Shavit Remove: searching while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; demote current

93 © Herlihy-Shavit Remove: searching while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = currNode; curr = curr.next; curr.lock(); } return false; Find and lock new current

94 © Herlihy-Shavit Remove: searching while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = currNode; curr = curr.next; curr.lock(); } return false; Lock invariant restored

95 © Herlihy-Shavit Remove: searching while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Otherwise, not present

96 © Herlihy-Shavit Why does this work? To remove node e –Must lock e –Must lock e’s predecessor Therefore, if you lock a node –It can’t be removed –And neither can its successor

97 © Herlihy-Shavit while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Why remove() is linearizable head ~> pred A --> curr A so the object is in the set

98 © Herlihy-Shavit while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Why remove() is linearizable Node locked, so no other thread can remove it ….

99 © Herlihy-Shavit while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Why remove() is linearizable Linearization point

100 © Herlihy-Shavit while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Why remove() is linearizable Object not present

101 © Herlihy-Shavit while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Why remove() is linearizable pred A --> curr A pred A.key < key key < curr A.key

102 © Herlihy-Shavit while (curr.key <= key) { if (object == curr.object) { pred.next = curr.next; return true; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } return false; Why remove() is linearizable Linearization point: when curr A set to node with higher key

103 © Herlihy-Shavit Adding Nodes To add node e –Must lock predecessor –Must lock successor Neither can be deleted –(Is successor lock actually required?)

104 © Herlihy-Shavit Same Abstraction Map S(head) = –{ x | there exists a such that A reachable from head and a.object = x –}

105 © Herlihy-Shavit Rep Invariant Easy to check that –Tail always reachable from head –Nodes sorted, no duplicates

106 © Herlihy-Shavit Drawbacks Better than coarse-grained lock –Threads can traverse in parallel Still not ideal –Long chain of acquire/release –Inefficient

107 © Herlihy-Shavit Optimistic Synchronization Find nodes without locking Lock nodes Check that everything is OK

108 © Herlihy-Shavit Optimistic Fine Grained b d e To insert C: Optimistically traverse list to find b Lock b.curr then lock b.succ Re-Traverse list to find b and verify pred  curr Perform insert and release locks a c 4

109 © Herlihy-Shavit Same Abstraction Map S(head) = –{ x | there exists a such that A reachable from head and a.object = x –}

110 © Herlihy-Shavit Preserving Invariants Not same as before as we traverse deleted elements But we establish properties by –Validation –After we lock target nodes

111 © Herlihy-Shavit Scanning Deleted Items abde remov e e A Pred and curr could point to removed nodes

112 © Herlihy-Shavit Removing a Node abcd remov e c return true

113 © Herlihy-Shavit What Can Go Wrong? abcd remov e c remov e b

114 © Herlihy-Shavit Check that Node is Still Accessible abcd remov e c

115 © Herlihy-Shavit What Can Go Wrong? abcd remov e c Add b’ b’

116 © Herlihy-Shavit What Can Go Wrong? abcd remov e c b’

117 © Herlihy-Shavit Check that Nodes Still Adjacent abcd remov e c

118 © Herlihy-Shavit Correctness If –Nodes b and c both locked –Node b still accessible –Node c still successor to b Then –Neither will be deleted –OK to delete and return true

119 © Herlihy-Shavit Removing an Absent Node abde remov e c return false

120 © Herlihy-Shavit Correctness If –Nodes b and d both locked –Node b still accessible –Node d still successor to b Then –Neither will be deleted –No thread can add c after b –OK to return false

121 © Herlihy-Shavit Validation private boolean validate(Node pred, Node curry) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; }

122 © Herlihy-Shavit private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Predecessor & current nodes

123 © Herlihy-Shavit private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Start at the beginning

124 © Herlihy-Shavit private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Search range of keys

125 © Herlihy-Shavit private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Predecessor reachable

126 © Herlihy-Shavit private boolean validate(Node pred, Node curry) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Is current node next?

127 © Herlihy-Shavit private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Otherwise move on

128 © Herlihy-Shavit private boolean validate(Node pred, Node curr) { Node node = head; while (node.key <= pred.key) { if (node == pred) return pred.next == curr; node = node.next; } return false; } Validation Predecessor not reachable

129 © Herlihy-Shavit Remove: searching public boolean remove(Object object) { int key = object.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (object == curr.object) break; pred = curr; curr = curr.next; } …

130 © Herlihy-Shavit public boolean remove(Object object) { int key = object.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (object == curr.object) break; pred = curr; curr = curr.next; } … Remove: searching Search key

131 © Herlihy-Shavit public boolean remove(Object object) { int key = object.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (object == curr.object) break; pred = curr; curr = curr.next; } … Remove: searching Retry on synchronization conflict

132 © Herlihy-Shavit public boolean remove(Object object) { int key = object.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (object == curr.object) break; pred = curr; curr = curr.next; } … Remove: searching Examine predecessor and current nodes

133 © Herlihy-Shavit public boolean remove(Object object) { int key = object.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (object == curr.object) break; pred = curr; curr = curr.next; } … Remove: searching Search by key

134 © Herlihy-Shavit public boolean remove(Object object) { int key = object.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (object == curr.object) break; pred = curr; curr = curr.next; } … Remove: searching Stop if we find object

135 © Herlihy-Shavit public boolean remove(Object object) { int key = object.hashCode(); retry: while (true) { Node pred = this.head; Node curr = pred.next; while (curr.key <= key) { if (object == curr.object) break; pred = curr; curr = curr.next; } … Remove: searching Move along

136 © Herlihy-Shavit On Exit from Loop If object is present –curr holds object –pred just before curr If object is absent –curr has first higher key –pred just before curr Assuming no synchronization problems

137 © Herlihy-Shavit Remove Method try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.object == object) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}}

138 © Herlihy-Shavit try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.object == object) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Remove Method Always unlock

139 © Herlihy-Shavit try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.object == object) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Remove Method Lock both nodes

140 © Herlihy-Shavit try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.object == object) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Remove Method Check for synchronization conflicts

141 © Herlihy-Shavit try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.object == object) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Remove Method Object found, remove node

142 © Herlihy-Shavit try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.object == object) { pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Remove Method Object not found

143 © Herlihy-Shavit Summary: Wait-free Traversals b c d a 1.Traverse removed nodes 2.Must have non-interference 3.Natural in GC languages like Java™ Reachable = in the set

144 © Herlihy-Shavit Summary Optimistic List b c e a 1.Limited Hotspots (Only at locked Add(), Remove(), Find() destination locations, not traversals) 2.But two traversals 3.Yet traversals are wait-free!

145 © Herlihy-Shavit So Far, So Good Much less lock acquisition/release –Performance –Concurrency Problems –Need to traverse list twice –contains() method acquires locks Most common method call

146 © Herlihy-Shavit Evaluation Optimistic works if –cost of scanning twice without locks < –cost of Scanning once with locks Drawback –Contains() acquires locks –90% of calls in many apps

147 © Herlihy-Shavit Lazy List Like optimistic, except –Scan once –Contains() never locks … Key insight –Removing nodes causes trouble –Do it “lazily”

148 © Herlihy-Shavit Lazy List Remove Method –Scans list (as before) –Locks predecessor & current (as before) Logical delete –Marks current node as removed (new!) Physical delete –Redirects predecessor’s next (as before)

149 © Herlihy-Shavit Lazy Removal a a b c 0 d 1 c Logical Removal = Set Mark Bit Physical Removal Add a separate mark bit field to each node: 1. mark node as deleted 2. redirect pointer

150 © Herlihy-Shavit Lazy Removal a a b c 0 d c 4 Logical Removal Physical Removal Release Locks 1.Optimistically traverse list to find b 2.Lock b.curr then lock b.succ 3.verify pred  curr 4.Perform logical then physical removal 5.Release locks

151 © Herlihy-Shavit Lazy List All Methods –Scan through locked and marked nodes –Removing a node doesn’t slow down other method calls … Must still lock pred and curr nodes.

152 © Herlihy-Shavit Validation No need to rescan list! Check that pred is not marked Check that curr is not marked Check that pred points to curr

153 © Herlihy-Shavit Business as Usual abcd

154 © Herlihy-Shavit Business as Usual abcd

155 © Herlihy-Shavit Business as Usual abcd

156 © Herlihy-Shavit Business as Usual abcd

157 © Herlihy-Shavit Business as Usual abcd

158 © Herlihy-Shavit Interference abcd

159 © Herlihy-Shavit Interference abcd

160 © Herlihy-Shavit Interference abcd

161 © Herlihy-Shavit Interference abcd Remove c

162 © Herlihy-Shavit Validation abcd b not marked

163 © Herlihy-Shavit Interference abcd c not marked

164 © Herlihy-Shavit Interference abcd b still points to c

165 © Herlihy-Shavit Logical Delete abcd mark

166 © Herlihy-Shavit Scan Through abcd So what? Remove c

167 © Herlihy-Shavit Physical Deletion abcd So what? Remove c

168 © Herlihy-Shavit New Abstraction Map S(head) = –{ x | there exists node a such that A reachable from head and a.object = x and a is unmarked –}

169 © Herlihy-Shavit Invariant If not marked then item in the set and reachable from head and if not yet traversed it is reachable from pred

170 © Herlihy-Shavit Validation private boolean validate(Node pred, Node curr) { return !pred.next.marked && !curr.next.marked && pred.next == curr); }

171 © Herlihy-Shavit private boolean validate(Node pred, Node curr) { return !pred.marked && !curr.marked && pred.next == curr); } List Validate Method Predecessor not Logically removed

172 © Herlihy-Shavit private boolean validate(Node pred, Node curr) { return !pred.next.marked && !curr.next.marked && pred.next == curr); } List Validate Method Current not Logically removed

173 © Herlihy-Shavit private boolean validate(Node pred, Node curr) { return !pred.next.marked && !curr.next.marked && pred.next == curr); } List Validate Method Predecessor still Points to current

174 © Herlihy-Shavit Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}}

175 © Herlihy-Shavit Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Validate as before

176 © Herlihy-Shavit Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Key found

177 © Herlihy-Shavit Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} Logical remove

178 © Herlihy-Shavit Remove try { pred.lock(); curr.lock(); if (validate(pred,curr) { if (curr.key == key) { curr.marked = true; pred.next = curr.next; return true; } else { return false; }}} finally { pred.unlock(); curr.unlock(); }}} physical remove

179 © Herlihy-Shavit Contains public boolean contains(Object object) { int key = object.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; }

180 © Herlihy-Shavit Contains public boolean contains(Object object) { int key = object.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Start at the head

181 © Herlihy-Shavit Contains public boolean contains(Object object) { int key = object.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Search key range

182 © Herlihy-Shavit Contains public boolean contains(Object object) { int key = object.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Traverse without locking (nodes may have been removed)

183 © Herlihy-Shavit Contains public boolean contains(Object object) { int key = object.hashCode(); Node curr = this.head; while (curr.key < key) { curr = curr.next; } return curr.key == key && !curr.marked; } Present and undeleted?

184 © Herlihy-Shavit Summary: Wait-free Contains a a b c 0 e 1 d Use Mark bit + Fact that List is ordered 1.Not marked  in the set 2.Marked or missing  not in the set

185 © Herlihy-Shavit Lazy List a a b c 0 e 1 d Lazy Add() and Remove() + Wait-free Contains()

186 © Herlihy-Shavit Evaluation Good: –Contains method doesn’t need to lock –In fact, its wait-free! –Good because typically high % contains() –Uncontended calls don’t re-traverse Bad –Contended calls do re-traverse –Traffic jam if one thread delays

187 © Herlihy-Shavit Traffic Jam Any concurrent data structure based on mutual exclusion has a weakness If one thread –Enters critical section –And “eats the big muffin” (stops running) Cache miss, page fault, descheduled … –Everyone else using that lock is stuck!

188 © Herlihy-Shavit Reminder: Lock-Free Data Structures No matter what … –Some thread will complete method call –Even if others halt at malicious times –Weaker than wait-free, yet Implies that –You can’t use locks –Um, that’s why they call it lock-free

189 © Herlihy-Shavit Lock-free Lists Next logical step Eliminate locking entirely Contains() wait-free and Add() and Remove() lock-free Use only compareAndSet() What could go wrong?

190 © Herlihy-Shavit Adding a Node abc

191 © Herlihy-Shavit Adding a Node abcb

192 © Herlihy-Shavit Adding a Node abcb CAS

193 © Herlihy-Shavit Adding a Node abcb

194 © Herlihy-Shavit Adding a Node abcb

195 © Herlihy-Shavit Removing a Node abcd remov e b remov e c CAS

196 © Herlihy-Shavit Look Familiar? abcd remov e b remov e c Still in list

197 © Herlihy-Shavit Problem Method updates node’s next field After node has been removed

198 © Herlihy-Shavit Solution Use AtomicMarkableReference Atomically –Swing reference and –Update flag Remove in two steps –Set mark bit in next field –Redirect predecessor’s pointer

199 © Herlihy-Shavit Marking a Node AtomicMarkableReference class –Java.util.concurrent.atomic package address F mark bit Reference

200 © Herlihy-Shavit Extracting Reference & Mark Public T get(boolean[] marked);

201 © Herlihy-Shavit Extracting Reference & Mark Public T get(boolean[] marked); Returns reference to object of type T Returns mark at array index 0!

202 © Herlihy-Shavit Changing State Public boolean compareAndSet( T expectedRef, T updateRef, boolean expectedMark, boolean updateMark);

203 © Herlihy-Shavit Changing State Public boolean compareAndSet( T expectedRef, T updateRef, boolean expectedMark, boolean updateMark); If this is the current reference … And this is the current mark …

204 © Herlihy-Shavit Changing State Public boolean compareAndSet( T expectedRef, T updateRef, boolean expectedMark, boolean updateMark); …then change to this new reference … … and this new mark

205 © Herlihy-Shavit Changing State public boolean attemptMark( T expectedRef, boolean updateMark);

206 © Herlihy-Shavit Changing State public boolean attemptMark( T expectedRef, boolean updateMark); If this is the current reference …

207 © Herlihy-Shavit Changing State public boolean attemptMark( T expectedRef, boolean updateMark);.. then change to this new mark.

208 © Herlihy-Shavit Removing a Node abcd remov e c CAS

209 © Herlihy-Shavit Removing a Node abd remov e b remov e c c CAS failed

210 © Herlihy-Shavit Removing a Node abd remov e b remov e c c

211 © Herlihy-Shavit Removing a Node ad remov e b remov e c

212 © Herlihy-Shavit Traversing the List Q: what do you do when you find a “logically” deleted node in your path? A: finish the job. –CAS the predecessor’s next field –Proceed (repeat as needed)

213 © Herlihy-Shavit Lock-Free Traversal abcd CAS Uh-oh

214 © Herlihy-Shavit The Window Class class Window { public Node pred; public Node curr; Window(Node pred, Node curr) { this.pred = pred; this.curr = curr; }

215 © Herlihy-Shavit The Window Class class Window { public Node pred; public Node curr; Window(Node pred, Node curr) { this.pred = pred; this.curr = curr; } A container for pred and current values

216 © Herlihy-Shavit Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr;

217 © Herlihy-Shavit Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Find returns window

218 © Herlihy-Shavit Using the Find Method Window window = find(head, key); Node pred = window.pred; curr = window.curr; Extract pred and curr

219 © Herlihy-Shavit The Find Method Window window = find(head, key); At some instant, predcurrsucc object or …

220 © Herlihy-Shavit The Find Method Window window = find(head,key); At some instant, pred curr= succ succ object not in list

221 © Herlihy-Shavit Remove public boolean remove(T object) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}}

222 © Herlihy-Shavit Remove public boolean remove(T object) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Keep trying

223 © Herlihy-Shavit Remove public boolean remove(T object) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Find neighbors

224 © Herlihy-Shavit Remove public boolean remove(T object) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} She’s not there …

225 © Herlihy-Shavit Remove public boolean remove(T object) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Try to mark node as deleted

226 © Herlihy-Shavit Remove public boolean remove(T object) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} If it doesn’t work, just retry, if it does, job essentially done

227 © Herlihy-Shavit Remove public boolean remove(T object) { Boolean snip; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key != key) { return false; } else { Node succ = curr.next.getReference(); snip = curr.next.attemptMark(succ, true); if (!snip) continue; pred.next.compareAndSet(curr, succ, false, false); return true; }}} Try to advance reference (if we don’t succeed, someone else did or will). a

228 © Herlihy-Shavit Add public boolean add(T object) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(object); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true; }}}}

229 © Herlihy-Shavit Add public boolean add(T object) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(object); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true; }}}} Object already there.

230 © Herlihy-Shavit Add public boolean add(T object) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(object); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true; }}}} create new node

231 © Herlihy-Shavit Add public boolean add(T object) { boolean splice; while (true) { Window window = find(head, key); Node pred = window.pred, curr = window.curr; if (curr.key == key) { return false; } else { Node node = new Node(object); node.next = new AtomicMarkableRef(curr, false); if (pred.next.compareAndSet(curr, node, false, false)) {return true }}}} Install new node, else retry loop

232 © Herlihy-Shavit Wait-free Contains public boolean contains(T object) { boolean marked; int key = object.hashCode(); Node curr = this.head; while (curr.key < key) curr = curr.next; Node succ = curr.next.get(marked); return (curr.key == key && !marked[0]) }

233 © Herlihy-Shavit Wait-free Contains public boolean contains(T object) { boolean marked; int key = object.hashCode(); Node curr = this.head; while (curr.key < key) curr = curr.next; Node succ = curr.next.get(marked); return (curr.key == key && !marked[0]) } Only diff from lazy list is that we get and check marked

234 © Herlihy-Shavit Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }}

235 © Herlihy-Shavit Lock-free Find public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} If list changes while traversed, start over Lock-Free because we start over only if someone else makes progress

236 © Herlihy-Shavit public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find Start looking from head

237 © Herlihy-Shavit public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find Move down the list

238 © Herlihy-Shavit public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find Get ref to successor and current deleted bit

239 © Herlihy-Shavit public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find Try to remove deleted nodes in path…code details soon

240 © Herlihy-Shavit public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find If found curr key that is greater or equal, return pred and curr

241 © Herlihy-Shavit public Window find(Node head, int key) { Node pred = null, curr = null, succ = null; boolean[] marked = {false}; boolean snip; retry: while (true) { pred = head; curr = pred.next.getReference(); while (true) { succ = curr.next.get(marked); while (marked[0]) { … } if (curr.key >= key) return new Window(pred, curr); pred = curr; curr = succ; } }} Lock-free Find Otherwise advance “window” and loop again

242 © Herlihy-Shavit Lock-free Find retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } …

243 © Herlihy-Shavit Lock-free Find retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … Try to snip out node

244 © Herlihy-Shavit Lock-free Find retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … if predecessor’s next field changed must retry whole traversal

245 © Herlihy-Shavit Lock-free Find retry: while (true) { … while (marked[0]) { snip = pred.next.compareAndSet(curr, succ, false, false); if (!snip) continue retry; curr = succ; succ = curr.next.get(marked); } … Otherwise move on to check if next node deleted

246 © Herlihy-Shavit Summary: Summary: Lock- free Removal a a b c 0 e 1 c Logical Removal = Set Mark Bit Physical Removal CAS pointer Use CAS to verify pointer is correct Not enough!

247 © Herlihy-Shavit Summary: Summary: Lock-free Removal a a b c 0 e 1 c Logical Removal = Set Mark Bit Physical Removal CAS 0 d Problem: d not added to list… Must Prevent manipulation of removed node’s pointer Node added Before Physical Removal CAS

248 © Herlihy-Shavit Our Solution: Combine Bit and Pointer a a b c 0 e 1 c Logical Removal = Set Mark Bit Physical Removal CAS 0 d Mark-Bit and Pointer are CASed together Fail CAS: Node not added after logical Removal

249 © Herlihy-Shavit A Lock-free Algorithm a a b c 0 e 1 c 1. Add() and Remove() physically clean up (remove) marked nodes 2. Wait-free find() traverses both marked and removed nodes

250 © Herlihy-Shavit Performance On 16 node shared memory machine Benchmark throughput of Java List-based Set algs. Vary % of Contains() method Calls.

251 © Herlihy-Shavit High Contains Ratio Lock-free Lazy list Course Grained Fine Lock-coupling

252 © Herlihy-Shavit Low Contains Ratio

253 © Herlihy-Shavit As Contains Ratio Increases % Contains()

254 © Herlihy-Shavit Summary Coarse-grained locking Fine-grained locking Optimistic synchronization Lock-free synchronization

255 © Herlihy-Shavit “To Lock or Not to Lock” Locking vs. Non-blocking: Extremist views on both sides The answer: nobler to compromise, combine locking and non-blocking –Example: Lazy list combines blocking add() and remove() and a wait-free contains() –Blocking/non-blocking is a property of a method


Download ppt "Linked Lists: Locking, Lock-Free, and Beyond … The Art of Multiprocessor Programming Spring 2007."

Similar presentations


Ads by Google