Presentation is loading. Please wait.

Presentation is loading. Please wait.

Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit.

Similar presentations


Presentation on theme: "Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit."— Presentation transcript:

1 Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

2 Art of Multiprocessor Programming© Herlihy-Shavit The Five-Fold Path Coarse-grained locking Fine-grained locking Optimistic synchronization Lazy synchronization Lock-free synchronization

3 Art of Multiprocessor Programming© Herlihy-Shavit Another Fundamental Problem We told you about –Sets implemented using linked lists Next: queues Next: stacks

4 Art of Multiprocessor Programming© Herlihy-Shavit Queues & Stacks Both: pool of items Queue –enq() & deq() –First-in-first-out (FIFO) order Stack –push() & pop() –Last-in-first-out (LIFO) order

5 Art of Multiprocessor Programming© Herlihy-Shavit Bounded vs Unbounded Bounded –Fixed capacity –Good when resources an issue Unbounded –Holds any number of objects

6 Art of Multiprocessor Programming© Herlihy-Shavit Blocking vs Non-Blocking Problem cases: –Removing from empty pool –Adding to full (bounded) pool Blocking –Caller waits until state changes Non-Blocking –Method throws exception

7 Art of Multiprocessor Programming© Herlihy-Shavit This Lecture Bounded, Blocking, Lock-based Queue Unbounded, Non-Blocking, Lock-free Queue Monitors ABA problem Unbounded Non-Blocking Lock-free Stack Elimination-Backoff Stack

8 Art of Multiprocessor Programming© Herlihy-Shavit Queue: Concurrency enq(x) y=deq() enq() and deq() work at different ends of the object tailhead

9 Art of Multiprocessor Programming© Herlihy-Shavit Concurrency enq(x) Challenge: what if the queue is empty or full? y=deq() tail head

10 Art of Multiprocessor Programming© Herlihy-Shavit Bounded Queue Sentinel head tail

11 Art of Multiprocessor Programming© Herlihy-Shavit Bounded Queue head tail First actual item

12 Art of Multiprocessor Programming© Herlihy-Shavit Bounded Queue head tail Lock out other deq() calls deqLock

13 Art of Multiprocessor Programming© Herlihy-Shavit Bounded Queue head tail Lock out other enq() calls deqLock enqLock

14 Art of Multiprocessor Programming© Herlihy-Shavit Not Done Yet head tail deqLock enqLock Need to tell whether queue is full or empty

15 Art of Multiprocessor Programming© Herlihy-Shavit Not Done Yet head tail deqLock enqLock Permission to enqueue 8 items permits 8

16 Art of Multiprocessor Programming© Herlihy-Shavit Not Done Yet head tail deqLock enqLock Incremented by deq() Decremented by enq() permits 8

17 Art of Multiprocessor Programming© Herlihy-Shavit Enqueuer head tail deqLock enqLock permits 8 Lock enqLock

18 Art of Multiprocessor Programming© Herlihy-Shavit Enqueuer head tail deqLock enqLock permits 8 Read permits OK

19 Art of Multiprocessor Programming© Herlihy-Shavit Enqueuer head tail deqLock enqLock permits 8 No need to lock tail

20 Art of Multiprocessor Programming© Herlihy-Shavit Enqueuer head tail deqLock enqLock permits 8 Enqueue Node

21 Art of Multiprocessor Programming© Herlihy-Shavit Enqueuer head tail deqLock enqLock permits 8 7 getAndDecrement()

22 Art of Multiprocessor Programming© Herlihy-Shavit Enqueuer head tail deqLock enqLock permits 8 Release lock 7

23 Art of Multiprocessor Programming© Herlihy-Shavit Enqueuer head tail deqLock enqLock permits 7 If queue was empty, notify/signal waiting dequeuers

24 Art of Multiprocessor Programming© Herlihy-Shavit Unsuccesful Enqueuer head tail deqLock enqLock permits 0 Uh-oh Read permits

25 Art of Multiprocessor Programming© Herlihy-Shavit Dequeuer head tail deqLock enqLock permits 8 Lock deqLock

26 Art of Multiprocessor Programming© Herlihy-Shavit Dequeuer head tail deqLock enqLock permits 7 Read sentinel’s next field OK

27 Art of Multiprocessor Programming© Herlihy-Shavit Dequeuer head tail deqLock enqLock permits 7 Read value

28 Art of Multiprocessor Programming© Herlihy-Shavit Dequeuer head tail deqLock enqLock permits 7 Make first Node new sentinel

29 Art of Multiprocessor Programming© Herlihy-Shavit Dequeuer head tail deqLock enqLock permits 8 Increment permits

30 Art of Multiprocessor Programming© Herlihy-Shavit Dequeuer head tail deqLock enqLock permits 7 Release deqLock

31 Art of Multiprocessor Programming© Herlihy-Shavit Unsuccesful Dequeuer head tail deqLock enqLock permits 8 Read sentinel’s next field uh-oh

32 Art of Multiprocessor Programming© Herlihy-Shavit Bounded Queue public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); }

33 Art of Multiprocessor Programming© Herlihy-Shavit Bounded Queue public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); } Enq & deq locks

34 Art of Multiprocessor Programming© Herlihy-Shavit Digression: Monitor Locks Java Synchronized objects and Java ReentrantLocks are monitors Allow blocking on a condition rather than spinning Threads: –acquire and release lock –wait on a condition

35 Art of Multiprocessor Programming© Herlihy-Shavit public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock; } The Java Lock Interface Acquire lock

36 Art of Multiprocessor Programming© Herlihy-Shavit public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock; } The Java Lock Interface Release lock

37 Art of Multiprocessor Programming© Herlihy-Shavit public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock; } The Java Lock Interface Try for lock, but not too hard

38 Art of Multiprocessor Programming© Herlihy-Shavit public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock; } The Java Lock Interface Create condition to wait on

39 Art of Multiprocessor Programming© Herlihy-Shavit The Java Lock Interface public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock; } Guess what this method does?

40 Art of Multiprocessor Programming© Herlihy-Shavit Lock Conditions public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); }

41 Art of Multiprocessor Programming© Herlihy-Shavit public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); } Lock Conditions Release lock and wait on condition

42 Art of Multiprocessor Programming© Herlihy-Shavit public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); } Lock Conditions Wake up one waiting thread

43 Art of Multiprocessor Programming© Herlihy-Shavit public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); } Lock Conditions Wake up all waiting threads

44 Art of Multiprocessor Programming© Herlihy-Shavit Await Releases lock associated with q Sleeps (gives up processor) Awakens (resumes running) Reacquires lock & returns q.await()

45 Art of Multiprocessor Programming© Herlihy-Shavit Signal Awakens one waiting thread –Which will reacquire lock q.signal();

46 Art of Multiprocessor Programming© Herlihy-Shavit Signal All Awakens all waiting threads –Which will each reacquire lock q.signalAll();

47 Art of Multiprocessor Programming© Herlihy-Shavit A Monitor Lock Critical Section waiting room lock() unLock()

48 Art of Multiprocessor Programming© Herlihy-Shavit Unsuccessful Deq Critical Section waiting room lock() await() deq() Oh no, empty!

49 Art of Multiprocessor Programming© Herlihy-Shavit Another One Critical Section waiting room lock() await() deq() Oh no, empty!

50 Art of Multiprocessor Programming© Herlihy-Shavit Enqueuer to the Rescue Critical Section waiting room lock() signalAll() enq( ) unLock() Yawn!

51 Art of Multiprocessor Programming© Herlihy-Shavit Yawn! Monitor Signalling Critical Section waiting room Yawn! Awakened thread might still lose lock to outside contender…

52 Art of Multiprocessor Programming© Herlihy-Shavit Dequeurs Signalled Critical Section waiting room Found it Yawn!

53 Art of Multiprocessor Programming© Herlihy-Shavit Yawn! Dequeurs Signalled Critical Section waiting room Still empty!

54 Art of Multiprocessor Programming© Herlihy-Shavit Dollar Short + Day Late Critical Section waiting room

55 Art of Multiprocessor Programming© Herlihy-Shavit Lost Wake-Up Critical Section waiting room lock() signal () enq( ) unLock() Yawn!

56 Art of Multiprocessor Programming© Herlihy-Shavit Lost Wake-Up Critical Section waiting room lock() enq( ) unLock() Yawn!

57 Art of Multiprocessor Programming© Herlihy-Shavit Lost Wake-Up Critical Section waiting room Yawn!

58 Art of Multiprocessor Programming© Herlihy-Shavit Lost Wake-Up Critical Section waiting room Found it

59 Art of Multiprocessor Programming© Herlihy-Shavit What’s Wrong Here? Critical Section waiting room zzzz….!

60 Solution to Lost Wakeup Always use signalAll and notifyAll Not signal and notify Art of Multiprocessor Programming© Herlihy-Shavit

61 Art of Multiprocessor Programming© Herlihy-Shavit public class Queue { int head = 0, tail = 0; T[QSIZE] items; public synchronized T deq() { while (tail – head == 0) this.wait(); T result = items[head % QSIZE]; head++; this.notifyAll(); return result; } … }} Java Synchronized Methods

62 Art of Multiprocessor Programming© Herlihy-Shavit public class Queue { int head = 0, tail = 0; T[QSIZE] items; public synchronized T deq() { while (tail – head == 0) this.wait(); T result = items[head % QSIZE]; head++; this.notifyAll(); return result; } … }} Java Synchronized Methods Each object has an implicit lock with an implicit condition

63 Art of Multiprocessor Programming© Herlihy-Shavit public class Queue { int head = 0, tail = 0; T[QSIZE] items; public synchronized T deq() { while (tail – head == 0) this.wait(); T result = items[head % QSIZE]; head++; this.notifyAll(); return result; } … }} Java Synchronized Methods Lock on entry, unlock on return

64 Art of Multiprocessor Programming© Herlihy-Shavit public class Queue { int head = 0, tail = 0; T[QSIZE] items; public synchronized T deq() { while (tail – head == 0) this.wait(); T result = items[head % QSIZE]; head++; this.notifyAll(); return result; } … }} Java Synchronized Methods Wait on implicit condition

65 Art of Multiprocessor Programming© Herlihy-Shavit public class Queue { int head = 0, tail = 0; T[QSIZE] items; public synchronized T deq() { while (tail – head == 0) this.wait(); T result = items[head % QSIZE]; head++; this.notifyAll(); return result; } … }} Java Synchronized Methods Signal all threads waiting on condition

66 Art of Multiprocessor Programming© Herlihy-Shavit (Pop!) The Bounded Queue public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); }

67 Art of Multiprocessor Programming© Herlihy-Shavit Bounded Queue Fields public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); } Enq & deq locks

68 Art of Multiprocessor Programming© Herlihy-Shavit Bounded Queue Fields public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); } Enq lock’s associated condition

69 Art of Multiprocessor Programming© Herlihy-Shavit Bounded Queue Fields public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); } Num permits: 0 to capacity

70 Art of Multiprocessor Programming© Herlihy-Shavit Bounded Queue Fields public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger permits; Node head; Node tail; int capacity; enqLock = new ReentrantLock(); notFullCondition = enqLock.newCondition(); deqLock = new ReentrantLock(); notEmptyCondition = deqLock.newCondition(); } Head and Tail

71 Art of Multiprocessor Programming© Herlihy-Shavit Enq Method Part One public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } … }

72 Art of Multiprocessor Programming© Herlihy-Shavit public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } … } Enq Method Part One Lock and unlock enq lock

73 Art of Multiprocessor Programming© Herlihy-Shavit public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } … } Enq Method Part One If queue is full, patiently await further instructions …

74 Art of Multiprocessor Programming© Herlihy-Shavit public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } … } Be Afraid How do we know the permits field won’t change?

75 Art of Multiprocessor Programming© Herlihy-Shavit public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } … } Enq Method Part One Add new node

76 Art of Multiprocessor Programming© Herlihy-Shavit public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (permits.get() == 0) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (permits.getAndDecrement() == capacity) mustWakeDequeuers = true; } finally { enqLock.unlock(); } … } Enq Method Part One If queue was empty, wake frustrated dequeuers

77 Art of Multiprocessor Programming© Herlihy-Shavit Enq Method Part Deux public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); }

78 Art of Multiprocessor Programming© Herlihy-Shavit Enq Method Part Deux public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } Are there dequeuers to be signaled?

79 Art of Multiprocessor Programming© Herlihy-Shavit public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } Enq Method Part Deux Lock and unlock deq lock

80 Art of Multiprocessor Programming© Herlihy-Shavit public void enq(T x) { … if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } Enq Method Part Deux Signal dequeuers that queue no longer empty

81 Art of Multiprocessor Programming© Herlihy-Shavit The Enq() & Deq() Methods Share no locks –That’s good But do share an atomic counter –Accessed on every method call –That’s not so good Can we alleviate this bottleneck?

82 Art of Multiprocessor Programming© Herlihy-Shavit Split the Counter The enq() method –Decrements only –Cares only if value is zero The deq() method –Increments only –Cares only if value is capacity

83 Art of Multiprocessor Programming© Herlihy-Shavit Split Counter Enqueuer decrements enqSidePermits Dequeuer increments deqSidePermits When enqueuer runs out –Locks deqLock –Transfers permits Intermittent synchronization –Not with each method call –Need both locks! (careful …)

84 Art of Multiprocessor Programming© Herlihy-Shavit A Lock-Free Queue Sentinel head tail

85 Art of Multiprocessor Programming© Herlihy-Shavit Compare and Set CAS

86 Art of Multiprocessor Programming© Herlihy-Shavit Enqueue head tail Enq( )

87 Art of Multiprocessor Programming© Herlihy-Shavit Enqueue head tail

88 Art of Multiprocessor Programming© Herlihy-Shavit Logical Enqueue head tail CAS

89 Art of Multiprocessor Programming© Herlihy-Shavit Physical Enqueue head tail Enqueue Node CAS

90 Art of Multiprocessor Programming© Herlihy-Shavit Enqueue These two steps are not atomic The tail field refers to either –Actual last Node (good) –Penultimate Node (not so good) Be prepared!

91 Art of Multiprocessor Programming© Herlihy-Shavit Enqueue What do you do if you find –A trailing tail? Stop and help fix it –If tail node has non-null next field –CAS the queue’s tail field to tail.next As in the universal construction

92 Art of Multiprocessor Programming© Herlihy-Shavit When CASs Fail During logical enqueue –Abandon hope, restart –Still lock-free (why?) During physical enqueue –Ignore it (why?)

93 Art of Multiprocessor Programming© Herlihy-Shavit Dequeuer head tail Read value

94 Art of Multiprocessor Programming© Herlihy-Shavit Dequeuer head tail Make first Node new sentinel CAS

95 Art of Multiprocessor Programming© Herlihy-Shavit Memory Reuse? What do we do with nodes after we dequeue them? Java: let garbage collector deal? Suppose there is no GC, or we prefer not to use it?

96 Art of Multiprocessor Programming© Herlihy-Shavit Dequeuer head tail CAS Can recycle

97 Art of Multiprocessor Programming© Herlihy-Shavit Simple Solution Each thread has a free list of unused queue nodes Allocate node: pop from list Free node: push onto list Deal with underflow somehow …

98 Art of Multiprocessor Programming© Herlihy-Shavit Why Recycling is Hard Free pool headtail Want to redirect tail from grey to red zzz…

99 Art of Multiprocessor Programming© Herlihy-Shavit Both Nodes Reclaimed Free pool zzz headtail

100 Art of Multiprocessor Programming© Herlihy-Shavit One Node Recycled Free pool Yawn! headtail

101 Art of Multiprocessor Programming© Herlihy-Shavit Why Recycling is Hard Free pool CAS headtail OK, here I go!

102 Art of Multiprocessor Programming© Herlihy-Shavit Final State Free pool zOMG what went wrong? headtail Bad news

103 Art of Multiprocessor Programming© Herlihy-Shavit The Dreaded ABA Problem Head pointer has value A Thread reads value A headtail

104 Art of Multiprocessor Programming© Herlihy-Shavit Dreaded ABA continued zzz Head pointer has value B Node A freed headtail

105 Art of Multiprocessor Programming© Herlihy-Shavit Dreaded ABA continued Yawn! Head pointer has value A again Node A recycled & reinitialized headtail

106 Art of Multiprocessor Programming© Herlihy-Shavit Dreaded ABA continued CAS succeeds because pointer matches even though pointer’s meaning has changed CAS headtail

107 Art of Multiprocessor Programming© Herlihy-Shavit The Dreaded ABA Problem Is a result of CAS() semantics –I blame Sun, Intel, AMD, … Not with Load-Locked/Store- Conditional –Good for IBM?

108 Art of Multiprocessor Programming© Herlihy-Shavit Dreaded ABA – A Solution Tag each pointer with a counter Unique over lifetime of node Pointer size vs word size issues Overflow? –Don’t worry be happy? –Bounded tags? AtomicStampedReference class

109 Art of Multiprocessor Programming© Herlihy-Shavit Atomic Stamped Reference AtomicStampedReference class –Java.util.concurrent.atomic package address S Stamp Reference Can get reference and stamp atomically

110 Art of Multiprocessor Programming© Herlihy-Shavit Concurrent Stack Methods –push(x) –pop() Last-in, First-out (LIFO) order Lock-Free!

111 Art of Multiprocessor Programming© Herlihy-Shavit Empty Stack Top

112 Art of Multiprocessor Programming© Herlihy-Shavit Push Top

113 Art of Multiprocessor Programming© Herlihy-Shavit Push Top CAS

114 Art of Multiprocessor Programming© Herlihy-Shavit Push Top

115 Art of Multiprocessor Programming© Herlihy-Shavit Push Top

116 Art of Multiprocessor Programming© Herlihy-Shavit Push Top

117 Art of Multiprocessor Programming© Herlihy-Shavit Push Top CAS

118 Art of Multiprocessor Programming© Herlihy-Shavit Push Top

119 Art of Multiprocessor Programming© Herlihy-Shavit Pop Top

120 Art of Multiprocessor Programming© Herlihy-Shavit Pop Top CAS

121 Art of Multiprocessor Programming© Herlihy-Shavit Pop Top CAS mine!

122 Art of Multiprocessor Programming© Herlihy-Shavit Pop Top CAS

123 Art of Multiprocessor Programming© Herlihy-Shavit Pop Top

124 Art of Multiprocessor Programming© Herlihy-Shavit public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff(); }} Lock-free Stack

125 Art of Multiprocessor Programming© Herlihy-Shavit public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public Boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack tryPush attempts to push a node

126 Art of Multiprocessor Programming© Herlihy-Shavit public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Read top value

127 Art of Multiprocessor Programming© Herlihy-Shavit public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack current top will be new node’s successor

128 Art of Multiprocessor Programming© Herlihy-Shavit public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Try to swing top, return success or failure

129 Art of Multiprocessor Programming© Herlihy-Shavit public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Push calls tryPush

130 Art of Multiprocessor Programming© Herlihy-Shavit public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Create new node

131 Art of Multiprocessor Programming© Herlihy-Shavit public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack If tryPush() fails, back off before retrying Makes scheduling benevolent so Method is effectively wait- free

132 Art of Multiprocessor Programming© Herlihy-Shavit Lock-free Stack Good –No locking Bad –Without GC, fear ABA –Without backoff, huge contention at top –In any case, no parallelism

133 Art of Multiprocessor Programming© Herlihy-Shavit Big Question Are stacks inherently sequential? Reasons why –Every pop() call fights for top item Reasons why not –Stay tuned …

134 Art of Multiprocessor Programming© Herlihy-Shavit Elimination-Backoff Stack How to –“turn contention into parallelism” Replace familiar –exponential backoff With alternative –elimination-backoff

135 Art of Multiprocessor Programming© Herlihy-Shavit Observation Push( ) Pop() linearizable stack After an equal number of pushes and pops, stack stays the same Yes!

136 Art of Multiprocessor Programming© Herlihy-Shavit Idea: Elimination Array Push( ) Pop() stack Pick at random Pick at random Elimination Array

137 Art of Multiprocessor Programming© Herlihy-Shavit Push Collides With Pop Push( ) Pop() stack continue No need to access stack Yes!

138 Art of Multiprocessor Programming© Herlihy-Shavit No Collision Push( ) Pop() stack If no collision, access stack If pushes collide or pops collide access stack

139 Art of Multiprocessor Programming© Herlihy-Shavit Elimination-Backoff Stack Lock-free stack + elimination array Access Lock-free stack, –If uncontended, apply operation –if contended, back off to elimination array and attempt elimination

140 Art of Multiprocessor Programming© Herlihy-Shavit Elimination-Backoff Stack Push( ) Pop() Top CAS If CAS fails, back off

141 Art of Multiprocessor Programming© Herlihy-Shavit Dynamic Range and Delay Push( ) Pick random range and max time to wait for collision based on level of contention encountered

142 Art of Multiprocessor Programming© Herlihy-Shavit Linearizability Un-eliminated calls –linearized as before Eliminated calls: –linearize pop() immediately after matching push() Combination is a linearizable stack

143 Art of Multiprocessor Programming© Herlihy-Shavit Un-Eliminated Linearizability push(v 1 ) time linearizable push(v 1 ) pop(v 1 )

144 Art of Multiprocessor Programming© Herlihy-Shavit Eliminated Linearizability pop(v 2 )push(v 1 ) push(v 2 ) time push(v 2 ) linearizable pop(v 2 ) push(v 1 ) pop(v 1 ) Collision Point Red calls are eliminated

145 Art of Multiprocessor Programming© Herlihy-Shavit Backoff Has Dual Effect Elimination introduces parallelism Backoff onto array cuts contention on lock-free stack Elimination in array cuts down total number of threads ever accessing lock-free stack

146 Art of Multiprocessor Programming© Herlihy-Shavit public class EliminationArray { private static final int duration =...; private static final int timeUnit =...; Exchanger [] exchanger; public EliminationArray(int capacity) { exchanger = new Exchanger[capacity]; for (int i = 0; i < capacity; i++) exchanger[i] = new Exchanger (); … } … } Elimination Array

147 Art of Multiprocessor Programming© Herlihy-Shavit public class EliminationArray { private static final int duration =...; private static final int timeUnit =...; Exchanger [] exchanger; public EliminationArray(int capacity) { exchanger = new Exchanger[capacity]; for (int i = 0; i < capacity; i++) exchanger[i] = new Exchanger (); … } … } Elimination Array An array of exchangers

148 Art of Multiprocessor Programming© Herlihy-Shavit Lock-free Exchanger EMPTY

149 Art of Multiprocessor Programming© Herlihy-Shavit EMPTY Lock-free Exchanger CAS

150 Art of Multiprocessor Programming© Herlihy-Shavit WAITING Lock-free Exchanger

151 Art of Multiprocessor Programming© Herlihy-Shavit Lock-free Exchanger In search of partner … WAITING

152 Art of Multiprocessor Programming© Herlihy-Shavit WAITING Lock-free Exchanger Slot Still waiting … Try to exchange item and set state to BUSY CAS

153 Art of Multiprocessor Programming© Herlihy-Shavit BUSY Lock-free Exchanger Slot Partner showed up, take item and reset to EMPTY item stamp/state

154 Art of Multiprocessor Programming© Herlihy-Shavit EMPTYBUSY Lock-free Exchanger Slot item stamp/state Partner showed up, take item and reset to EMPTY

155 Art of Multiprocessor Programming© Herlihy-Shavit public class Exchanger { AtomicStampedReference slot = new AtomicStampedReference (null, 0); A Lock-Free Exchanger

156 Art of Multiprocessor Programming© Herlihy-Shavit public class Exchanger { AtomicStampedReference slot = new AtomicStampedReference (null, 0); A Lock-Free Exchanger Atomically modifiable reference + time stamp

157 Art of Multiprocessor Programming© Herlihy-Shavit Atomic Stamped Reference AtomicStampedReference class –Java.util.concurrent.atomic package In C or C++: address S Stamp Reference

158 Art of Multiprocessor Programming© Herlihy-Shavit Extracting Reference & Stamp Public T get(int[] stampHolder);

159 Art of Multiprocessor Programming© Herlihy-Shavit Extracting Reference & Stamp Public T get(int[] stampHolder); Returns reference to object of type T Returns stamp at array index 0!

160 Art of Multiprocessor Programming© Herlihy-Shavit Exchanger Status enum Status {EMPTY, WAITING, BUSY};

161 Art of Multiprocessor Programming© Herlihy-Shavit Exchanger Status enum Status {EMPTY, WAITING, BUSY}; Nothing yet

162 Art of Multiprocessor Programming© Herlihy-Shavit Exchange Status enum Status {EMPTY, WAITING, BUSY}; Nothing yet One thread is waiting for rendez-vous

163 Art of Multiprocessor Programming© Herlihy-Shavit Exchange Status enum Status {EMPTY, WAITING, BUSY}; Nothing yet One thread is waiting for rendez-vous Other threads busy with rendez-vous

164 Art of Multiprocessor Programming© Herlihy-Shavit public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {EMPTY}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: … // slot is free case WAITING: … // someone waiting for me case BUSY: … // others exchanging } The Exchange

165 Art of Multiprocessor Programming© Herlihy-Shavit public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {EMPTY}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: … // slot is free case WAITING: … // someone waiting for me case BUSY: … // others exchanging } The Exchange Item & timeout

166 Art of Multiprocessor Programming© Herlihy-Shavit public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {EMPTY}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: … // slot is free case WAITING: … // someone waiting for me case BUSY: … // others exchanging } The Exchange Array to hold timestamp

167 Art of Multiprocessor Programming© Herlihy-Shavit public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {0}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: // slot is free case WAITING: // someone waiting for me case BUSY: // others exchanging } }} The Exchange Loop until timeout

168 Art of Multiprocessor Programming© Herlihy-Shavit public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {0}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: // slot is free case WAITING: // someone waiting for me case BUSY: // others exchanging } }} The Exchange Get other’s item and timestamp

169 Art of Multiprocessor Programming© Herlihy-Shavit public T Exchange(T myItem, long nanos) throws TimeoutException { long timeBound = System.nanoTime() + nanos; int[] stampHolder = {0}; while (true) { if (System.nanoTime() > timeBound) throw new TimeoutException(); T herItem = slot.get(stampHolder); int stamp = stampHolder[0]; switch(stamp) { case EMPTY: … // slot is free case WAITING: … // someone waiting for me case BUSY: … // others exchanging } }} The Exchange Exchanger slot has three states

170 Art of Multiprocessor Programming© Herlihy-Shavit case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY

171 Art of Multiprocessor Programming© Herlihy-Shavit case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY Slot is free, try to insert myItem and change state to WAITING

172 Art of Multiprocessor Programming© Herlihy-Shavit case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY Loop while still time left to attempt exchange

173 Art of Multiprocessor Programming© Herlihy-Shavit case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, WAITING, BUSY)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY Get item and stamp in slot and check if state changed to BUSY

174 Art of Multiprocessor Programming© Herlihy-Shavit case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY If successful reset slot state to EMPTY

175 Art of Multiprocessor Programming© Herlihy-Shavit case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, WAITING, BUSY)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY and return item found in slot

176 Art of Multiprocessor Programming© Herlihy-Shavit case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY Otherwise we ran out of time, try to reset state to EMPTY, if successful time out

177 Art of Multiprocessor Programming© Herlihy-Shavit case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, WAITING, BUSY)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY If reset failed, someone showed up after all, take that item

178 Art of Multiprocessor Programming© Herlihy-Shavit case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY Set slot to EMPTY with new timestamp and return item found

179 Art of Multiprocessor Programming© Herlihy-Shavit case EMPTY: // slot is free if (slot.compareAndSet(herItem, myItem, EMPTY, WAITING)) { while (System.nanoTime() < timeBound){ herItem = slot.get(stampHolder); if (stampHolder[0] == BUSY) { slot.set(null, EMPTY); return herItem; }} if (slot.compareAndSet(myItem, null, WAITING, EMPTY)){throw new TimeoutException(); } else { herItem = slot.get(stampHolder); slot.set(null, EMPTY); return herItem; } } break; Exchanger State EMPTY If initial CAS failed then someone else changed state from EMPTY to WAITING so retry from start

180 Art of Multiprocessor Programming© Herlihy-Shavit case WAITING: // someone waiting for me if (slot.CAS(herItem, myItem, WAITING, BUSY)) return herItem; break; case BUSY: // others in middle of exchanging break; default: // impossible break; } States WAITING and BUSY

181 Art of Multiprocessor Programming© Herlihy-Shavit case WAITING: // someone waiting for me if (slot.CAS(herItem, myItem, WAITING, BUSY)) return herItem; break; case BUSY: // others in middle of exchanging break; default: // impossible break; } States WAITING and BUSY Someone is waiting for an exchange, so try to CAS in my item in & change state to BUSY

182 Art of Multiprocessor Programming© Herlihy-Shavit case WAITING: // someone waiting for me if (slot.CAS(herItem, myItem, WAITING, BUSY)) return herItem; break; case BUSY: // others in middle of exchanging break; default: // impossible break; } States WAITING and BUSY If successful, return that item. Otherwise another thread got it.

183 Art of Multiprocessor Programming© Herlihy-Shavit case WAITING: // someone waiting for me if (slot.CAS(herItem, myItem, WAITING, BUSY)) return herItem; break; case BUSY: // others in middle of exchanging break; default: // impossible break; } States WAITING and BUSY Other threads using slot, so start over.

184 Art of Multiprocessor Programming© Herlihy-Shavit The Exchanger Slot Exchanger is lock-free Because the only way an exchange can fail is if others repeatedly succeeded or no-one showed up The slot we need does not require symmetric exchange

185 Art of Multiprocessor Programming© Herlihy-Shavit public class EliminationArray { … public T visit(T value, int Range) throws TimeoutException { int slot = random.nextInt(Range); int nanodur = convertToNanos(duration, timeUnit)); return (exchanger[slot].exchange(value, nanodur) }} Elimination Array

186 Art of Multiprocessor Programming© Herlihy-Shavit public class EliminationArray { … public T visit(T value, int Range) throws TimeoutException { int slot = random.nextInt(Range); int nanodur = convertToNanos(duration, timeUnit)); return (exchanger[slot].exchange(value, nanodur) }} Elimination Array Visit elimination array with value and range

187 Art of Multiprocessor Programming© Herlihy-Shavit public class EliminationArray { … public T visit(T value, int Range) throws TimeoutException { int slot = random.nextInt(Range); int nanodur = convertToNanos(duration, timeUnit)); return (exchanger[slot].exchange(value, nanodur) }} Elimination Array Pick a random array entry

188 Art of Multiprocessor Programming© Herlihy-Shavit public class EliminationArray { … public T visit(T value, int Range) throws TimeoutException { int slot = random.nextInt(Range); int nanodur = convertToNanos(duration, timeUnit)); return (exchanger[slot].exchange(value, nanodur) }} Elimination Array Exchange value or time out

189 Art of Multiprocessor Programming© Herlihy-Shavit public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push

190 Art of Multiprocessor Programming© Herlihy-Shavit public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push First try to push

191 Art of Multiprocessor Programming© Herlihy-Shavit public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push If failed back-off to try to eliminate

192 Art of Multiprocessor Programming© Herlihy-Shavit public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push Value being pushed and range to try

193 Art of Multiprocessor Programming© Herlihy-Shavit public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push Only a pop inserts null value so elimination was successful

194 Art of Multiprocessor Programming© Herlihy-Shavit public void push(T value) {... while (true) { if (tryPush(node)) { return; } else try { T otherValue = eliminationArray.visit(value,policy.Range); if (otherValue == null) { return; } Elimination Stack Push Else retry push on lock-free stack

195 Art of Multiprocessor Programming© Herlihy-Shavit public T pop() {... while (true) { if (tryPop()) { return returnNode.value; } else try { T otherValue = eliminationArray.visit(null,policy.Range; if otherValue != null) { return otherValue; } }} Elimination Stack Pop

196 Art of Multiprocessor Programming© Herlihy-Shavit public T pop() {... while (true) { if (tryPop()) { return returnNode.value; } else try { T otherValue = eliminationArray.visit(null,policy.Range; if ( otherValue != null) { return otherValue; } }} Elimination Stack Pop Like push, non-null value pushed by other thread, so elimination succeeds

197 Art of Multiprocessor Programming© Herlihy-Shavit Summary We saw both lock-based and lock- free implementations of queues and stacks Not every data structure that looks sequential is sequential. –Linearizable stack not inherently sequential ABA is a real problem, fear it.

198 Art of Multiprocessor Programming© Herlihy-Shavit This work is licensed under a Creative Commons Attribution- ShareAlike 2.5 License.Creative Commons Attribution- ShareAlike 2.5 License You are free: –to Share — to copy, distribute and transmit the work –to Remix — to adapt the work Under the following conditions: –Attribution. You must attribute the work to “The Art of Multiprocessor Programming” (but not in any way that suggests that the authors endorse you or your use of the work). –Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to –http://creativecommons.org/licenses/by-sa/3.0/. Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights.


Download ppt "Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit."

Similar presentations


Ads by Google