Lecture 6-2 : Concurrent Queues and Stacks Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

pool Data Structure similar to Set –Does not necessarily provide contains() method –Allows the same item to appear more than once –get() and set() Art of Multiprocessor Programming© Herlihy-Shavit public interface Pool { void put(T item); T get(); }

Art of Multiprocessor Programming© Herlihy-Shavit Queues & Stacks Both: pool of items Queue –enq() & deq() –First-in-first-out (FIFO) order Stack –push() & pop() –Last-in-first-out (LIFO) order

Art of Multiprocessor Programming© Herlihy-Shavit Bounded vs Unbounded Bounded –Fixed capacity –Good when resources an issue Unbounded –Holds any number of objects

Art of Multiprocessor Programming© Herlihy-Shavit Blocking vs Non-Blocking Problem cases: –Removing from empty pool –Adding to full (bounded) pool Blocking –Caller waits until state changes Non-Blocking –Method throws exception

Art of Multiprocessor Programming© Herlihy-Shavit Queue: Concurrency enq(x) y=deq() enq() and deq() work at different ends of the object tailhead

Art of Multiprocessor Programming© Herlihy-Shavit Concurrency enq(x) Challenge: what if the queue is empty or full? y=deq() tail head

Art of Multiprocessor Programming© Herlihy-Shavit A Bounded Lock-Based Queue public class BoundedQueue { ReentrantLock enqLock, deqLock; Condition notEmptyCondition, notFullCondition; AtomicInteger size; Node head; Node tail; int capacity; public BoundedQueue(int capacity) { this.capacity = capacity; this.head = new Node(null); this.tail = head; this.size = new AtomicInteger(0); this.enqLock = new ReentrantLock(); this.notFullCondition = enqLock.newCondition(); this.deqLock = new ReentrantLock(); this.notEmptyCondition = deqLock.newCondition(); } protected class Node { public T value; public Node next; public Node(T x) { value = x; next = null; }

9 public void enq(T x) { boolean mustWakeDequeuers = false; enqLock.lock(); try { while (size.get() == capacity) notFullCondition.await(); Node e = new Node(x); tail.next = e; tail = e; if (size.getAndIncrement() == 0) { mustWakeDequeuers = true; } } finally { enqLock.unlock(); } if (mustWakeDequeuers) { deqLock.lock(); try { notEmptyCondition.signalAll(); } finally { deqLock.unlock(); } public T deq() { T result; boolean mustWakeEnqueuers = false; deqLock.lock(); try { while (size.get() == 0) { notEmptyCondition.await(); result = head.next.value; head = head.next; if (size.getAndDecrement() == capacity) { mustWakeEnqueuers = true; } } finally { deqLock.unlock(); } if (mustWakeEnqueuers) { enqLock.lock(); try { notFullCondition.signalAll(); } finally { enqLock.unlock(); } return result; }

lock enqLock/deqLock –At most one enqueuer/dequeuer at a time can manipulate the queue’s fields Two locks –Enqueuer does not lock out dequeuer –vice versa Association –enqLock associated with notFullCondition –deqLock associated with notEmptyCondition Art of Multiprocessor Programming© Herlihy-Shavit

enqueue 1.Acquires enqLock 2.Reads the size field 3.If full, enqueuer must wait until dequeuer makes room 4.enqueuer waits on notFullCondition field, releasing enqLock temporarily, and blocking until that condition is signaled. 5.Each time the thread awakens, it checks whether there is a room, and if not, goes back to sleep 6.Insert new item into tail 7.Release enqLock 8.If queue was empty, notify/signal waiting dequeuers Art of Multiprocessor Programming© Herlihy-Shavit

dequeue 1.Acquires deqLock 2.Reads the size field 3.If empty, dequeuer must wait until item is enqueued 4.dequeuer waits on notEmptyCondition field, releasing deqLock temporarily, and blocking until that condition is signaled. 5.Each time the thread awakens, it checks whether item was enqueued, and if not, goes back to sleep 6.Assigne the value of head’s next node to “result” and reset head to head’s next node 7.Release deqLock 8.If queue was full, notify/signal waiting enqueuers 9.Return “result” Art of Multiprocessor Programming© Herlihy-Shavit

Art of Multiprocessor Programming13 Bounded Queue Sentinel head tail

Art of Multiprocessor Programming14 Bounded Queue head tail First actual item

Art of Multiprocessor Programming15 Bounded Queue head tail Lock out other deq() calls deqLock

Art of Multiprocessor Programming16 Bounded Queue head tail Lock out other enq() calls deqLock enqLock

Art of Multiprocessor Programming17 Not Done Yet head tail deqLock enqLock Need to tell whether queue is full or empty

Art of Multiprocessor Programming18 Not Done Yet head tail deqLock enqLock Max size is 8 items size 1

Art of Multiprocessor Programming19 Not Done Yet head tail deqLock enqLock Incremented by enq() Decremented by deq() size 1

Art of Multiprocessor Programming20 Enqueuer head tail deqLock enqLock size 1 Lock enqLock

Art of Multiprocessor Programming21 Enqueuer head tail deqLock enqLock size 1 Read size OK

Art of Multiprocessor Programming22 Enqueuer head tail deqLock enqLock size 1 No need to lock tail

Art of Multiprocessor Programming23 Enqueuer head tail deqLock enqLock size 1 Enqueue Node

Art of Multiprocessor Programming24 Enqueuer head tail deqLock enqLock size 1 2 getAndincrement()

Art of Multiprocessor Programming25 Enqueuer head tail deqLock enqLock size 8 Release lock 2

Art of Multiprocessor Programming26 Enqueuer head tail deqLock enqLock size 2 If queue was empty, notify waiting dequeuers

Art of Multiprocessor Programming27 Unsuccesful Enqueuer head tail deqLock enqLock size 8 Uh- oh Read size …

Art of Multiprocessor Programming28 Dequeuer head tail deqLock enqLock size 2 Lock deqLock

Art of Multiprocessor Programming29 Dequeuer head tail deqLock enqLock size 2 Read sentinel’s next field OK

Art of Multiprocessor Programming30 Dequeuer head tail deqLock enqLock size 2 Read value

Art of Multiprocessor Programming31 Dequeuer head tail deqLock enqLock size 2 Make first Node new sentinel

Art of Multiprocessor Programming32 Dequeuer head tail deqLock enqLock size 1 Decrement size

Art of Multiprocessor Programming33 Dequeuer head tail deqLock enqLock size 1 Release deqLock

Art of Multiprocessor Programming© Herlihy-Shavit Monitor Locks Java ReentrantLocks are monitors Allow blocking on a condition rather than spinning Threads: –acquire and release lock –wait on a condition

Art of Multiprocessor Programming© Herlihy-Shavit public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long time, TimeUnit unit); Condition newCondition(); void unlock; } The Java Lock Interface Create condition to wait on

Art of Multiprocessor Programming© Herlihy-Shavit Lock Conditions public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); }

Art of Multiprocessor Programming© Herlihy-Shavit public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); } Lock Conditions Release lock and wait on condition

Art of Multiprocessor Programming© Herlihy-Shavit public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); } Lock Conditions Wake up one waiting thread

Art of Multiprocessor Programming© Herlihy-Shavit public interface Condition { void await(); boolean await(long time, TimeUnit unit); … void signal(); void signalAll(); } Lock Conditions Wake up all waiting threads

Art of Multiprocessor Programming© Herlihy-Shavit Await Releases lock associated with q Sleeps (gives up processor) Awakens (resumes running) Reacquires lock & returns q.await()

Art of Multiprocessor Programming© Herlihy-Shavit Signal Awakens one waiting thread –Which will reacquire lock q.signal();

Art of Multiprocessor Programming© Herlihy-Shavit Signal All Awakens all waiting threads –Which will each reacquire lock q.signalAll();

Unbounded Lock-Free Queue (Nonblocking) Unbounded –No need to count the number of items Lock-free –Use AtomicReference An object reference that may be updated atomically. –boolean compareAndSet(V expect, V update) Atomically sets the value to the given updated value if the current value == the expected value. Nonblocking –No need to provide conditions on which to wait 43

Art of Multiprocessor Programming44 A Lock-Free Queue Sentinel head tail

Art of Multiprocessor Programming45 Enqueue head tail Enq( )

Art of Multiprocessor Programming46 Enqueue head tail

Art of Multiprocessor Programming47 Logical Enqueue head tail CAS

Art of Multiprocessor Programming48 Physical Enqueue head tail CAS

Art of Multiprocessor Programming49 Enqueue These two steps are not atomic The tail field refers to either –Actual last Node (good) –Penultimate Node (not so good) Be prepared!

Art of Multiprocessor Programming50 Enqueue What do you do if you find –A trailing tail? Stop and help fix it –If tail node has non-null next field –CAS the queue’s tail field to tail.next As in the universal construction

Art of Multiprocessor Programming51 When CASs Fail During logical enqueue –Abandon hope, restart –Still lock-free (why?) During physical enqueue –Ignore it (why?)

Art of Multiprocessor Programming52 Dequeuer head tail Read value

Art of Multiprocessor Programming53 Dequeuer head tail Make first Node new sentinel CAS

Art of Multiprocessor Programming© Herlihy-Shavit Unbounded Lock-Free Queue (Nonblocking) public class LockFreeQueue { private AtomicReference head; private AtomicReference tail; public LockFreeQueue() { this.head = new AtomicReference (null); this.tail = new AtomicReference (null); } public class Node { public T value; public AtomicReference next; public Node(T value) { this.value = value; this.next = new AtomicReference (null); }

55 public void enq(T item) { Node node = new Node(item); while (true) { Node last = tail.get(); Node next = last.next.get(); if (last == tail.get()) { if (next == null) { if (last.next.compareAndSet(next, node)) { tail.compareAndSet(last, node); return; } } else { tail.compareAndSet(last, next); } public T deq() throws EmptyException { while (true) { Node first = head.get(); Node last = tail.get(); Node next = first.next.get(); if (first == head.get()) { if (first == last) { if (next = null) { throw new EmptyException(); } tail.compareAndSet(last, next); } else { T value = next.value; if (head.compareAndSet(first,next)) return value; } Unbounded Lock-Free Queue (Nonblocking)

Art of Multiprocessor Programming56 Concurrent Stack Methods –push(x) –pop() Last-in, First-out (LIFO) order Lock-Free!

Art of Multiprocessor Programming57 Empty Stack Top

Art of Multiprocessor Programming58 Push Top

Art of Multiprocessor Programming59 Push Top CAS

Art of Multiprocessor Programming60 Push Top

Art of Multiprocessor Programming61 Push Top

Art of Multiprocessor Programming62 Push Top

Art of Multiprocessor Programming63 Push Top CAS

Art of Multiprocessor Programming64 Push Top

Art of Multiprocessor Programming65 Pop Top

Art of Multiprocessor Programming66 Pop Top CAS

Art of Multiprocessor Programming67 Pop Top CAS mine !

Art of Multiprocessor Programming68 Pop Top CAS

Art of Multiprocessor Programming69 Pop Top

Art of Multiprocessor Programming70 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff(); }} Lock-free Stack

Art of Multiprocessor Programming71 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public Boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack tryPush attempts to push a node

Art of Multiprocessor Programming72 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Read top value

Art of Multiprocessor Programming73 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack current top will be new node’s successor

Art of Multiprocessor Programming74 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Try to swing top, return success or failure

Art of Multiprocessor Programming75 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Push calls tryPush

Art of Multiprocessor Programming76 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack Create new node

Art of Multiprocessor Programming77 public class LockFreeStack { private AtomicReference top = new AtomicReference(null); public boolean tryPush(Node node){ Node oldTop = top.get(); node.next = oldTop; return(top.compareAndSet(oldTop, node)) } public void push(T value) { Node node = new Node(value); while (true) { if (tryPush(node)) { return; } else backoff.backoff() }} Lock-free Stack If tryPush() fails, back off before retrying

78 protected boolean tryPush(Node node) { Node oldTop = top.get(); node.next = oldTop; return (top.compareAndSet(oldTop, node)); } public void push( T value ) { Node node = new Node( value ); while (true) { if (tryPush(node)) { return; } else { backoff.backoff( ); } } protected Node tryPop( ) throws EmptyException { Node oldTop = top.get(); if ( oldTop == null ) { throw new EmptyException( ); } Node newTop = oldTop.next; if ( top.compareAndSet( oldTop, newTop ) ) { return oldTop; } else { return null; } } public T pop() throws EmptyException { while (true) { Node returnNode = tryPop( ); if ( returnNode != null ) { return returnNode.value; } else { backoff.backoff( ); } } Unbounded Lock-Free Stack

Art of Multiprocessor Programming79 Lock-free Stack Good –No locking Bad –Without GC, fear ABA –Without backoff, huge contention at top –In any case, no parallelism

Art of Multiprocessor Programming80 Question Are stacks inherently sequential? Reasons why –Every pop() call fights for top item Reasons why not –Think about it!