1. Gal Milman Based on Chapter 10 (Concurrent Queues and the ABA Problem) in The Art of Multiprocessor Programming by Herlihy and Shavit Seminar 2 (236802) in Advanced Topics in Concurrent Programming Winter 15/16 Instructor: Erez Petrank Concurrent Queues

2. background: Pool Provides Put, Get methods. The same item can appear more than once. Often acts as producer-consumer buffer.

3. Provides Put, Get methods. The same item can appear more than once. Often acts as producer-consumer buffer. Pool Properties: Capacity: bounded / unbounded Methods: total / partial / synchronous Fairness: queue (FIFO) / stack (LIFO) /…

4. Queue DequeueEnqueue Tail Head (Get)(Put) next

5. Q u e u e We will review: Blocking Queue Lock-Free Queue o ABA Problem Synchronous Dual Queue

6. Blocking Queue Initialization: tail = head = Node(value=null) capacity = input_capacity size = AtomicInteger(0) enqLock, deqLock notFullCondition, notEmptyCondition tail head next tail head next value next value Dummy node

7. public void addNode(T x) { Node newNode = new Node(x); tail.next = newNode; tail = newNode; } tail head tail x newNode Blocking Queue - enq head 1 2

8. public T removeNodeAndGetValue() { result = head.next.value; head = head.next; return result; } Blocking Queue - deq head tail x head tail x

9. public void enq(T x) { addNode(x); size.getAndIncrement() } Blocking Queue - enq enqLock.lock(); while (size.get() == capacity) notFullCondition.await(); enqLock.unlock(); boolean mustWakeDequeuers = false; if ( == 0) mustWakeDequeuers = true; if (mustWakeDequeuers) { deqLock.lock(); notEmptyCondition.signalAll(); deqLock.unlock(); }

10. public void enq(T x) { addNode(x); size.getAndIncrement() } Blocking Queue - enq enqLock.lock(); while (size.get() == capacity) notFullCondition.await(); enqLock.unlock(); boolean mustWakeDequeuers = false; if ( == 0) mustWakeDequeuers = true; if (mustWakeDequeuers) { deqLock.lock(); notEmptyCondition.signalAll(); deqLock.unlock(); } deq is symmetrical. Why should we lock before signaling? deqenq deqlock.lock() while (queue is empty) notEmptyCondition.await() addNode() notEmptyCondition.signalAll() Why should we lock before signaling? To avoid the lost wakeup problem.

11. Unbounded Blocking Queue Unbounded => size and capacity are unnecessary Total => conditions are unnecessary Initialization: tail = head = Node(value=null) enqLock, deqLock tail head next

12. Unbounded Blocking Queue - enq, deq public void enq(T x) { addNode(x); } enqLock.lock(); enqLock.unlock(); public T deq() throws EmptyException { if (head.next == null) throw new EmptyException(); T result = removeNodeAndGetValue(x); return result; } deqLock.lock(); deqLock.unlock();

13. Abstract vs. Concrete The queue’s actual (abstract) head and tail are not necessarily the items referenced by head and tail. The actual head is the successor of the node referenced by head. The actual tail is the last item reachable from the head. Unbounded Blocking Queue

14. Q u e u e We will review: Blocking Queue Lock-Free Queue o ABA Problem Synchronous Dual Queue progress performance

15. Blocking Algorithms Cons Performance on a multi-core processor Using several locks is exposed to error conditions like deadlock Contention instead of progress: A blocked thread does not make any progress. If the thread that holds the lock gets stuck - the threads that try to acquire the lock get stuck. A data structure protected by a lock cannot safely be accessed in an interrupt handler, as the preempted thread may be the one holding the lock. Sources: https://en.wikipedia.org/wiki/Non-blocking_algorithm, https://en.wikipedia.org/wiki/Lock_(computer_science)

16. Non-Blocking Algorithms Non-blocking – failure or suspension of any thread cannot cause failure or suspension of another thread. Lock-free – There is guaranteed system-wide progress. / Some thread finishes in a finite number of steps. Wait-free – There is guaranteed per-thread progress. / Every thread finishes in a finite number of steps. Source: https://en.wikipedia.org/wiki/Non-blocking_algorithm

17. Lock-Free Algorithms In the absence of locks: Use atomic read-modify-write hardware primitives, like CAS But what about an action that is composed of several atomic primitive calls? o It should mark the data structure, so other threads may finish the action. "lock" "unlock"

18. Initialization: Unbounded ⇒ size and capacity are unnecessary Lock-free ⇒ locks and conditions are unnecessary tail = head = Node(value=null) * head, tail and Node.next are AtomicReference tail head next Lock-Free Queue

19. while (true) { if (next == null) { if ( ) { } } else { tail.compareAndSet(last, next); } } Lock-Free Queue - enq public void enq(T x) { Node node = new Node(x); Node last = tail.get(); Node next = last.next.get(); last.next.compareAndSet(next, node) tail.compareAndSet(last, node); return; }

20. Lock-Free Queue - deq while (true) { if ( ) tail.compareAndSet(last, next); } public T deq() throws EmptyException { Node first = head.get(); Node last = tail.get(); Node next = first.next.get(); if (first != last) { T value = next.value; head.compareAndSet(first, next) return value; } else { if (next == null) throw new EmptyException(); } }

21. ABA Problem Consider our lock-free queue But stop relying on Java’s garbage collector Each thread will maintain a private free list Now we encounter the ABA problem

22. head tail x a b head tail y b head tail b a free list (2) a tail a head b free list (2) tail a head b enq(y) deq 1 2 free list (2)

23. ABA Solution: A Stamp AtomicStampedReference encapsulates: a reference to Node an integer stamp AtomicReference AtomicStampedReference stamp is incremented in each compareAndSet. replaced by

24. Q u e u e We will review: Blocking Queue Lock-Free Queue o ABA Problem Synchronous Dual Queue

25. reservationfulfilment

26. head tail y x head tail head NotEmpty Condition -deq1 -deq2 -… Blocking Queue States head tail y x Synchronous Dual Queue States head tail null tail head enq reservations deq reservations In the dual queue - enq and deq are symmetrical. Both check queue’s state to figure out if they should be a reserver or a fulfiller. fair vs.

27. Synchronous Dual Queue Reserver adds an item to the queue, Other threads might finish the addition then spins on the item Fulfiller changes the item rendezvous To end the rendezvous - the item is removed from the queue Other threads might execute the removal lock- free lock- free