1. A Mile-High View of Concurrent Algorithms Hagit Attiya Technion

2. May 1, 2008 Concurrent algorithms @ COVA 2 A Simplistic View of Concurrent Systems A collection of processes Each a sequential thread of execution Communicating through shared data structures

3. May 1, 2008 Concurrent algorithms @ COVA 3 Alternative Routes for Developing Concurrent Algorithms Refinement: Implementing high-level ADT from lower-level ADTs –Safety conditions, liveness properties –Hierarchical Transactional: Support for running sequential applications concurrently –Safety conditions, liveness properties

4. May 1, 2008 Concurrent algorithms @ COVA 4 Abstract Data Types (ADT) Cover most concurrent applications –At least encapsulate their data needs –An object-oriented programming point of view Abstract representation of data & set of methods (operations) for accessing it –Signature –Specification data

5. May 1, 2008 Concurrent algorithms @ COVA 5 data Implementing High-Level ADT

6. May 1, 2008 Concurrent algorithms @ COVA 6 Implementing High-Level ADT data ------------------ ------------------- ------------------ ---------------- --------------- ------------------ ------------------- ------------------ ------------------- ------------------ ---------------- --------------- ------------------ ------------------- Using lower-level ADTs & procedures

7. May 1, 2008 Concurrent algorithms @ COVA 7 Lower-Level Operations High-level operations translate into primitives on base objects –Obvious: read, write (restrictions?) –Common: compare&swap (CAS) –LL/SC, Double-CAS (2CAS, DCAS), kCAS, … –Generic: read-modify-write (RMW), kRMW Low-level operations are often implemented from more primitive operations –A hierarchy of implementations

8. May 1, 2008 Concurrent algorithms @ COVA 8 Executing Operations P1P1 invocationresponse P2P2 P3P3

9. May 1, 2008 Concurrent algorithms @ COVA 9 Interleaving Operations Concurrent execution

10. May 1, 2008 Concurrent algorithms @ COVA 10 Interleaving Operations (External) behavior

11. May 1, 2008 Concurrent algorithms @ COVA 11 Interleaving Operations, or Not Sequential execution

12. May 1, 2008 Concurrent algorithms @ COVA 12 Interleaving Operations, or Not Sequential behavior: invocations & response alternate and match (on process & object) Sequential specification: All the legal sequential behaviors, satisfying the semantics of the ADT –E.g., for a (LIFO) stack: pop returns the last item pushed

13. May 1, 2008 Concurrent algorithms @ COVA 13 Correctness: Sequential consistency [Lamport, 1979] For every concurrent execution there is a sequential execution that –Contains the same operations –Is legal (obeys the sequential specification) –Preserves the order of operations by the same process

14. May 1, 2008 Concurrent algorithms @ COVA 14 Sequential Consistency: Examples push(4) pop():4push(7) Concurrent (LIFO) stack push(4) pop():4push(7) Last In First Out

15. May 1, 2008 Concurrent algorithms @ COVA 15 Sequential Consistency: Examples push(4) pop():7push(7) Concurrent (LIFO) stack Last In First Out

16. May 1, 2008 Concurrent algorithms @ COVA 16 Example 1: Multi-Writer Registers Add logical time (Lamport timestamps) to values Write(v) read TS 1,...,read TS n TS i = max TS j +1 write  v,TS i  Read only own value Read() read  v,TS i  return v Once in a while read TS 1,...,read TS n write max TS j to TS i ~[Attiya, Welch TOCS 1994] Using (multi-reader) single-writer registers Need to ensure writes are eventually visible

17. May 1, 2008 Concurrent algorithms @ COVA 17 Multi-Writer Registers: Proof Write(v,X) read TS 1,...,read TS n TS i = max TS j +1 write  v,TS i  Read(X) read  v,TS i  return v Once in a while read TS 1,...,read TS n write max TS j to TS i Create sequential execution: –Place writes in timestamp order –Insert reads after the appropriate write

18. May 1, 2008 Concurrent algorithms @ COVA 18 Multi-Writer Registers: Proof Create sequential execution: –Place writes in timestamp order –Insert reads after the appropriate write Legality is immediate Per-process order is preserved since a read returns a value (with timestamp) larger than the preceding write by the same process

19. May 1, 2008 Concurrent algorithms @ COVA 19 Sequential Consistency is not Composable enq(Q 1, X ) enq(Q 2, X)deq(Q 1, Y)enq(Q 2,Y)enq(Q 1,Y)deq(Q 2,X) The execution is not sequentially consistent

20. May 1, 2008 Concurrent algorithms @ COVA 20 Sequential Consistency is not Composable enq(Q 1, X)deq(Q 1, Y) enq(Q 1,Y) enq(Q 2, X) enq(Q 2,Y)deq(Q 2,X) The execution projected on each object is sequentially consistent Bad news for verification!

21. May 1, 2008 Concurrent algorithms @ COVA 21 Correctness: Linearizability [Herlihy & Wing, 1990] For every concurrent execution there is a sequential execution that –Contains the same operations –Is legal (obeys the specification of the ADTs) –Preserves the real-time order of non-overlapping operations Each operation appears to takes effect instantaneously at some point between its invocation and its response (atomicity)

22. May 1, 2008 Concurrent algorithms @ COVA 22 Linearizable Multi-Writer Registers Add logical time to values Write(v,X) read TS 1,...,read TS n TS i = max TS j +1 write  v,TS i  Read(X) read TS 1,...,read TS n return value with max TS Using (multi-reader) single-writer registers [Vitanyi & Awerbuch, 1987]

23. May 1, 2008 Concurrent algorithms @ COVA 23 Multi-writer registers: Linearization order Create linearization: –Place writes in timestamp order –Insert each read after the appropriate write Write(v,X) read TS 1,...,read TS n TS i = max TS j +1 write  v,TS i  Read(X) read TS 1,...,read TS n return value with max TS

24. May 1, 2008 Concurrent algorithms @ COVA 24 Multi-Writer Registers: Proof Create linearization: –Place writes in timestamp order –Insert each read after the appropriate write Legality is immediate Real-time order is preserved since a read returns a value (with timestamp) larger than all preceding operations

25. May 1, 2008 Concurrent algorithms @ COVA 25 Linearizability is Composable The whole system is linearizable  each object is linearizable Allows to implement and verify each object separately Good news for verification!

26. May 1, 2008 Concurrent algorithms @ COVA 26 Example 3: Atomic Snapshot m components Update a single component Scan all the components “at once” (atomically) Provides an instantaneous view of the whole memory update ok scan v 1,…,v m

27. May 1, 2008 Concurrent algorithms @ COVA 27 Atomic Snapshots: Algorithm Update(v,k) A[k] =  v,seq i, i  Scan() repeat read A[1],…,A[m] if equal return A[1,…,m] Linearize: Updates with their writes Scans inside the double collects double collect [Afek, Attiya, Dolev, Gafni, Merritt, Shavit, JACM 1993]

28. May 1, 2008 Concurrent algorithms @ COVA 28 Atomic Snapshot: Linearizability Double collect (read a set of values twice) If equal, there is no write between the collects –Assuming each write has a new value (seq#) Creates a “safe zone”, where the scan can be linearized read A[1],…,A[m] write A[j]

29. May 1, 2008 Concurrent algorithms @ COVA 29 Liveness Conditions (Eventual) Wait-free: every operation completes within a finite number of (its own) steps  no starvation for mutex Nonblocking: some operation completes within a finite number of (some other process) steps  deadlock-freedom for mutex Obstruction-free: an operation (eventually) running solo completes within a finite number of (its own) steps –Also called solo termination wait-free  nonblocking  obstruction-free

30. May 1, 2008 Concurrent algorithms @ COVA 30 Liveness Conditions (Bounded) Wait-free: every operation completes within a bounded number of (its own) steps  no starvation for mutex Nonblocking: some operation completes within a bounded number of (some other process) steps  deadlock-freedom for mutex Obstruction-free: an operation (eventually) running solo completes within a bounded number of (its own) steps –Also called solo termination Bounded wait-free  bounded nonblocking  bounded obstruction-free

31. May 1, 2008 Concurrent algorithms @ COVA 31 Wait-free Atomic Snapshot [Afek, Attiya, Dolev, Gafni, Merritt, Shavit, JACM 1993] Embed a scan within the Update. Update(v,k) V = scan A[k] =  v,seq i, i,V  Scan() repeat read A[1],…,A[m] if equal return A[1,…,m] else record diff if twice p j return V j Linearize: Updates with their writes Direct scans as before Borrowed scans with source direct scan borrowed scan

32. May 1, 2008 Concurrent algorithms @ COVA 32 Atomic Snapshot: Borrowed Scans Interference by process p j And another one…  p j does a scan inbeteween Linearizing with the borrowed scan is OK. write A[j] read A[j] …… …… embedded scan write A[j] read A[j] …… ……

33. May 1, 2008 Concurrent algorithms @ COVA 33 Alternative Routes for Developing Concurrent Algorithms Refinement: Implementing high-level ADT from lower-level ADTs –Safety conditions, liveness properties –Hierarchical Transactional: Support for running sequential applications concurrently –Safety conditions, liveness properties

34. May 1, 2008 Concurrent algorithms @ COVA 34 The Transactional Approach [Herlihy, Moss, ISCA 1993] Systematic approach for implementing concurrent data structures Program sequentially But run concurrently  Should appear to execute sequentially  No high-level signature / semantics –The sequential program is the specification

35. May 1, 2008 Concurrent algorithms @ COVA 35 Transactional Synchronization A transaction aggregates a sequence of resource accesses to be executed atomically –A sequence of atomic actions –Like in database systems A transaction ends either by committing –all its updates take effect or by aborting –no update is effective Read X Write X Read Z Read Y Read X Write X Read Z Read Y

36. May 1, 2008 Concurrent algorithms @ COVA 36 Safety: Serializability [Papadimitriou 79][Weikum, Vossen, 2002, Chapter 3] An analogue of sequential consistency Any interleaving of the transactions yields a result that could be achieved in a sequential execution of the same set of transactions (a serialization) –Just the committed transactions? (Aborted transactions have no effect.)  Final state serializability

37. May 1, 2008 Concurrent algorithms @ COVA 37 Safety: View Serializability [Yannakakis 1984] What about intermediate values read? –Could be “corrected” later –But still cause harm, e.g., division by 0 Any interleaving of the transactions has an equivalent sequential execution of the same transactions –Where all reads return the same value  Makes no sense in the context of implementing a high-level ADT –Where the internals are not exposed

38. May 1, 2008 Concurrent algorithms @ COVA 38 Safety: Strict Serializability [Papadimitriou 79][Bernstein, Shipman & Wong, 1979] The serialization must preserve the real-time order of (non-overlapping) transactions  An analogue of linearizability  Called ordered serializabililty in W&V  Strict view serializability

39. May 1, 2008 Concurrent algorithms @ COVA 39 Opacity [Guerraoui & Kapalka, PPoPP 08] Essentially, strict view serializability –Applied to all transactions (not only committed or completed ones) –Generalized to arbitrary object types (not just reads and writes)

40. May 1, 2008 Concurrent algorithms @ COVA 40 Liveness Properties As in high-level implementations But restricted to successful completions (commit) E.g., wait-freedom  every transaction eventually commits, obsruction-freedom  every transaction (eventually) running solo terminates

41. May 1, 2008 Concurrent algorithms @ COVA 41 Nonblocking Obstruction-free A Shift in Terminology < 2003≥ 2003 Lock-free Nonblocking Wait-free Lock-free Wait-free

42. May 1, 2008 Concurrent algorithms @ COVA 42 Where’s the Confusion? [Herlihy & Wing, TOPLAS 1990] [Herlihy, TOPLAS 1991] [Herlihy, Luchangco, Moir, ICDCS 2003]

43. May 1, 2008 Concurrent algorithms @ COVA 43 The Road Ahead Concurrent algorithms pose a great challenge It is “easy” to write –Correct algorithms and let efficiency take care of itself –Efficient algorithms and let correctness take care of itself But very hard to write correct & efficient algorithms Systematically…