A Mile-High View of Concurrent Algorithms Hagit Attiya Technion

May 1, 2008 Concurrent COVA 2 A Simplistic View of Concurrent Systems A collection of processes Each a sequential thread of execution Communicating through shared data structures

May 1, 2008 Concurrent COVA 3 Alternative Routes for Developing Concurrent Algorithms Refinement: Implementing high-level ADT from lower-level ADTs –Safety conditions, liveness properties –Hierarchical Transactional: Support for running sequential applications concurrently –Safety conditions, liveness properties

May 1, 2008 Concurrent COVA 4 Abstract Data Types (ADT) Cover most concurrent applications –At least encapsulate their data needs –An object-oriented programming point of view Abstract representation of data & set of methods (operations) for accessing it –Signature –Specification data

May 1, 2008 Concurrent COVA 5 data Implementing High-Level ADT

May 1, 2008 Concurrent COVA 6 Implementing High-Level ADT data Using lower-level ADTs & procedures

May 1, 2008 Concurrent COVA 7 Lower-Level Operations High-level operations translate into primitives on base objects –Obvious: read, write (restrictions?) –Common: compare&swap (CAS) –LL/SC, Double-CAS (2CAS, DCAS), kCAS, … –Generic: read-modify-write (RMW), kRMW Low-level operations are often implemented from more primitive operations –A hierarchy of implementations

May 1, 2008 Concurrent COVA 8 Executing Operations P1P1 invocationresponse P2P2 P3P3

May 1, 2008 Concurrent COVA 9 Interleaving Operations Concurrent execution

May 1, 2008 Concurrent COVA 10 Interleaving Operations (External) behavior

May 1, 2008 Concurrent COVA 11 Interleaving Operations, or Not Sequential execution

May 1, 2008 Concurrent COVA 12 Interleaving Operations, or Not Sequential behavior: invocations & response alternate and match (on process & object) Sequential specification: All the legal sequential behaviors, satisfying the semantics of the ADT –E.g., for a (LIFO) stack: pop returns the last item pushed

May 1, 2008 Concurrent COVA 13 Correctness: Sequential consistency [Lamport, 1979] For every concurrent execution there is a sequential execution that –Contains the same operations –Is legal (obeys the sequential specification) –Preserves the order of operations by the same process

May 1, 2008 Concurrent COVA 14 Sequential Consistency: Examples push(4) pop():4push(7) Concurrent (LIFO) stack push(4) pop():4push(7) Last In First Out

May 1, 2008 Concurrent COVA 15 Sequential Consistency: Examples push(4) pop():7push(7) Concurrent (LIFO) stack Last In First Out

May 1, 2008 Concurrent COVA 16 Example 1: Multi-Writer Registers Add logical time (Lamport timestamps) to values Write(v) read TS 1,...,read TS n TS i = max TS j +1 write  v,TS i  Read only own value Read() read  v,TS i  return v Once in a while read TS 1,...,read TS n write max TS j to TS i ~[Attiya, Welch TOCS 1994] Using (multi-reader) single-writer registers Need to ensure writes are eventually visible

May 1, 2008 Concurrent COVA 17 Multi-Writer Registers: Proof Write(v,X) read TS 1,...,read TS n TS i = max TS j +1 write  v,TS i  Read(X) read  v,TS i  return v Once in a while read TS 1,...,read TS n write max TS j to TS i Create sequential execution: –Place writes in timestamp order –Insert reads after the appropriate write

May 1, 2008 Concurrent COVA 18 Multi-Writer Registers: Proof Create sequential execution: –Place writes in timestamp order –Insert reads after the appropriate write Legality is immediate Per-process order is preserved since a read returns a value (with timestamp) larger than the preceding write by the same process

May 1, 2008 Concurrent COVA 19 Sequential Consistency is not Composable enq(Q 1, X ) enq(Q 2, X)deq(Q 1, Y)enq(Q 2,Y)enq(Q 1,Y)deq(Q 2,X) The execution is not sequentially consistent

May 1, 2008 Concurrent COVA 20 Sequential Consistency is not Composable enq(Q 1, X)deq(Q 1, Y) enq(Q 1,Y) enq(Q 2, X) enq(Q 2,Y)deq(Q 2,X) The execution projected on each object is sequentially consistent Bad news for verification!

May 1, 2008 Concurrent COVA 21 Correctness: Linearizability [Herlihy & Wing, 1990] For every concurrent execution there is a sequential execution that –Contains the same operations –Is legal (obeys the specification of the ADTs) –Preserves the real-time order of non-overlapping operations Each operation appears to takes effect instantaneously at some point between its invocation and its response (atomicity)

May 1, 2008 Concurrent COVA 22 Linearizable Multi-Writer Registers Add logical time to values Write(v,X) read TS 1,...,read TS n TS i = max TS j +1 write  v,TS i  Read(X) read TS 1,...,read TS n return value with max TS Using (multi-reader) single-writer registers [Vitanyi & Awerbuch, 1987]

May 1, 2008 Concurrent COVA 23 Multi-writer registers: Linearization order Create linearization: –Place writes in timestamp order –Insert each read after the appropriate write Write(v,X) read TS 1,...,read TS n TS i = max TS j +1 write  v,TS i  Read(X) read TS 1,...,read TS n return value with max TS

May 1, 2008 Concurrent COVA 24 Multi-Writer Registers: Proof Create linearization: –Place writes in timestamp order –Insert each read after the appropriate write Legality is immediate Real-time order is preserved since a read returns a value (with timestamp) larger than all preceding operations

May 1, 2008 Concurrent COVA 25 Linearizability is Composable The whole system is linearizable  each object is linearizable Allows to implement and verify each object separately Good news for verification!

May 1, 2008 Concurrent COVA 26 Example 3: Atomic Snapshot m components Update a single component Scan all the components “at once” (atomically) Provides an instantaneous view of the whole memory update ok scan v 1,…,v m

May 1, 2008 Concurrent COVA 27 Atomic Snapshots: Algorithm Update(v,k) A[k] =  v,seq i, i  Scan() repeat read A[1],…,A[m] if equal return A[1,…,m] Linearize: Updates with their writes Scans inside the double collects double collect [Afek, Attiya, Dolev, Gafni, Merritt, Shavit, JACM 1993]

May 1, 2008 Concurrent COVA 28 Atomic Snapshot: Linearizability Double collect (read a set of values twice) If equal, there is no write between the collects –Assuming each write has a new value (seq#) Creates a “safe zone”, where the scan can be linearized read A[1],…,A[m] write A[j]

May 1, 2008 Concurrent COVA 29 Liveness Conditions (Eventual) Wait-free: every operation completes within a finite number of (its own) steps  no starvation for mutex Nonblocking: some operation completes within a finite number of (some other process) steps  deadlock-freedom for mutex Obstruction-free: an operation (eventually) running solo completes within a finite number of (its own) steps –Also called solo termination wait-free  nonblocking  obstruction-free

May 1, 2008 Concurrent COVA 30 Liveness Conditions (Bounded) Wait-free: every operation completes within a bounded number of (its own) steps  no starvation for mutex Nonblocking: some operation completes within a bounded number of (some other process) steps  deadlock-freedom for mutex Obstruction-free: an operation (eventually) running solo completes within a bounded number of (its own) steps –Also called solo termination Bounded wait-free  bounded nonblocking  bounded obstruction-free

May 1, 2008 Concurrent COVA 31 Wait-free Atomic Snapshot [Afek, Attiya, Dolev, Gafni, Merritt, Shavit, JACM 1993] Embed a scan within the Update. Update(v,k) V = scan A[k] =  v,seq i, i,V  Scan() repeat read A[1],…,A[m] if equal return A[1,…,m] else record diff if twice p j return V j Linearize: Updates with their writes Direct scans as before Borrowed scans with source direct scan borrowed scan

May 1, 2008 Concurrent COVA 32 Atomic Snapshot: Borrowed Scans Interference by process p j And another one…  p j does a scan inbeteween Linearizing with the borrowed scan is OK. write A[j] read A[j] …… …… embedded scan write A[j] read A[j] …… ……

May 1, 2008 Concurrent COVA 33 Alternative Routes for Developing Concurrent Algorithms Refinement: Implementing high-level ADT from lower-level ADTs –Safety conditions, liveness properties –Hierarchical Transactional: Support for running sequential applications concurrently –Safety conditions, liveness properties

May 1, 2008 Concurrent COVA 34 The Transactional Approach [Herlihy, Moss, ISCA 1993] Systematic approach for implementing concurrent data structures Program sequentially But run concurrently  Should appear to execute sequentially  No high-level signature / semantics –The sequential program is the specification

May 1, 2008 Concurrent COVA 35 Transactional Synchronization A transaction aggregates a sequence of resource accesses to be executed atomically –A sequence of atomic actions –Like in database systems A transaction ends either by committing –all its updates take effect or by aborting –no update is effective Read X Write X Read Z Read Y Read X Write X Read Z Read Y

May 1, 2008 Concurrent COVA 36 Safety: Serializability [Papadimitriou 79][Weikum, Vossen, 2002, Chapter 3] An analogue of sequential consistency Any interleaving of the transactions yields a result that could be achieved in a sequential execution of the same set of transactions (a serialization) –Just the committed transactions? (Aborted transactions have no effect.)  Final state serializability

May 1, 2008 Concurrent COVA 37 Safety: View Serializability [Yannakakis 1984] What about intermediate values read? –Could be “corrected” later –But still cause harm, e.g., division by 0 Any interleaving of the transactions has an equivalent sequential execution of the same transactions –Where all reads return the same value  Makes no sense in the context of implementing a high-level ADT –Where the internals are not exposed

May 1, 2008 Concurrent COVA 38 Safety: Strict Serializability [Papadimitriou 79][Bernstein, Shipman & Wong, 1979] The serialization must preserve the real-time order of (non-overlapping) transactions  An analogue of linearizability  Called ordered serializabililty in W&V  Strict view serializability

May 1, 2008 Concurrent COVA 39 Opacity [Guerraoui & Kapalka, PPoPP 08] Essentially, strict view serializability –Applied to all transactions (not only committed or completed ones) –Generalized to arbitrary object types (not just reads and writes)

May 1, 2008 Concurrent COVA 40 Liveness Properties As in high-level implementations But restricted to successful completions (commit) E.g., wait-freedom  every transaction eventually commits, obsruction-freedom  every transaction (eventually) running solo terminates

May 1, 2008 Concurrent COVA 41 Nonblocking Obstruction-free A Shift in Terminology < 2003≥ 2003 Lock-free Nonblocking Wait-free Lock-free Wait-free

May 1, 2008 Concurrent COVA 42 Where’s the Confusion? [Herlihy & Wing, TOPLAS 1990] [Herlihy, TOPLAS 1991] [Herlihy, Luchangco, Moir, ICDCS 2003]

May 1, 2008 Concurrent COVA 43 The Road Ahead Concurrent algorithms pose a great challenge It is “easy” to write –Correct algorithms and let efficiency take care of itself –Efficient algorithms and let correctness take care of itself But very hard to write correct & efficient algorithms Systematically…