1. 1 WS Technologies III Transactions Roberto Bruni Dipartimento di Informatica Università di Pisa Models and Languages for Coordination and Orchestration IMT- Institutions Markets Technologies - Alti Studi Lucca

2. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 2 Models and Languages for Coordination and Orchestration Contents Orchestration and Choreography WSFL Transactions and Compensations BPEL4WS

3. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 3 Models and Languages for Coordination and Orchestration Motivation I A top-ranked global management issue in IT: "connecting to customer, suppliers and partners electronically" WS Orchestration / Choreography is about providing an open, standard-based approach for connecting WS together to create high-level business process executable, abstract, collaborative, reusable reducing complexity, time-to-market, costs increasing efficiency, scalability, adaptability, accuracy

4. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 4 Models and Languages for Coordination and Orchestration Motivation II Orchestrated WS that are long-running must also manage exceptions how the system will respond to faults (when building business process, nearly 80% of time is spent in exception management!) transactional integrity to reach mutually agreed outcomes ACID-based is not ok for long-running transactions compensating transactions are more appropriate Not dealt with in WSFL!

5. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 5 Models and Languages for Coordination and Orchestration Transactions and All the Rest We shall survey classic aspects of transactions commitment protocols (postponed after Easter) compensations WS challenges

6. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 6 Models and Languages for Coordination and Orchestration Concurrency Control and Recovery in DataBase Systems Concurrency control is the activity of coordinating the actions of processes that operate in parallel, access shared data, and therefore potentially interfere with each other Recovery is the activity of ensuring that Sw and Hw failures do not corrupt persistent data Transaction is a sequence of actions on a shared database The goal is to ensure that transactions execute atomically (no interference)

7. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 7 Models and Languages for Coordination and Orchestration DataBase Systems State Set of named data items with values Main operations Read(x) – query – returns the value of the data item x Write(x,v) – update – sets the data item x to value v We abstract from the granularity of data items Words in main memory Pages of a disk Records in a file Fields of a record

8. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 8 Models and Languages for Coordination and Orchestration Execution Operations should be executed sequentially, but some operations can be executed concurrently The final effect must be the same of some sequential execution 1.Read(x) 2.Write(x,1) | Read(y) 3.Write(y,0) The order is inessential

9. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 9 Models and Languages for Coordination and Orchestration Transaction Primitives Start begin a new transaction assign a unique transaction identifier Commit successfully terminate the transaction its effects are made permanent Abort invoke the abnormal termination of the transaction its effects are obliterated can be imposed by failures

10. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 10 Models and Languages for Coordination and Orchestration Terminology Unique entry / exit points a transaction is defined by a Start operation, followed by a possibly concurrent execution of (Read / Write) operations followed by either a Commit or Abort Committed transactions Aborted transactions Active transactions started but not yet committed (or aborted) Uncommitted transactions active or aborted

11. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 11 Models and Languages for Coordination and Orchestration Database Transactions Sequences of operations / actions which preserve database consistency ACID properties: Atomicity Consistency Isolation Durability Transaction’s changes to the states are ATOMIC: either all happen or none happen

12. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 12 Models and Languages for Coordination and Orchestration Database Transactions Sequences of operations / actions which preserve database consistency ACID properties: Atomicity Consistency Isolation Durability A transaction is a correct transformation of the state: its actions, taken as a group, do not violate any integrity constraint associated with the states

13. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 13 Models and Languages for Coordination and Orchestration Database Transactions Sequences of operations / actions which preserve database consistency ACID properties: Atomicity Consistency Isolation Durability Transactions can be executed concurrently, but it appears to each transaction T that others are executed either before T or after T

14. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 14 Models and Languages for Coordination and Orchestration Database Transactions Sequences of operations / actions which preserve database consistency ACID properties: Atomicity Consistency Isolation Durability Once a transaction successfully commits, its changes to the states survive failures

15. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 15 Models and Languages for Coordination and Orchestration Database Transactions Sequences of operations / actions which preserve database consistency ACID properties: Atomicity Consistency Isolation Durability Interest on serializability any T 1 |…|T n can be seen as T i1 ;…;T in

16. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 16 Models and Languages for Coordination and Orchestration Recoverability When T aborts, its effects are wiped out Effects on data (updates) undo restore the original value as if T never executed Effects on each T’ which read values updated by T (dependencies) cascading aborts each T’ must be aborted but remember that committed transactions cannot be aborted delaying commits can be necessary to ensure recoverability

17. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 17 Models and Languages for Coordination and Orchestration Recoverability: Example T1: Start 1 … Write 1 (x,2) … Abort 1 T2: Start 2 … Read 2 (x) … Write 2 (y,3) … Commit 2 T1|T2: … Write 1 (x,2) … Read 2 (x) … Write 2 (y,3) … Commit 2 … Abort 1 1. When T1 aborts, Write 1 (x,2) must be undone 2. Undoing Write 1 (x,2) can compromise T2 that read from x 3. But T2 did already commit… it should have been delayed!

18. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 18 Models and Languages for Coordination and Orchestration Avoiding Cascading Aborts Cascading aborts are unpleasant, but often necessary to ensure recoverability Require bookkeeping of dependencies Uncontrollably many transactions might be forced to abort Not necessary if every transaction reads values written by committed transactions Delay Read i (x) until all transactions that issued Write k (x,v) are either aborted or committed

19. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 19 Models and Languages for Coordination and Orchestration Strict Execution When undoing Write(x,v) which value should be restored? the initial value? but other transactions can have issued Write(x,v’) Before image the value of x just before the operation to be undone Later overwritten values should not be updated Delay Write i (x,v) until all transactions that issued Write k (x,w) are either aborted or committed

20. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 20 Models and Languages for Coordination and Orchestration Why Strict Execution: Example … Write 1 (x,1) Write 1 (y,3) Write 2 (y,1) Commit 1 Read 2 (x) Abort 2 When T2 aborts, the value of y should be 3 (the before image) … Write 1 (x,2) Write 2 (x,3) Abort 1 Suppose initially x=1. When T1 aborts, the value of x should be 3 not 1 … Write 1 (x,2) Write 2 (x,3) Abort 1 Abort 2 The before image of T2 is not meaningful after T1 aborts

21. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 21 Models and Languages for Coordination and Orchestration Non-Recoverable Interactions It is not always possible to reverse effects e.g. input / output statements Deferred output approach output messages of T only after T commits not always feasible (user input might depend on output messages)

22. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 22 Models and Languages for Coordination and Orchestration More on Serializability When two transactions execute concurrently, their operations are interleaved Even if they commit, interference can cause incorrect results Lost updates Inconsistent retrieval Avoid interleaving at all correct but inefficient Restrict to serializable executions

23. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 23 Models and Languages for Coordination and Orchestration Lost Updates: Example T_Deposit(a,n): Start t:=Read(a) t:=t+n Write(a,t) Commit T_Deposit(a,10) | T_Deposit(a,20) : Start 1 Read 1 (a) // returns v Start 2 Read 2 (a) // returns the same v Write 1 (a,v+10) Write 2 (a,v+20) Commit 2 Commit 1 At the end the account is increased by 20 (not by 30 as in any serial execution)

24. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 24 Models and Languages for Coordination and Orchestration Inconsistent Retrieval: Example T_Transfer(£,$,n): Start s:=Read(£) s:=s-n Write(£,s) t:=Read($) t:=t+n*rate[£,$] Write($,t) Commit T_Transfer(a,b,5)|T_Total(a,b): Start 1 Read 1 (a) // returns v Start 2 Read 2 (a) // returns the same v Write 1 (a,v-5) Read 1 (b) // returns w Write 1 (b,w+5*r) Read 2 (b) // returns w+5*r Commit 2 Commit 1 The output does not correspond to the total amount T_Total(£,$): Start s:=Read(£) t:=Read($) t:=t+s*rate[£,$] output(t) Commit

25. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 25 Models and Languages for Coordination and Orchestration DataBase System Model I Transaction Manager (TM) receives requests for transactions and operations forwards them to the scheduler Scheduler controls the relative order of execution Recovery Manager (RM) responsible for commits and aborts Distributed Architecture sites connected by a network

26. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 26 Models and Languages for Coordination and Orchestration DataBase System Model II TM Sched RM Network … DB TM Sched RM DB TM Sched RM DB TM Sched RM DB

27. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 27 Models and Languages for Coordination and Orchestration Some ACID Variations I Flat Transactions Traditional model, strictly meets ACID properties Not adequate to model complex situations A task’s abort causes all activities to be rolled back Flat with Savepoints Introduces programmable savepoints Rollback command restore savepoints without aborting Transactions can be rolled back individually Chained transactions Add savepoints via the chain operation Commits and start a new transaction with same locks

28. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 28 Models and Languages for Coordination and Orchestration Some ACID Variations II Nested Transactions Hierarchical tree of transactions Top-level is ACID Other nodes are ACI (subtransactions) If a parent aborts all its children abort (even if committed) If a subtransaction aborts its parent can still commit A subtransaction commit is seen outside only when its parent commits (protected nested transactions) The structure can exploit parallelism within transactions Localize failures to subtransactions

29. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 29 Models and Languages for Coordination and Orchestration Some ACID Variations III Multi-level Transactions Nested transaction that allow pre-commits to export partial results to other transactions at the same level perfect compensations to abort after pre-commit Abstraction layers Operations at level n are implemented using operations of the lower level immediately below no shortcuts

30. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 30 Models and Languages for Coordination and Orchestration Some ACID Variations IV Distributed Transactions Flat but executed in a distributed environment Partitioned in sub-transactions on network’s nodes Decomposition reflects neither the hierarchical structure of the application, nor its functional structure Decomposition depends on data distribution over the nodes Decisions made by a sub-transaction affect the entire transaction Commit protocols are needed All these models are traditional ones see e.g. [Gray, Reuter 1993]

31. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 31 Models and Languages for Coordination and Orchestration Some Non ACID Flavors I Long-Lived Transactions (LLT) Execution takes a considerable amount of time hours, days, weeks, … Not possible to block all short transactions Open Nested Transactions [Gray 1981] Compensations are not perfect Relaxed multi-level No abstraction layers Shortcuts are allowed

32. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 32 Models and Languages for Coordination and Orchestration Some Non ACID Flavors II Sagas [Garcia-Molina, Salem 1987] Special case of open-nested Only two levels of nesting LLT broken in a non-atomic sequence of ACID sub-transactions T 1 …T n Each T i has a (non-perfect) compensation C i Partial execution of the sequence are undesirable If it cannot be completed, then must be compensated Executed as either T 1 …T n or T 1 …T k C k …C 1 Partial effects are visible to other transactions no notification / compensation to siblings

33. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 33 Models and Languages for Coordination and Orchestration Some Non ACID Flavors III Flexible [Elmagarmid et al. 1990] Same task implemented by different (functionally equivalent) sub-transactions Programmable failure / success dependencies T1: Rent a car from AVIZ T2: Rent a car from HERTS if T1 fail Pre-committed actions and compensations Mixed (with or without compensations) transactions coexist within a single global transaction Incorporates time for flexible scheduling

34. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 34 Models and Languages for Coordination and Orchestration Some Non ACID Flavors IV Split and Join Transactions [Pu et al. 1988] Open-ended activities Long-running, not foreseeable concurrent interactions Computer-Aided Design/Computer-Aided Manufacturing Editing and Sw development Split command Divide a running transaction in two independent ones Can export results, release/share resources Join command Merges two running transactions into one All resources become shared Results are visible only after the commit of the joint transaction

35. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 35 Models and Languages for Coordination and Orchestration Some Non ACID Flavors V Multi-Transactions To be used in distributed objects systems with heterogeneous databases Complex transactions by composing building blocks (as needed by applications) Mixture of flat, nested, open-nested are possible Countermeasure, Retriable, Timed, …

36. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 36 Models and Languages for Coordination and Orchestration Transactions in GC? ACID would generate too many aborts Local consistency vs global consistency Asynchronous commit Heterogeneous policies Several entry / exit points Messages in place of shared data Dynamic topology of the network Mobility For an overview: "A framework for analyzing mobile transaction models" A. Coratella, M. Felder, R. Hirsch, E. Rodriguez (Journal of DataBase Management 12(13), 2001)

37. Roberto Bruni @ IMT Lucca 23 March 2005 Institutions Markets Technologies IMT 37 Models and Languages for Coordination and Orchestration Some References Concurrency control and recovery in database systems (Addison-Wesley 1987) P. Bernstein, N. Goodman, V. Hadzilacos Transaction processing: concepts and techniques (Morgan Kaufmann 1993) J. Gray, A. Reuter Sagas (Proc. SIGMod’87, ACM, pp. 249- 259) H. Garcia-Molina, K. Salem