1. XA transaction & integration Use case with OpenESB

2. Agenda Pymma Consulting: Services versioning with WSDL 2 XA for an ideal world XA and the real world XA and Integration OpenESB features Conclusions

3. THE IDEAL WORLD Pymma Consulting: Services versioning with WSDL 3

4. The ideal world Pymma Consulting: Services versioning with WSDL 4 Process Context N Partner 1 Context N Partner 2 Context N Partner 3 Before the process starts

5. The ideal world Pymma Consulting: Services versioning with WSDL 5 Process Context N+5 Context N+3 Partner 1 Context N+2 Partner 2 Context N +7 Partner 3 During the process

6. The ideal world Pymma Consulting: Services versioning with WSDL 6 Process Context N+8 Context N+5 Partner 1 Context N+6 Partner 2 Context N +9 Partner 3 An error occurs during the process

7. The ideal world Pymma Consulting: Services versioning with WSDL 7 Process Context N+8 Context N+5 Partner 1 Context N+6 Partner 2 Context N +9 Partner 3 Partners are informed about the error

8. The ideal world Pymma Consulting: Services versioning with WSDL 8 Process Context N Partner 1 Context N Partner 2 Context N Partner 3 Partners and process rollback to the original contexts

9. The ideal world Pymma Consulting: Services versioning with WSDL 9 Process Context N Partner 1 Context N Partner 2 Context N Partner 3 Using XA for transaction and recovery XA Commit Rollback XA

10. The ideal world XA is perfect for our ideal world We are all familiar with XA or two phases commit transaction – Mainframe, Database, Transaction manager, J2EEserver XA Transactions provide – Once and only once delivery /consistency XA (2 phases commit) has been the basis of 70’s 80’s and 90’s IT softwares Pymma Consulting: Services versioning with WSDL 10

11. THE REAL WORLD Pymma Consulting: Services versioning with WSDL 11

12. The ideal world XA is a concept for an care bear’s world The ideal world is difficult to implement XA needs special conditions for working well – Uniformity / Proximity Uniformity – Does not work in a non uniform environment – Not implemented with heterogeneous applications Proximity – XA works only when partners collocated – Being on the same network is not a collocation In the 00’s, Internet blasted XA and the ideal world Pymma Consulting: Services versioning with WSDL 12

13. Internet blasted the ideal world From stand alone to multi-partners Pymma Consulting: Services versioning with WSDL 13 Silo Application Module Collaborate Provider client Partner Client Provider Partner Client Provider

14. Internet blasted the ideal world From one to multiple technologies Pymma Consulting: Services versioning with WSDL 14 IBM Mainframe UnixUnix Mainframe Linux Windows Web Software Package External Partners FTP Http JMS NFS TCP/IP Java.Net C++ Cobol

15. Internet blasted the ideal world Web applications need more power than ever Pymma Consulting: Services versioning with WSDL 15 Up to 10k users Up to 100 M users

16. Internet blasted the ideal world Web applications need more power than ever Pymma Consulting: Services versioning with WSDL 16 ServerServer ServerServer ServerServer ServerServer ServerServer ServerServer ServerServer ServerServer ServerServer ServerServer ServerServer

17. The Real world In the 00’s – Many projects failed in distributed environment – Business & management worried about this fact Why IT could not provide what it did in the 70’s 80’ and 90’s ? High management said: – Difficulties to implement XA came from A lack of knowledge from IT people Bad technologies and platforms choices Moreover in the 00’s big editors lied – Saying their tools support distributed XA – Helped by famous analysts Let’s explain why XA was so difficult to implement Pymma Consulting: Services versioning with WSDL 17

18. The real world Pymma Consulting: Services versioning with WSDL 18 Client /Server Architecture 18 Centralised Architecture

19. The real world The network is not a simple extension – Latency creates partitioning – One group can work an not the other Pymma Consulting: Services versioning with WSDL 19 Latency Many architects thought that distributed architectures behave like centralized architectures.

20. The real world You create partitions – When you work with many platforms – When you use different technologies – When you create a geographical distribution Partition means: – A part of your system can work when one is stopped Pymma Consulting: Services versioning with WSDL 20 Partition

21. The real world and CAP theorem Pymma Consulting: Services versioning with WSDL 21 In 2002 the CAP theorem saved us – CAP stands for Consistency Availability and Partitioning – Set out by Eric Brewer Mathematically proved by S. Gilbert et N. Lynch (MIT) CAP is a mathematical answer to – The 00’s management and Editors Let’s see what it says

22. The real world and CAP theorem CAP Theorem (Consistency, Availability, Partition) – We need 3 properties needed for a network: consistency, availability and partitioning. At most, it is possible to get two of them. Pymma Consulting: Services versioning with WSDL 22 C A P Consistency + Availability Single site / Cluster database Two phase commit C A P Consistency + Partition Distributed database / Distributed looking Pessimistic looking C A P Availability + Partition Distributed cache/ DNS Optimistic looking

23. The real world and CAP theorem Pymma Consulting: Services versioning with WSDL 23 office

24. The real world and CAP theorem First Case I want consistency Pymma Consulting: Services versioning with WSDL 24 Office Brussels Office Berlin Office Nice Inform Inform Lock C A P

25. The real world and CAP theorem First Case I want consistency Pymma Consulting: Services versioning with WSDL 25 Office Brussels Office Berlin Office Nice Get Data Data not available C A P

26. The real world and CAP theorem First Case, we chose consistency Pymma Consulting: Services versioning with WSDL 26 Office Brussels Office Berlin Office Nice Inform Inform Unlock C A P Data are consistent but not always available

27. The real world and CAP theorem Second case I want availability Pymma Consulting: Services versioning with WSDL 27 Office Brussels Office Berlin Office Nice Get data C A P

28. The real world and CAP theorem Second case I want availability Pymma Consulting: Services versioning with WSDL 28 Office Brussels Office Berlin Office Nice process data C A P Get data process data

29. The real world and CAP theorem Second case I want availability Pymma Consulting: Services versioning with WSDL 29 Office Brussels Office Berlin Office Nice C A P Put data Data are available but not always consistence

30. The real world and CAP theorem CAP Theorem (Consequences) – In a partitioned environment We cannot get at the same time consistency and availability Mathematically demonstrated that XA is not available We must choose which property to promote – Ex : statistics data/ Historic data / transfer / Customer account Why is the CAP theorem is new for us? – Limited latency in the local area network – Database editor propaganda – Not teach in the universities yet – Many architects ignore CAP theorem existence – Similitude with the uncertainly principle of Heisenberg Pymma Consulting: Services versioning with WSDL 30

31. The real world and CAP theorem Concept difficult to admit: Impact on mentality and habit – Business people are use to design with the concept of transaction – B.As write specifications relying on XA transactions – Going out transactions seems a regression for the business – Most of architects are comfortably relying on transactions to provide consistency – They don’t know how to do without it – Some developers cannot live without “Commit and Rollback” Pymma Consulting: Services versioning with WSDL 31

32. The Real world (summary) In the 00’s IT world changed – Internet destroyed the ideal world It introduce biodiversity in IT systems – Many partners (some not XA) – Multiple technologies (some not XA) – Many protocols (some not XA) XA does not work in a non uniform environment XA doesn’t exist in the real world XA doesn’t exist in real life Pymma Consulting: Services versioning with WSDL 32

33. INTEGRATION AND XA Pymma Consulting: Services versioning with WSDL 33

34. Integration and XA Pymma Consulting: Services versioning with WSDL 34 All together Manual Validation Receive Transfer order by email Check rights Swift transfer Send email for confirmation Check account Banker validation Log transaction

35. Integration and XA Integration means – Multiple partners – Multiple Technologies – Multiple protocols Typically an heterogeneous environment – Impossible to apply XA For these environment could we provide: – Guarantee of delivery – Consistency Question: – Can integration stand with strong consistency and high guarantee of delivery? Pymma Consulting: Services versioning with WSDL 35

36. OPENESB FEATURES: CONSISTENCY & GUARANTEE OF DELIVERY Pymma Consulting: Services versioning with WSDL 36

37. OpenESB features OpenESB is an integration tool – Relying on JBI specifications (Java Business Integration) OpenESB proposes natively advanced features – High availability – Quality of services – Consistency We will review these features They are used by architects & designers to increase quality Pymma Consulting: Services versioning with WSDL 37

38. Pymma notation Pymma Consulting: Services versioning with WSDL 38 Binding Component Binding Component XA implementation Service without side effects Service without side effects XA Implementation Service with side effects Service with side effects XA Implementation

39. Pymma notation Pymma Consulting: Services versioning with WSDL 39 BPEL Catch Compensation The process is atomic The process is persistent Regular process When an error occurs Compensation part of the process N Redelivery N times

40. Integration and business processes Pymma Consulting: Services versioning with WSDL 40 Short processes Few partners Automatic tasks Emitter Receiver Integration process BPEL Few seconds max Automatic tasks Integration process

41. OpenESB Technical concepts Pymma Consulting: Services versioning with WSDL 41 Longer processes Many partners Automatic & Manual tasks Emitter Receiver Business process BPEL From seconds to weeks Automatic tasks Manual tasks Business process

42. Service acknowledgement Pymma Consulting: Services versioning with WSDL 42 partner JBI environment Acknowledgement Acknowledgment is required in a JBI environment Emitter Receiver Acknowledgement with BPEL ACK BPEL ACK

43. BPEL Atomic property Pymma Consulting: Services versioning with WSDL 43 Emitter Receiver ACK is sent just after the reception: Atomic = false ACK BPEL Receive Activity invoke

44. BPEL Atomic property Pymma Consulting: Services versioning with WSDL 44 Emitter Receiver No ACK is sent when a non catch error occurs in the process : atomic = true ACK BPEL Receive Activity invoke X X

45. Side effect and compensation Pymma Consulting: Services versioning with WSDL 45 Emitter Receiver process Simple process A B

46. Side effect and compensation Pymma Consulting: Services versioning with WSDL 46 Emitter Receiver process Simple process with side effect Side effect A B Catch

47. Side effect and compensation Pymma Consulting: Services versioning with WSDL 47 Emitter Receiver process Simple transactional process with side effect A B There is no Rollback

48. Side effect and compensation Pymma Consulting: Services versioning with WSDL 48 Compensation A A Side effects C: A Service A Compensation Increases side effects Suppresses or decreases side effects Service A

49. Side effect and compensation Pymma Consulting: Services versioning with WSDL 49 Insert in database Delete in database Compensation complete Write in audit file Compensation partial Put message in Topic Compensation Partial Add number in a total Subtract number in total Compensation complete

50. Side effect and compensation Pymma Consulting: Services versioning with WSDL 50 C C Service A C C Service B C C Service C C C Service D Service AService BService CService D Normal process Compensation process C C Service A C C Service B C C Service C C C Service D Service AService BService CService D

51. Side effect and compensation Pymma Consulting: Services versioning with WSDL 51 Save a record C: delete the record DB Insert into.... Audit service. V1 Delete from.... SQL Save a record C: delete the record Write in file File Audit service. V2 Delete from.... SQL

52. Side effect and compensation Pymma Consulting: Services versioning with WSDL 52 Save a record C: delete the record DB Insert into.... Audit service V1 Audit service V1 Delete from.... Save a record C: delete the record Write in file Audit service V2 Audit service V2 Delete in file File F -1 (x) F(x)

53. Qualities & Redelivery Extension Pymma Consulting: Services versioning with WSDL 53 Redelivery Extension – Max attempts N times – Wait time – On failure Delete / error / redirect / suspend BPEL N N N N N N

54. Qualities & Redelivery Extension Pymma Consulting: Services versioning with WSDL 54 BPEL Http Non reliable connection 3 Wait 2s External Service BPEL JMS MQ 2 Redirect Http Web Service 2 Error FTP FTP Server Suspend BPEL 2 Monitoring Try to fix the problem

55. Persistence Principle Pymma Consulting: Services versioning with WSDL 55 C C C C C C C C Side effect No transactional process After a failure Full transactional process Rollback After a failure

56. Qualities & Redelivery Extension Pymma Consulting: Services versioning with WSDL 56 C C C C C C C C BPEL internal context DB C C C C C C C C BPEL internal context DB

57. Qualities & Redelivery Extension Limits side effect Increases guaranty of delivery Decreases the performances – Powerful DB server can be used – Cache distributed instead DB Solutions can be found to use efficiently persistence Pymma Consulting: Services versioning with WSDL 57 C C C C C C C C BPEL internal context DB

58. High availability Pymma Consulting: Services versioning with WSDL 58 Machine 3 Machine 2 Machine 1 Domain Cluster Instance ESB Instance ESB Instance ESB App. Node agent Cluster

59. High Availability OpenESB has been designed for multi-instance – Native design – Don’t use containers cluster features It provides easily high Availability Architects uses this features – To increase Quality of services – Useful to secure processes and services Must be associated with good BP designs Pymma Consulting: Services versioning with WSDL 59

60. HOW TO USE OPENESB FEATURES Pymma Consulting: Services versioning with WSDL 60

61. Cleverness and pragmatism There are 3 delivery types – 0,n / 1,n / 1,1 delivery There is not system that can really provide 1,1 – XA system are close from it – OpenESB can provide 1,n in a non XA environment – Many native OpenESB features to achieve 1,n delivery Pragmatism – Do not try to reach 1,1 – increase business and management awareness about it Cleverness – Identify what is really important, exceptional, vital – Don’t focus on the exceptional – Don’t forget that human is the best to process exception Pymma Consulting: Services versioning with WSDL 61

62. Cleverness and pragmatism Cleverness start with BP design – Avoid stateful processes promote stateless Use hydration and dehydration – Avoid Multipart conversation – Plan entry features to avoid doubloon – Plan compensation messages at the design level Be pragmatic – Cancelation message is often better that transaction – Human is very smart to solve complex issue Recording vital data in a data is critical Process it in a simple way Process automatically the most common errors Human or external process to solve complex error Pymma Consulting: Services versioning with WSDL 62 80% -20%

63. Cleverness and pragmatism Spend time on happy ways and simple errors – Design and processing are straightforward – Simple error can be process easily QOS / Redelivery Compensation Atomicity – That’s architect and designer job Don’t spend time on exceptional – Don’t try to replace (or replay) XA – What happens if a meteorite fails on my computer – You will increase development and budget – increase maintenance complexity Pymma Consulting: Services versioning with WSDL 63

64. Cleverness and pragmatism Does this philosophy works – Google, Ebay, Amazon follows this philosophy – They cancelled XA for scalability or Consistency Pymma helps its customers to follow this way – Large to very large projects > many K men days – KBC Bank, Leroy Merlin, Kiabi, IFDS – They all need reliability and consistency – But scalability and efficiency for integration They found in OpenESB a solution We are sure you will find in OpenESB your solution Pymma Consulting: Services versioning with WSDL 64

65. QUESTIONS AND COMMENTS Pymma Consulting: Services versioning with WSDL 65