A Distributed Framework for Reliable and Efficient Service Choreographies Young Yoon, Chunyang Ye and Hans-Arno Jacobsen

What is service choreography? Autonomous and distributed collaboration amongst independently developed services. 2 Young YoonWWW'11

Service choreography Example: Credit card fraud detection process 3 Auditing partner sends fraud warnings to Accounting partner G0G0 G1G1 Customer service issues refund to Customer Young YoonWWW'11

4 Auditing partner sends fraud warnings to Accounting partner G0G0 G1G1 Customer service issues refund to Customer Decompose Send fraud warning messages Receive fraud warning message Issue refund Receive refund message Auditing service Accounting serviceCRM serviceClient service Young Yoon WWW'11 L0L0 L0L0 L0L0 L0L0

Semantic conflict Behavior that does not conform to the collaboration specification 5 Young YoonWWW'11

6 Send fraud warning messages Receive fraud warning message Issue refund Receive refund message Auditing service Accounting serviceCRM serviceClient service “ Someone used my credit card to purchase train ticket!” Issue refund Refund Customer account CRM service behaved arbitrarily and caused semantic conflict. Young YoonWWW'11 Travel agency

7 Auditing partner sends fraud warnings to Accounting partner G0G0 G1G1 Customer service issues refund to Customer GxGx Accounting partner orders customer service to issue refund The cause of the semantic conflict The missing synchronization Young YoonWWW'11

8 Another example: Internal operation in a car company HQ  ”clear out”  Dealer Operations manager  “decrease production rate”  change to Factory Marketing  “Sales trend update”  Factory Dealers ask Coordinator to pick G 1 or G 2 Young YoonWWW'11

9 Goal 1: Prevent semantic conflict. Enrich safety constraints Goal 2: Build a distributed framework Govern the enforcement of safety constraints Goal 3: Make the framework be efficient Don’t let the safety constraints hamper the performance of the collaboration Young YoonWWW'11

10 Goal 1: Prevent semantic conflict Goal 2: Build a distributed framework Goal 3: Make the framework be efficient Young YoonWWW'11

11 Enrich safety constraints during the decomposition. Young YoonWWW'11

r1  r2r1  r2 m1m1 r3  r4r3  r4 m2m2 r1  r5r1  r5 m3m3 r1  r3r1  r3 m4m4 G1G1 G2G2 G4G4 G3G3 Young Yoon12WWW'11 r x : collaborating partner x r1  r2r1  r2 m1m1 : r 1 sends m 1 r 2

r1  r2r1  r2 m1m1 r3  r4r3  r4 m2m2 r1  r5r1  r5 m3m3 r1  r3r1  r3 m4m4 r2  r3r2  r3 c1c1 r2  r1r2  r1 c2c2 r4  r1r4  r1 c3c3 r5  r1r5  r1 c4c4 G1G1 G2G2 G4G4 G3G3 r1  r2r1  r2 m1m1 r3  r4r3  r4 m2m2 r1  r5r1  r5 m3m3 r1  r3r1  r3 m4m4 G1G1 G2G2 G4G4 G3G3 Young Yoon13WWW'11

 (G 1 ) : projection of G 1 x(m 1 ): send m 1 x(m 1 ): receive m 1 r1  r2r1  r2 m1m1 r3  r4r3  r4 m2m2 r1  r5r1  r5 m3m3 r1  r3r1  r3 m4m4 r2  r3r2  r3 c1c1 r2  r1r2  r1 c2c2 r4  r1r4  r1 c3c3 r5  r1r5  r1 c4c4 G1G1 G2G2 G3G3 G3G3 Young Yoon14WWW'11

x(m 1 ) (G1)(G1) r1r1 r1  r2r1  r2 m1m1 r3  r4r3  r4 m2m2 r1  r5r1  r5 m3m3 r1  r3r1  r3 m4m4 r2  r3r2  r3 c1c1 r2  r1r2  r1 c2c2 r4  r1r4  r1 c3c3 r5  r1r5  r1 c4c4 G1G1 G2G2 G3G3 G3G3 Young Yoon15WWW'11

x(m 1 ) (G1)(G1) x(c 3 ) (G2)(G2) r1r1 r1  r2r1  r2 m1m1 r3  r4r3  r4 m2m2 r1  r5r1  r5 m3m3 r1  r3r1  r3 m4m4 r2  r3r2  r3 c1c1 r2  r1r2  r1 c2c2 r4  r1r4  r1 c3c3 r5  r1r5  r1 c4c4 G1G1 G2G2 G4G4 G3G3 Young Yoon16WWW'11

x(m 1 ) x(m 4 ) (G1)(G1) x(c 3 ) (G2)(G2) (G4)(G4) r1r1 r1  r2r1  r2 m1m1 r3  r4r3  r4 m2m2 r1  r5r1  r5 m3m3 r1  r3r1  r3 m4m4 r2  r3r2  r3 c1c1 r2  r1r2  r1 c2c2 r4  r1r4  r1 c3c3 r5  r1r5  r1 c4c4 G1G1 G2G2 G4G4 G3G3 Young Yoon17WWW'11

x(m 1 ) x(m 4 ) (G1)(G1) x(c 3 )x(c 2 ) (G2)(G2) (G4)(G4) (G1)(G1) r1r1 r1  r2r1  r2 m1m1 r3  r4r3  r4 m2m2 r1  r5r1  r5 m3m3 r1  r3r1  r3 m4m4 r2  r3r2  r3 c1c1 r2  r1r2  r1 c2c2 r4  r1r4  r1 c3c3 r5  r1r5  r1 c4c4 G1G1 G2G2 G3G3 G3G3 Young Yoon18WWW'11

x(m 1 ) x(m 4 ) x(m 3 ) (G1)(G1) x(c 3 )x(c 2 ) (G2)(G2) (G4)(G4) (G1)(G1) (G3)(G3) r1r1 r1  r2r1  r2 m1m1 r3  r4r3  r4 m2m2 r1  r5r1  r5 m3m3 r1  r3r1  r3 m4m4 r2  r3r2  r3 c1c1 r2  r1r2  r1 c2c2 r4  r1r4  r1 c3c3 r5  r1r5  r1 c4c4 G1G1 G2G2 G3G3 G3G3 Young Yoon19WWW'11

x(m 1 ) x(m 4 ) x(m 3 ) (G1)(G1) x(c 3 )x(c 2 ) (G2)(G2) (G4)(G4) (G1)(G1) (G3)(G3) r1r1 x(c 4 ) (G3)(G3) r1  r2r1  r2 m1m1 r3  r4r3  r4 m2m2 r1  r5r1  r5 m3m3 r1  r3r1  r3 m4m4 r2  r3r2  r3 c1c1 r2  r1r2  r1 c2c2 r4  r1r4  r1 c3c3 r5  r1r5  r1 c4c4 G1G1 G2G2 G3G3 G3G3 Young Yoon20WWW'11

Young YoonWWW'1121 Send fraud warning messages Receive fraud warning message Issue refund Receive refund message Auditing service Accounting service CRM serviceClient service Receive confirmation Send confirmation Revisiting the credit card fraud detection process

22 Young YoonWWW'11 Decomposition overhead

23 Young YoonWWW'11 Task increase per partner

24 Goal 1: Prevent semantic conflict Goal 2: Build a distributed framework Goal 3: Make the framework be efficient Young YoonWWW'11

B B B B B B LP CA B LP Coordinator Constraint Deployer CALP LP (Sub-constraints): Local Process CA: Choreography Agent B: Pub/Sub Broker LP 25 Young YoonWWW'11 SSS S SS S SS SS SS S

B B B B B B LP CA B LP Coordinator Constraint Deployer CALP LP (Sub-constraints): Local Process CA: Choreography Agent B: Pub/Sub Broker Process Instance Initializer Local Process Interpreter Choreography Message Generator LP Choreography Message Listener Pub/Sub Translator Pub/Sub Communication Interface CA / Coordinator 26 Young YoonWWW'11

27 Young YoonWWW'11 Coordination overhead

28 Young YoonWWW'11 Scalability 60% ↑ 14% ↑

29 Goal 1: Prevent semantic conflict. Goal 2: Build a distributed framework. Goal 3: Make the framework be efficient. Young YoonWWW'11

30 Minimize the coordination overhead. Young YoonWWW'11 Distance from candidate location to task r i Weight: Execution time of task r i

31 r1  r2r1  r2 m1m1 r3  r4r3  r4 m2m2 r1  r5r1  r5 m3m3 r6  r7r6  r7 m4m4 G1G1 G2G2 G4G4 G3G3 r2  r8r2  r8 m5m5 G5G5 Pick Higher weights given to a task with more following tasks. There can be nested picks. Thus weights are recursively computed. Young YoonWWW'11

32 The bottom line 1: We prevent semantic conflict even when the collaboration is dynamic. Because of the agent-based framework and enrichment of safety constraints during reliable decomposition. Young YoonWWW'11

33 The bottom line 2: Efficient and scalable enforcement of safety constraints Because of the novel coordinator placement method and concurrent execution of decomposed tasks through parallel and distributed choreography agents. Young YoonWWW'11

34 Thank You! Young YoonWWW'11

35 Evaluation Young YoonWWW'11 15 nodes on IBM x3550 cluster: 1Gbps Ethernet, Intel Xeon 5120 dual-core 1.82GHz, 4GB RAM PADRES content-based pub/sub broker overlay: Random overlay of node degree up to 2 Service choreography generated as a directed graph: Max 1,000 tasks, partners and executions