Nitin Kumar Yadav RMIT University, Melbourne Minor thesis - semester 2, 2009, under the supervision of Dr. Sebastian Sardina, RMIT university An empirical evaluation of regression and progression approaches for behavior composition

Behavior Composition Regression and Progression techniques Regression approach Implementation Framework for behavior composition experiments Experiments Conclusion Contents An empirical evaluation of regression and progression approaches for behavior composition

Behavior Composition 3 What is a behavior ? Behavior – Logic of a machine – Web service – Stand alone component Abstracted as finite transition systems Available behaviors can be non-deterministic 1. Sardina,Patrizi & De Giacomo, Proceedings of Principles of Knowledge Representation and Reasoning (KR),pages , September AAAI Press. Painting Blocks Example 1

Behavior Composition 4 Behaviors perform actions in a shared environment ArmB: prepare

Behavior Composition 5 Behaviors perform actions in a shared environment ArmA: clean Behavior guards over the environments must be satisfied.

Behavior Composition 6 Behaviors perform actions in a shared environment ArmA: dispose

Behavior Composition 7 Behaviors perform actions in a shared environment ArmC: recharge

8 Behavior Composition What is behavior composition ? Can the behaviors, executing one at a time realize a virtual target ? Does there exist a controller which can guarantee that the available behaviors can always do the actions that the target can request.

Behavior Composition 9 Enacted system of available behaviors All what the system can do a 1 b 1 c 1 e1e1 a 1 b 2 c 1 e2e2 a 1 b 1 c 1 e2e2 a 1 b 3 c 1 e2e2 a 2 b 1 c 1 e2e2 e3e3 a 2 b 3 c 1 e2e2 e3e3 a 1 b 1 c 2 e1e1 e2e2 B: prepare B: paint A: clean C: recharge C: prepare A: recharge a 2 b 2 c 1 e2e2 e3e3 A: clean

Behavior Composition 10 Enacted target system All what the target can request t1t1 e1e1 t2t2 e2e2 t3t3 e2e2 t3t3 e3e3 t4t4 e3e3 t4t4 e2e2 t5t5 e1e1 t5t5 e4e4 recharge prepare recharge clean paint clean paint dispose paint

Does the enacted system ‘behave’ like the enacted target ? 11 Behavior Composition

Techniques 12 Two approaches for behavior composition based on simulation Regression based approach [Sardina,Patrizi & De Giacomo, KR 2008] Progression based approach [Stroeder & Pagnucco, 2009, IJCAI 2009]

Simulation 13 A transition system T1 simulates another transition system T2 iff T1 can ‘mimic’ all the states of T2 A state in the available system mimics another state in the target system if: – It can do all the actions that the target state can do – The successor state in the available system as a result of such an action simulates the resulting state in the target system Simulation is a relation of states of the enacted system and the states of the enacted target which can be ‘mimicked’.

Progression approach Start from the initial state of the enacted system and the enacted target. ‘Expand’ the enacted system with respect to what the target can request. After every expansion step, ‘mark’ the bad states. Enacted system can realize the target iff the initial state is not marked. 14

Progression approach How to expand states ? – Instantiate the next states by performing all the actions that the target can request from its current state. – Evolve both system and the target to their next states as a result of these actions. Note: the system states are expanded with respect to the target. 15

Progression approach How to mark bad states ? – A enacted state is considered ‘bad’ if It cannot perform the actions that the target can request. OR For one action, a child w.r.t each behavior is marked as bad. OR Target state is final but System is not 16 This is similar to the simulation concept !

17 Example a 1 b 1 c 1 e1e1 t1t1 e1e1 t2t2 e2e2 prepare Enacted Target Enacted System a 1 b 2 c 1 e2e2 B:prepare t4t4 e2e2 paint a 1 b 1 c 1 e2e2 a 1 b 3 c 1 e2e2 B: paint dispose t5t5 e1e1 A: dispose a 1 b 1 c 1 e2e2 a 1 b 3 c 1 e2e2 recharge t1t1 e1e1 a 1 b 3 c 1 e2e2 a 1 b 1 c 1 e2e2 A: Recharge B: Recharge C: Recharge a 1 b 1 c 2 e2e2 a 1 b 3 c 1 e2e2 C: Recharge a 1 b 1 c 1 e2e2 A: Recharge Progression approach

Progression approach – IJCAI ‘09 Salient features – Works by expanding ‘decision nodes’ and marking ‘bad’ states and propagating marking backwards. – Expands states in relation to the target – Similar to how we reason – Implementation available Java based Uses recursion to expand and mark states 18

Regression approach Step1 – Build the enacted system and enacted target independent of each other Step2 – Assume each system states simulates every target state. i.e. add a potential simulation link from each system state to every target state. 19

Regression approach Step3 – Iteratively remove the links that violate the simulation definition i.e. Can the state simulate the target state ? Is the resulting state in simulation with the resulting state of the target ? If the target state is final but the behavior states are not Stop when no more links can be removed A solution exists if the initial state is in the simulation relation 20

21 Example Enacted Target Regression approach a 1 b 1 c 1 e1e1 t1t1 e1e1 t2t2 e2e2 prepare a 1 b 2 c 1 e2e2 B:prepare t4t4 e2e2 paint a 1 b 1 c 1 e2e2 a 1 b 3 c 1 e2e2 B: paint a 2 b 2 c 1 e2e2 e3e3 A: clean dispose t5t5 e1e1 A: dispose a 1 b 1 c 1 e2e2 a 1 b 3 c 1 e2e2 recharge t1t1 e1e1 a 1 b 3 c 1 e2e2 a 1 b 1 c 1 e2e2 A: Recharge B: Recharge C: Recharge a 1 b 1 c 2 e2e2 a 1 b 3 c 1 e2e2 C: Recharge a 1 b 1 c 1 e2e2 A: Recharge a 1 b 1 c 2 e1e1 C:recharge Enacted System X X X

22 Example Enacted Target Regression approach a 1 b 1 c 1 e1e1 t1t1 e1e1 t2t2 e2e2 prepare a 1 b 2 c 1 e2e2 B:prepare t4t4 e2e2 paint a 1 b 1 c 1 e2e2 a 1 b 3 c 1 e2e2 B: paint a 2 b 2 c 1 e2e2 e3e3 A: clean dispose t5t5 e1e1 A: dispose a 1 b 1 c 1 e2e2 a 1 b 3 c 1 e2e2 recharge t1t1 e1e1 a 1 b 3 c 1 e2e2 a 1 b 1 c 1 e2e2 A: Recharge B: Recharge C: Recharge a 1 b 1 c 2 e2e2 a 1 b 3 c 1 e2e2 C: Recharge a 1 b 1 c 1 e2e2 A: Recharge a 1 b 1 c 2 e1e1 C:recharge Enacted System

23 Example Enacted Target Regression approach a 1 b 1 c 1 e1e1 t1t1 e1e1 t2t2 e2e2 prepare a 1 b 2 c 1 e2e2 B:prepare t4t4 e2e2 paint a 1 b 1 c 1 e2e2 a 1 b 3 c 1 e2e2 B: paint a 2 b 2 c 1 e2e2 e3e3 A: clean dispose t5t5 e1e1 A: dispose a 1 b 1 c 1 e2e2 a 1 b 3 c 1 e2e2 recharge t1t1 e1e1 a 1 b 3 c 1 e2e2 a 1 b 1 c 1 e2e2 A: Recharge B: Recharge C: Recharge a 1 b 1 c 2 e2e2 a 1 b 3 c 1 e2e2 C: Recharge a 1 b 1 c 1 e2e2 A: Recharge a 1 b 1 c 2 e1e1 C:recharge Enacted System X X X X X Assume

Salient features – Enacted system and the enacted target are built independent of each other – Allows recovery from behavior failure and any uncontrolled action – The enacted system can be stored in memory and different targets can then be checked for solution 24 Regression approach

Related tools available – Symfony, Opus [ De Giacomo et all, 2009, University of Rome ‘La Sapienza’ ] Used for web services composition Do not have a concept of environment and guards Our implementation – First serious and complete implementation – Optimizations added on top of the original approach 25 Regression approach Implementation

Step2 : Original – Starts with links from every system state connected to every target state – In each iteration check If the state is in simulation ? Is the resulting state in simulation ? Is the state final ? 26 Regression approach Optimizations Step2: Optimization 1: -Starts with links between the ‘similar’ states. i.e. add link only if the system state can do all what a target state can request. -In each iteration Is the resulting state in simulation ?

27 Step2: Original Enacted Target Regression approach a 1 b 1 c 1 e1e1 t1t1 e1e1 t2t2 e2e2 prepare a 1 b 2 c 1 e2e2 B:prepare t4t4 e2e2 paint a 1 b 1 c 1 e2e2 a 1 b 3 c 1 e2e2 B: paint a 2 b 2 c 1 e2e2 e3e3 A: clean dispose t5t5 e1e1 A: dispose a 1 b 1 c 1 e2e2 a 1 b 3 c 1 e2e2 recharge t1t1 e1e1 a 1 b 3 c 1 e2e2 a 1 b 1 c 1 e2e2 A: Recharge B: Recharge C: Recharge a 1 b 1 c 2 e2e2 a 1 b 3 c 1 e2e2 C: Recharge a 1 b 1 c 1 e2e2 A: Recharge a 1 b 1 c 2 e1e1 C:recharge Enacted System

28 Step2: Optimized Enacted Target Regression approach a 1 b 1 c 1 e1e1 t1t1 e1e1 t2t2 e2e2 prepare a 1 b 2 c 1 e2e2 B:prepare t4t4 e2e2 paint a 1 b 1 c 1 e2e2 a 1 b 3 c 1 e2e2 B: paint a 2 b 2 c 1 e2e2 e3e3 A: clean dispose t5t5 e1e1 A: dispose a 1 b 1 c 1 e2e2 a 1 b 3 c 1 e2e2 recharge t1t1 e1e1 a 1 b 3 c 1 e2e2 a 1 b 1 c 1 e2e2 A: Recharge B: Recharge C: Recharge a 1 b 1 c 2 e2e2 a 1 b 3 c 1 e2e2 C: Recharge a 1 b 1 c 1 e2e2 A: Recharge a 1 b 1 c 2 e1e1 C:recharge Enacted System

Parent state checking. – If a link is removed in the iteration, we need to check ONLY the parent states. – Consider the chain example. 29 Regression approach Optimization 2 t1t1 e1e1 t2t2 e1e1 a t3t3 e1e1 a a t4t4 e1e1 a t5t5 e1e1 s1s1 e1e1 s2s2 e1e1 a s3s3 e1e1 a a s4s4 e1e1 b s5s5 e1e1 Enacted Target Enacted System

Original approach (also optimization 1) Optimization 2 30 Regression approach Optimization 2 t1t1 e1e1 t2t2 e1e1 a t3t3 e1e1 a a t4t4 e1e1 a t5t5 e1e1 s1s1 e1e1 s2s2 e1e1 a s3s3 e1e1 a a s4s4 e1e1 b s5s5 e1e1 System Target t1t1 e1e1 t2t2 e1e1 a t3t3 e1e1 a a t4t4 e1e1 a t5t5 e1e1 s1s1 e1e1 s2s2 e1e1 a s3s3 e1e1 a a s4s4 e1e1 b s5s5 e1e1 System Target NO SOLUTION

Original approach (also optimization 1) Optimization 2 31 Regression approach Optimization 2 t1t1 e1e1 t2t2 e1e1 a t3t3 e1e1 a a t4t4 e1e1 a t5t5 e1e1 s1s1 e1e1 s2s2 e1e1 a s3s3 e1e1 a a s4s4 e1e1 b s5s5 e1e1 System Target t1t1 e1e1 t2t2 e1e1 a t3t3 e1e1 a a t4t4 e1e1 a t5t5 e1e1 s1s1 e1e1 s2s2 e1e1 a s3s3 e1e1 a a s4s4 e1e1 b s5s5 e1e1 System Target NO SOLUTION

3 rd optimization – Check if the initial state of the system is still in simulation – Reduces the number of iterations if no solution exists – Consider the modified chain example 32 Regression approach Optimization 3 t1t1 e1e1 t2t2 e1e1 a t3t3 e1e1 a a t4t4 e1e1 a t5t5 e1e1 s1s1 e1e1 s2s2 e1e1 b s3s3 e1e1 a a s4s4 e1e1 b s5s5 e1e1 System Target

Original approach (also optimization 1) Optimization 3 33 Regression approach Optimization 3 t1t1 e1e1 t2t2 e1e1 a t3t3 e1e1 a a t4t4 e1e1 a t5t5 e1e1 s1s1 e1e1 s2s2 e1e1 b s3s3 e1e1 a a s4s4 e1e1 b s5s5 e1e1 System Target t1t1 e1e1 t2t2 e1e1 a t3t3 e1e1 a a t4t4 e1e1 a t5t5 e1e1 s1s1 e1e1 s2s2 e1e1 b s3s3 e1e1 a a s4s4 e1e1 b s5s5 e1e1 System Target NO SOLUTION

Benchmarking Framework Benchmarking in Java. – compilation First generation of JVM entirely interpreted Just In Time compilation Hot Spot dynamic compilation – Class loading – Garbage Collection – Accuracy of system.currentTimeMillis() is platform dependent. 34

Benchmarking Framework Japex. – measures class loading, compilation time, garbage collection time. – Easy to implement driver classes public void prepare(TestCase testCase); public void warmup(TestCase testCase); public void run(TestCase testCase); – Define test cases in xml 35

Benchmarking Framework Designing Benchmark Problems using a library of available behaviors by manipulating them. 36 Benchmarks Number of Behaviors Target Complexity Environment Complexity ND Amplification Chains

Benchmarking Framework No of behaviors – If a solution exists, increasing the number of behaviors exponentially complicates the problem without affecting the ‘reliazability’. – Example armA:1111 armB:1234 armC: Test case 1 Test case 2Test case 3 Test case 4

Benchmarking Framework Target Complexity – Make the target do multiple loops without affecting the solution. – Example 38 t1t1 t2t2 t3t3 t4t4 t5t5 prepare clean paint dispose recharge t1t1 t2t2 t3t3 t4t4 t5t5 prepare clean paint dispose recharge t1t1 t2t2 t3t3 t4t4 t5t5 prepare clean paint dispose recharge

Benchmarking Framework Target Complexity with solution break – The last loop of the target requests an unknown action. – Multiple break points 39 Target 1 recharge breakAction Target 1 breakAction recharge breakAction

Benchmarking Framework Environment Complexity – Similar to target complexity except for Removal of guards from the target/behaviors Chains – Single behavior and target with ‘n’ states. – Behavior cannot do the last action of the target 40 t1t1 e1e1 t2t2 e1e1 a t3t3 e1e1 a a t4t4 e1e1 a t5t5 e1e1 s1s1 e1e1 s2s2 e1e1 a s3s3 e1e1 a a s4s4 e1e1 b s5s5 e1e1 Enacted Target Enacted System

Benchmarking Framework ND Amplification – Add more non-deterministic states to an already existing non-deterministic behavior 41 b1b1 b2b2 prepare b3b3 b4b4 paint Paint, clean recharge clean b1b1 b2b2 prepare b3b3 b4b4 paint Paint, clean recharge clean b2`b2` Paint, clean paint prepare ND Amplification = 1

Experiments No of Behaviors 42

Experiments No of Behaviors 43

Experiments Target Complexity 44

Experiments Target Complexity 45

Experiments Complex web services example 46 From honor’s thesis of Paolo Felli, Service Composition through Synthesis Techniques based on Alternating-time Temporal Logic, University of Rome

Experiments 47 ToolTime Symfony~50 mins Opus18 hrs Regression Original23 seconds Regression Optimized15 seconds TLV100 ms

Conclusion First serious implementation of regression approach The first optimization should always be done The second optimization is beneficial of patterns similar to chain Benchmark problems creation by modifying existing problems. 48

Questions ? Comparing the speed of the techniques