1. An Agent Oriented Programming Language integrating Temporal Planning and the Plan Coordination Mechanisms Muhammad Adnan HASHMI Thesis Supervisor: Amal EL FALLAH SEGHROUCHNI

2. 2 Objectives AOPL + PlanningPlanning + Coordination (Temporal Planning, Reactivity, Dynamic Environments) (Temporal Planning, Different Priority Goals) AOPL integrating Planning, Plan Repairing and Coordination

3. 3 Outline Context and Objectives Plan Coordination Mechanisms  Coordinated Planning Problem  Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning  Language Definition  Planning Mechanism Conclusion and Perspectives

4. 4 Outline Context and Objectives Plan Coordination Mechanisms  Coordinated Planning Problem  Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning  Language Definition  Planning Mechanism Conclusion and Perspectives

5. 5 AOP Languages Allow to program intelligent and autonomous agents Main Characteristics  Mental State: Beliefs, Goals, Commitments  Reasoning Mechanism  Capabilities, Services  Communication  Concurrence Some Languages  Agent-0 [Shoham 1993], 2APL [Dastani 2008], AgentSpeak (L) [Rao 1996]

6. 6 Objectives (AOP Languages) Current languages  Most AOP languages follow a reactive (PRS based) approach  AOP languages do not support temporal planning Only a few support planning Problems  Execution without planning may result in the goal failures  Agent can reach a dead end  Conflicts can arise among different plans  Real world actions take place over a timespan

7. 7 Objectives (AOP Languages) Current languages  Most AOP languages follow a reactive (PRS based) approach  AOP languages do not support temporal planning Only a few support planning Our aim  Propose an extension to a programming language to endow it with planning skills Has temporal planning Deals with uncertainty of the environment Incorporate reactivity by dealing with on the fly goals having different priorities

8. 8 Related Work CANPLAN [De Silva et al. 2005] combines JSHOP with JACK  Programmer’s responsibility to call the planner  Doesn’t deal with uncertainty of the environment X-BDI [Meneguzzi et al. 2004] incorporates classical planning in a BDI framework  Efficiency concerns  Loss of the domain information CYPRESS [Myers et al. 1994] integrates SIPE-2 with PRS

9. 9 Plans Coordination Planning for single agents  Computing plan from Initial State to Goal State Coordination of plans  Removing conflicts (negative interactions)  Utilizing help relations (positive interactions) abcabc A1A2 dede q I G

10. 10 Related Work (Plans Coordination) Coordination before planning  Social Laws [Shoham 1995], [Buzzing 2006] Coordination during planning  Partial Global Planning [Durfee and Lesser 1987]  Incremental Plan Merging [Alami et al. 1994]  Recursive Petri Nets [El Fallah Seghrouchni and Haddad 1996] Coordination after planning  Temporal Plan Merging [Tsamardinos et al. 2000]  Merging Hierarchical Plans [von Martial 1992]

11. 11 Objectives (Plans Coordination) Propose plans coordination mechanisms for the plans having different priorities Two different scenarios  Coordination during planning Coordinated Planning Problem (CPP)  Coordination after planning Proactive-Reactive Coordination Problem (PRCP)

12. 12 Outline Context and Objectives Plan Coordination Mechanisms  Coordinated Planning Problem  Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning  Language Definition  Planning Mechanism Conclusion and Perspectives

13. 13 Assumptions Two agents α and β sharing the same environment  Agent α having higher priority (reactive) goals  Agent β having normal priority (proactive) goals Two possible conflicts among plans  Causal link threat  Parallel actions interference

14. 14 Two Possible Conflicts Causal Link (A1, A2, p)  Action A1 adds an effect p  Action A2 needs this effect  No action between A1 and A2 adding p Causal Link Threat  If an action A deletes p and lies between A1 and A2, then A threatens the causal link (A1, A2, p) A1A2 p A ¬p Threat

15. 15 Two Possible Conflicts Causal Link (A1, A2, p)  Action A1 adds an effect p  Action A2 needs this effect  No action between A1 and A2 adding p Parallel Actions Interference  Actions A1 and A2 lie in parallel  Either one of them deletes the preconditions or add effects of the other A1 A2 ¬p p A1 A2 p ¬p

16. 16 Outline Context and Objectives Plan Coordination Mechanisms  Coordinated Planning Problem  Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning  Language Definition  Planning Mechanism Conclusion and Perspectives

17. 17 Coordinated Planning Problem Prerequisite:  Plan P α of Agent α Our Aim:  Compute a Plan P β for Agent β Has no conflict with P α Avails the cooperative opportunities offered by P α Solution:  Non Temporal Domains  µ-SATPLAN  Temporal Domains  Coordinated-Sapa

18. 18 Coordinated-Sapa Our extension of the well-known temporal planner Sapa [M. Do & S. Kambhampati 2001] Input  Initial State I  A Set of Actions D  A Set of Proactive Goals G P with deadlines  A Reactive Plan P R Output  A plan P P from I to G P P P is consistent with P R OR  Deadline Violated  Failure + Goal name whose deadline is violated OR  No Solution Possible  Failure

19. SAPA [Do and Kambhampati 2001] A Multi-Objective Heuristic Based Temporal Planner

20. 20 Sapa Search through the space of time-stamped states S=(P,M, ,Q,t) Set of predicates pi and the time of their last achievement t i < t. Set of functions represent resource values. Set of protected persistent conditions. Event queue containing delayed effects. Time stamp of S.

21. 21 Sapa Search through the space of time-stamped states S=(P,M, ,Q,t) Set of predicates pi and the time of their last achievement t i < t. Set of functions represent resource values. Set of protected persistent conditions. Event queue containing delayed effects. Time stamp of S.

22. 22 Main Flow-Diagram of SAPA S=(P,M, ,Q,t) S=(P = I, M, , Q =φ, t = 0)

23. 23 Some Important Concepts Goal Satisfaction: S=(P,M, ,Q,t)  G if   G either:    P, tj < ti and no event in Q deletes pi   e  Q that adds pi at time te < ti Action Application: Action A is applicable in S =(P,M, ,Q,t) if:  All preconditions of A are satisfied by P  A’s effects do not interfere with Q When A is applied to S =(P,M, ,Q,t) :  P is updated according to A’s instantaneous effects  Delayed effects of A are put in Q. Special Advance-Time Action  Advances time to next earliest event in the queue, and adds the event to P S=(P,M, ,Q,t)

24. Coordinated-Sapa An Extension of Sapa that Finds Coordinated Plans in Temporal Domains

25. 25 Handling Positive Interactions Before starting planning for Agent β  Create an event e = (st, et, effect) corresponding to every effect k of every action A in P α e.st = st(A) e.et = et(A) e.effect = k  Put all these events in the event queue Q WHY? At every state, Coordinated-Sapa will take into account all the changes made by Agent α  Advanced-Time action will add the effects generated by Agent α, to P

26. 26 Handling Negative Interactions Add another action applicability condition to the planning mechanism of Agent β Action Application: Action A is applicable in S=(P,M, ,Q,t) if:  All preconditions of A are satisfied by P  A’s effects do not interfere with Q  A does not threaten any causal link at t S=(P,M, ,Q,t)

27. 27 Example Plan Generated Agent β drives person P4 in Car1 to city C2  So that P4 can board the plane Pl1  Agent α is bringing the plane Pl1 to C2 from C3 Agent β makes person P4 board the plane Pl1 at C2  Agent α flies plane Pl1 from C2 to C5

28. 28 Experimental Results Domains and problems taken from 3rd International Planning Competition A multi-agent problem is generated by taking the original problem and dividing the goals in two sets, one for each agent

29. 29 Outline Context and Objectives Plan Coordination Mechanisms  Coordinated Planning Problem  Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning  Language Definition  Planning Mechanism Conclusion and Perspectives

30. 30 Proactive-Reactive Coordination Prerequisite:  Reactive plan P α of Agent α  Proactive plan P β of Agent β Our Aim:  Modify plan P β such that: It has no conflict with P α Avails the cooperative opportunities offered by P α Solution:  Plan Merging Algorithm

31. 31 Case Study Tasks of Rescue Agent  Rescue the Victims Tasks of Analyzer Agent  Analyze the goal cells  Call the central agent Constraints  One agent in a cell  Hyper energy cells Needs fuel or energy to enter  Agent should have key to open door Rescue Agent : α Analyzer Agent : β D 123456789101112131415 C B A

32. 32 Conflict Resolution Threat-Repair Link (A1, A2, p)  Action A1 deletes p  A2 is a subsequent action and adds p  A1 is called Threat Action  A2 is called Repair Action B1B2 p A1 -p A2 p Threat Repair

33. 33 Valid and Possibly Valid Time Stamps Possibly Valid Time Slot for an action A  All preconditions are met  No parallel actions interference P[1]  a bhbh P[2]  b c -d P[3]  c eP[4]  e fP[5]  f g P[1]  b d -h P[6]  g iP[7]  i  h g Valid Time Slot for an action A  All preconditions are met  No parallel actions interference  Either: No causal link threat Repair Action exist before the deadline P[1]  a bhbh P[2]  b c -d P[3]  c eP[4]  e fP[5]  f g P[1]  b d -h P[6]  g iP[7]  i  h g P[2]  d k P[3]  kh

34. 34 Plan Merging Algorithm Fix all the actions of Reactive Plan P α on timeline For every action CA of Proactive Plan  Search for the first Possibly Valid Time Slot T on timeline  Reason about the time slot T There could be 5 cases at T

35. 35 Plan Merging Algorithm Case 1: No causal link threat by CA at T  Assign Time Slot T to CA EXAMPLE Current Action: Move(A1, A2)  Returned Time Slot: 0 - 5  Any Threat? : No  Assign Time Slot 0 – 5 to CA D 123456789101112131415 C B A

36. 36 Plan Merging Algorithm Case 2: CA threatens a Causal Link but Repair Action exist  Assign Time Slot T to CA  Save a Possible Threat EXAMPLE Current Action: Move(A4, A5)  Time Slot: 20 - 25  Any Threat? : Yes (Agent α needs A5 at time 40-45)  Repair Action: Move(A5, A6)  Assign Time Slot 20 - 25 to Move (A4, A5)  Save D 123456789101112131415 C B A

37. 37 Plan Merging Algorithm Case 3: It is a Repair Action but can not meet a deadline of some Threat Action  Backtrack to the Threat Action,find another time stamp EXAMPLE Current Action: Move (A8, A9)  Returned Time Slot: 50 - 55  Any Threat?: Yes (Agent α needs A9 at 85-110)  Repair Action : Move (A9, B9)  Save Next Action: AnalyzeCell (A9)  Time Slot Assigned: 55 - 70 D 123456789101112131415 C B A

38. 38 Plan Merging Algorithm Next Action: CallCentral (A9)  Time Slot Assigned: 80 – 90 Next Action: Move (A9, B9)  Is it a Repair Action? : Yes  Meet all deadlines?: No (Agent α needs A9 at 85)  Backtrack to action Move(A8, A9) Find another Time Slot New Time Slot: 110 – 115 (Valid Time Slot) D 123456789101112131415 C B A Attention

39. 39 Plan Merging Algorithm Case 4: All the effects of CA are already achieved WHAT TO DO? Mark CA as redundant POST PROCESSING Remove all redundant actions from the plan Recursively remove all actions which achieve only the preconditions of removed action

40. 40 Plan Merging Algorithm EXAMPLE Current Action: OpenDoor (C11)  Returned Time Slot: 172 - 175  Redundant(OpenDoor(C11))  true Because openedDoor(C11) is true at time 172  When the plan is returned Remove OpenDoor(C11) from plan Also remove TakeKey(C11, key1) from plan D 123456789101112131415 C B A

41. 41 Plan Merging Algorithm Case 5: Action CA’s preconditions can not be achieved  Remove action CA from the plan and compute a plan to achieve effects of CA I = State just before CA G = Effects (CA)  Plan should have no conflict with Reactive Plan P β and if CA is a repair action, repair effects must meet their deadline ReplacementPlan = Coordinated-Sapa (I, G, P β )  If a plan is returned, replace the removed actions with the plan  If a deadline is violated, backtrack to the threat action  If no plan possible, then remove another action CA + 1 G = G U Effects (CA + 1) \ Pre (CA + 1) Use Coordinated-Sapa

42. 42 Plan Merging Algorithm EXAMPLE Current Action: TakeEnergy(B13, energy1) Preconditions can not be achieved  Repair the plan D 123456789101112131415 C B A

43. 43 Plan Repair Algorithm Create a CPP by removing TakeEnergy(B13, energy1)  I = { at(β, B13), at(energy1, B13), at(energy2, B13) }  G = { hasEnergy(β, energy), at(β, B13)} Call Coordinated-Sapa to solve this CPP  Coordinated-Sapa returns fail  Why? energy2 is also needed by Agent α D 123456789101112131415 C B A

44. 44 Plan Repair Algorithm Create another CPP by removing Move(B13, A12)  I = { at(β, B13), at(energy1, B13), at(energy2, C15) }  G = { at(β, A12) } Call Coordinated-Sapa to solve this CPP  A plan is returned to enter A12 by taking the fuel from D14 POST PROCESSING This plan will become a replacement for both TakeEnergy(B13, energy1) and Move(B13, A12) D 123456789101112131415 C B A

45. 45 Outline Context and Objectives Plan Coordination Mechanisms  Coordinated Planning Problem  Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning  Language Definition  Planning Mechanism Conclusion and Perspectives

46. 46 An AOP language having:  Cognitive aspects specific to intelligent agents  Communication primitives  Mobility primitives CLAIM Agent:  Is autonomous, intelligent and mobile  Has a mental state containing knowledge, goals, and capabilities  Is able to communicate with other agents  Entails a reactive behaviour CLAIM [Suna and El Fallah Seghrouchni 2005]

47. 47 Agent Definition in CLAIM defineAgent agentName{ parent = null | agentName ; knowledge = null ;| {knowledge 1 ; …; knowledge m } goals = null ; | {goal 1 ; … ; goal n } capabilities = null ; {capability 1 … capability o } agents = null ; | {agName 1, agName 2, …, agName q } } capability = name { message = null | message ; conditions = null | condition ; do { process } effects = null ;| { effect1 ; …; effectf } }

48. 48 An AOP language having:  Cognitive aspects specific to intelligent agents  Communication primitives  Mobility primitives  Temporal planning capability (New) P-CLAIM Agent:  Is autonomous, intelligent and mobile  Has a mental state containing knowledge, goals, and capabilities  Is able to communicate with other agents  Entails a planning based behaviour (New)  Achieves goals based on their priorities (New)  Maintains the stability of the plan in the dynamic environments (New) P-CLAIM

49. 49 Agent Definition in P-CLAIM Similar to CLAIM, but  Added priorities to goals High Preemptive (reactive): Should be immediately planned for and achieved High and Normal (proactive): High priority goals should be achieved before normal priority goals  Capabilities in CLAIM  (Activities + Actions) in P-CLAIM Activities are short plans to achieve tasks Actions are primitive tasks with duration

50. 50 Outline Context and Objectives Plan Coordination Mechanisms  Coordinated Planning Problem  Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning  Language Definition  Planning Mechanism Conclusion and Perspectives

51. 51 Agent Definition to Planning (Translator) Translator (JavaCC) KnowledgeGoalsActivitiesActions Initial StateGoalsMethodsOperators Problem FileDomain File Agent Description File Planner

52. 52 Agent Life Cycle

53. 53 Messages Handler

54. 54 Waits for messages from other agents Message is a request to achieve a goal?  Assigns priority to the goal  High Preemptive  puts it in Global Reactive Goals (GRG) queue  High or Normal  puts it in Global Proactive Goals (GPG) priority queue Message is an information?  Store the information in the knowledge base of the agent Messages Handler

55. 55 Goal Priority Assignment Message Priority Sender’s Importance HighNormal High Preemptive High Normal

56. 56 Planner

57. 57 Temporal Converter Compute Plan Main Algorithm GRGGPG 1- Fetch goals one by one from GRG and GPG and calls Compute_Plan to compute a plan for the goal 2- GPG Goals are accessed only when GRG is empty 3- Sends a suspension signal to Executor if the goal is reactive i.e. from GRG Planner

58. 58 Plan Computation JSHOP2 algorithm [Nau et el. 2003] is used to compute a totally ordered plan for each goal  An HTN planning algorithm  Decomposes the task into sub-tasks by applying methods  Recursively applies the same procedure on every composite sub-task until there are only primitive tasks T1T1 M1M1 T 11 T 12 M 12 M 11

59. 59 Temporal Converter Input to procedure  A totally ordered plan  Actions’ information (Add, Del, Pre, Durations) Output of procedure  A position constrained parallel plan Every action is assigned a time stamp Multiple actions can possibly lie in parallel

60. 60 P[1] aa -a +d +e +f P[2] ff -f +g +h P[3] ee+i P[4] hh -h +j P[5] gigi +k P[6] kk+l P[7] ll+m P[8] mm+c P[9] cmcm -c P[10] ee -k -c +h I = {a, b, c} Dur(P[1])=15Dur(P[2])=15Dur(P[3])=15Dur(P[4])=15Dur(P[5])=15Dur(P[6])=15Dur(P[7])=15Dur(P[8])=15Dur(P[9])=15Dur(P[10])=15 Temporal Converter (Example)

61. 61 P[1] aa -a +d +e +f P[2] ff -f +g +h P[3] ee+i P[4] hh -h +j P[5] gigi +k P[6] kk+l P[7] ll+m P[8] mm +n +c P[9] cmncmn -c P[10] ee -k -c +h MaxTPreTMinT Forbidden 142 5 MinOrderingT 3 6 7 Temporal Converter (Example) P[10] ee -k -c +h Action Under Consideration:

62. 62 P[1] aa -a +d +e +f P[2] ff -f +g +h P[3] ee+i P[4] hh -h +j P[5] gigi +k P[6] kk+l P[7] ll+m P[8] mm +n +c P[9] cmncmn -c P[10] ee -k -c +h P[1] aa -a +d +e +f P[2] ff -f +g +h P[3] ee+i P[4] hh -h +j P[5] gigi +k P[6] kk+l P[7] ll+m P[8] mm+c P[9] cmcm -c P[10] ee -k -c +h Input Plan Output Plan Makespan Gain: 30% Temporal Converter (Example)

63. 63 Experimental Results (Temporal Converter)

64. 64 Merging the New Plan to Global Plan

65. 65 Merging the New Plan to Global Plan Planner Append at the end of P exec Merge at the start of P exec Proactive GoalReactive Goal Plan Under Execution (P exec )

66. 66 Schedule Handler and Executor

67. 67 Schedule Handler and Executor

68. 68 Plan Mender

69. 69 Plan Mender SP(Pexec[1]) SP(Pexec[2]) SP(Pexec[3]) G SP(Pexec[4]) SW Compute Plan Using Sapa No Plan Compute Plan Using Sapa ReplacementPlan ContinuationPlan P execnew = ReplacementPlan + ContinuationPlan

70. 70 Experimental Results (Plan Mender) To perform tests, we explicitly modified the knowledge base of the agent to cause plan failure

71. 71 Outline Context and Objectives Plan Coordination Mechanisms  Coordinated Planning Problem  Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning  Language Definition  Planning Mechanism Conclusion and Perspectives

72. 72 Conclusion An agent oriented programming language supporting:  Temporal Planning  Plan Repairing  Dealing with different priority goals Coordinated Planning Problem  Computing plan while coordinating it with another plan SATPLAN  µ-SATPLAN Sapa  Coordinated-Sapa Proactive-Reactive Coordination Problem  Modifying a plan to remove conflicts with a higher priority plan Plan Merging Algorithm

73. 73 Conclusion Properties of the temporal conversion algorithm  Soundness (Proof: By construction)  Termination Properties of the plan merging algorithm  Soundness (Verified by experimental evaluation) Computational Complexity  Temporal Converter: Quadratic  Plan Merging Algorithm: Worst Case: Exponential Average Case: Quadratic

74. 74 Perspectives Coordination of plans for same priority goals  Negotiation based strategy Planning with incomplete information  Information gathering actions Improvement in the computational complexity of plan merging algorithm  Propose efficient heuristics to reduce the number of backtracks

75. 75 Publications 1. A. El Fallah Seghrouchni, M. A. Hashmi, “Multi-Agent Planning”, Book chapter in Software Agents, Agent Systems and Their Applications. M. Essaaidi et al. (Eds.), 2012, IOS Press. 2. Y. Dimopoulos, M. A. Hashmi, P. Moraitis, “µ-SATPLAN: Multi-Agent Planning as Satisfiability”, Knowledge Based Systems Journal, 2011 3. M. A. Hashmi, A. El Fallah Seghrouchni, “Merging of Temporal Plans supported by Plan Repairing”, Proceedings of 22nd IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2010), Aras, France 4. M. A. Hashmi, A. El Fallah Seghrouchni, “Coordination of Temporal Plans for the Reactive and Proactive Goals”, Proceedings of IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT’ 10), Toronto, Canada 5. Y. Dimopoulos, M. A. Hashmi, P. Moraitis, “Extending SATPLAN to Multiple Agents”, Proceedings of 30th SGAI International Conference on Artificial Intelligence 2010, Cambridge, UK. 6. M. A. Hashmi, A. El Fallah Seghrouchni, “Temporal Planning in Dynamic Environments for P-CLAIM Agents”, In proceedings of Languages, Methodologies and Development Tools for Multi-Agent Systems (LADS’09), Torino, Italy, Springer-Verlag.

76. 76 Publications 7. M. A. Hashmi, A. El Fallah Seghrouchni, “Temporal Planning in Dynamic Environments for Mobile Agents”, Proceedings of International Conference on Frontiers of Information Technology (FIT’ 09), Abottabad, Pakistan, Publisher ACM press. 8. M. A. Hashmi, “A Planning Component for CLAIM Agents”, In the proceedings of 17th International Conference on Control Systems and Computer Science, Bucharest, Romania, Volume 2, pages 485-492, Politehnica Press

77. 77 Thanks for your attention!!!