1. Boğaziçi University Planning and Coordination in A Multi-Agent Environment. Gökay Burak AKKUŞ 2003700717 cmpe530

2. Gökay Burak AKKUŞ Agent An agent is something that acts. An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators.

3. Gökay Burak AKKUŞ Planning The task of coming up with a sequence of actions that will achieve a goal is called planning

4. Gökay Burak AKKUŞ Planning Classical Planning Current State →→ Desired Goal Fully Observable Deterministic Finite Static Discrete Non-Classical Planning Hierarchical, Decision-Theoretic, Continual, Distributed

5. Gökay Burak AKKUŞ An agent plans on the basis of an incrementally learnt world model and reacts on the basis of incrementally learnt values that indicate the usefulness of his potential actions.

6. Gökay Burak AKKUŞ Planning Single-Agent Environments The agent is alone Multi-Agent Environments Other agents in enviroment considered The solution to a global problem emerges from the collective activities of independent agents.

7. Gökay Burak AKKUŞ Planner Figure 1 : Planner [7]

8. Gökay Burak AKKUŞ Multi-Agent Planning Cooperation Joint-goals & plans Coordination Multy-body planning(centralized planning agent) Synchronization Cooperation - Competition

9. Gökay Burak AKKUŞ Planning Agent A Single Agent for Planning & Coordination Execution & monitoring Re-Planning(when something goes wrong) in real- time responses Adaptation to dynamic environment Performance Prediction of agents A single agent search for multi-agent problems. Search for actions to be taken Search in state space (Agent + Action)

10. Gökay Burak AKKUŞ Planning Agent Distributed Planning Cooperative Distributed Planning (CDP), takes the individual planning process and distributes it among a subset of an agent society such that the generation and execution of a plan requires the interaction of several specialized agents Negotiated Distributed Planning (NDP), is based on the concept that agents should primarily be focused on the individual, but can use plans as a means of coordinating actions

11. Gökay Burak AKKUŞ agents should work together according to the following model: 1. Each agent works on large grained sub-problems 2. Agents should be able to generate partial results, even if these results are uncertain or the agent is missing information 3. Agents should communicate partial results to other agents asynchronously as the partial results are developed 4. Agents should use partial results from other agents to help resolve their own uncertainties and guide individual solutions Our planning agent helps in these, as it is the root of coordination hierarchy

12. Gökay Burak AKKUŞ Learning, Planning, Reacting Figure-2 : relationships between learning, planning, and reacting.

13. Gökay Burak AKKUŞ Some Definitions Reactive Coordination vs. Planning coordination The Knowledge needed: Ag = {A1,...An} : Available Agents in Environment Set of Actions that can be performed by an agent in an environment Set of Actions that can be performed at a certain state Set of agents that can carry out actions to reach a successor state (S → T) Estimated usefullnes of actions Validity, Satisfiability of Results, Partial results Joint Learning: Learning is realized by the agents through adjusting the estimates of their actions’ usefulness.

14. Gökay Burak AKKUŞ Performance Measures A performance measure embodies the criterion for success of an agent’s behavior. Objective performance measure : Typically one imposed by the designer. As a general rule it is better to design performance measures according to what one actually wants in the environment, rather than according to how one thinks the agent should behave.

15. Gökay Burak AKKUŞ Performance Evaluation PACE : Performance Analysis and Characterisation Environment PACE provides a means of dynamically obtaining runtime estimates for different applications on different resources through the performance information services framework. multiple (and often conflicting) metrics must be considered

16. Gökay Burak AKKUŞ Metrics Total execution time Average advance time Resource Utilization Rate Load balancing level

17. Gökay Burak AKKUŞ What To Do? A certain goal is defined, Planning agent searches the problem domain for possible instances of agents working in that domain (agent society) If problem involves more domains, tries to divide problem into sub-problems considering domain interactions Planning agent reaches the execution history of agents it will deal with. Planning agent evaluates the agent in [0..1] scale by means of Goal Satisfaction, (d1) Resource usage, (d2) Problem usefulnes, (d3) Coordination capabilities (d4)... 0<= d1*w1+d2*w2+d3*w3+d4*w4+...+dn*wn <= 1 w: specifies the impotance weight of a metric for the problem Then, it generates a joint plan using the ranking of agents to employ appropriate agents on specific tasks, Monitors the ongoing activities, communication between agents, and when needed Re-planning takes place

18. Gökay Burak AKKUŞ Evaluation For a better planning prediction of next action of an agent, in terms of predefined metrics, is important. The prediction of next action has two components: Previous experiences about the agent, Possible results that will be generated by the agent for the new problem So, a model that can be used to predict future behavior of an agent must be generated. Possible modelling options: Extrapolation,

19. Gökay Burak AKKUŞ Main Goal A model that can work both under perfect and imperfect monitoring needs to be developed for evaluation process of agents. underlying knowledge structure, previous task experiences and solution generation capabilities over a feasible time period, will be used as weights of evaluation coordination for timely samplings of goal satisfaction for multi-agent systems will be taken into account

20. Gökay Burak AKKUŞ References [1] William Hansel Turkett, Jr. (turkett@cse.sc.edu), Robust Multiagent Plan Generation and Execution with Decision-Theoretic Planners Dissertation Proposal, Department of Computer Science and Engineering University of South Carolina Columbia, SC 29208, Spring 2003turkett@cse.sc.edu [2]Gerhard Weiß (weissg@in.tum.de ), An Architectural Framework for Integrated Multiagent Planning, Reacting, and Learning Institut für Informatik, Technische Universitat München D-80290 München, Germanyweissg@in.tum.de [3] Romen I. Brafman (brafman@cs.bgu.ac.il), Moshe Tennenholtz (moshe@robotics.stanford.edu), Learning to Coordinate Efficiently: A Model Based Approach, 2003brafman@cs.bgu.ac.ilmoshe@robotics.stanford.edu [4] Michael Brenner (brenner@informatik.uni-freiburg.de), MAPL: a Framework for Multiagent Planning with Partially Ordered Temporal Plans, Institut für Informatik, Universitat Freiburg, 79110 Freiburg, Germanybrenner@informatik.uni-freiburg.de [5]Junwei Cao, Subhash Saini, Stephen A. Jarvis, Daniel P. Spooner, Helene N. Lim Choi Keung, Graham R. Nudd High Performance Systems Group, University of Warwick, Coventry, UK, Performance Prediction and its use in Parallel and Distributed Computing Systems [6]Nils J. Nilsson, Artificial Intelligence A New Sysnthesis, Stanford University, 1998 [7]Jonathan Gratch, Reasoning about Multiple Plans in Dynamic Multi-agent Environments, Information Sciences Institute University of Southern California, October 1998.

21. Gökay Burak AKKUŞ Thanks... Questions ?