1. Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Multi-agent Systems, Communication and Coordination Koen Hindriks Delft University of Technology, The Netherlands Multi-Agent Systems Course

2. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen G OAL So Far mental states (beliefs, goals) mental state conditions & macros action specifications, built-in actions action rules environment & perception modules

3. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Outline (today) Intro Multi-Agent Systems (MAS) Coordination in MAS Specifying a MAS in G OAL Communication in G OAL

4. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Multi-Agent Systems (MAS)

5. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Multi-Agent System Environment A1 Agents A2 A3 A4 Intersecting spheres of influence of multiple agents acting in an environment Intersecting spheres: Action perspective: may interfere, change same parts of environment Perception perspective: agents may have different views on environment

6. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Multi-Agent System Environment A1 Agents A2 A3 A4 Various approaches to coordinate activity of multiple agents Various coordination techniques have been proposed, main ideas: Communication protocols impose structure on agent communication Add organization structure on top of multi-agent system Note: Without explicit agent communication still implicit communication possible via environment if perception spheres overlap

7. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Multi-Agent System Environment A1 Agents A2 A3 A4 Not all agents need to be ‘connected’ to the environment

8. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Kinds of multi-agent systems Cooperative versus competitive –Agents have a common goal of solving a problem together or compete for resources. Homogeneous versus heterogeneous –All agents have the same abilities and decision logic, or agents have different abilities. Interaction protocols –Which agents can talk to which other agents? Organizational structure –Agents may be organized in various ways (with a leader, or not, different roles, etc.)

9. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Example – Search and Rescue Search & Rescue –Unknown environment (map may be unreliable or absent) –Search for victims –Providing aid to victims –Aim to minimize time to provide aid to all victims.

10. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Example – Package Delivery Package delivery –Known environment (map) –Initial and target location of packages known –Aim to minimize delivery time of all packages.

11. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Advantages of MAS Complete tasks quicker through shared effort. Achieve tasks otherwise not possible. A MAS provides solutions in situations where action is spatially and temporally distributed. A MAS can be distributed over machines across a network. A decentralized MAS does not suffer from the "single point of failure" problem.

12. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Cooperation & Coordination Maaike Harbers,

13. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Examples of Cooperation http://cacm.acm.org/news/98755-how-football-playing-robots-have-the-future-of-ai-at-their-feet/fulltext Maaike Harbers,

14. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Cooperation Challenges Cooperation yields more effective (e.g., quicker) solutions to a problem… But… 1.Avoid duplication of effort. 2.Avoid interference, i.e. harmful interactions. 3.Avoid communication overhead, i.e., need to share information with as few messages as possible. 4.Need to synchronize behaviors. Coordination needed to address challenges

15. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen What is coordination?

16. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Coordination problems Without coordination, what would happen here? What is the problem: duplication of effort, interference, communication overhead, synchronization?

17. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Coordination problems http://www.wired.com/gadgetlab/2010/06/dance-dance-revolution-20-robots-think-they-can-dance/ And what would happen here? What is the problem: duplication of effort, interference, communication overhead, synchronization?

18. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Coordination problems And what would happen here? What is the problem: duplication of effort, interference, communication overhead, synchronization?

19. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Coordination problems Player A / Player B DefectCooperate Defect 2, 2 5, 0 Cooperate 0, 53, 3 Outcomes for actors A and B depend on the combination of A's and B's action. Prisoner's dilemma Two persons are being questioned in separate rooms by the police. Each of them can either admit to a crime (defect) or not (cooperate). And what would happen here? What is the problem: duplication of effort, interference, communication overhead, synchronization?

20. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Need to be smart And what would happen here? What is the problem: duplication of effort, interference, communication overhead, synchronization? The wise men problem A king wishing to know which of his three wise men is the wisest, paints a white spot on each of their foreheads, tells them at least one spot is white, and asks each to determine the color of his spot. …

21. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Types of coordination Implicit coordination e.g., flocking behavior Explicit coordination exchange of messages

22. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Klein, G., Feltovich, P., Bradshaw, J. M., & Woods, D. D. (2005). Common ground and coordination in joint activity. Organizational Simulation. W. B. Rouse and K. R. Boff. New York City, NY, John Wiley. Aspects of joint activity Coordinating behavior makes multiple agents act like a team, they perform a joint activity.

23. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen A. Criteria: example Making an assignment together No intention – no assignment or two assignment No interdepencence – each making one half of the assignment without seeing the other half

24. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Interpredictability –Need to be able to accurately predict what others will do –Not just actions, but the coordination itself Common ground –Pertinent mutual knowledge, beliefs, and assumptions about others’ skills and capabilities –Joint activity is guided by signaling and coordination devices Directability –Capacity for modifying the actions of the other parties as conditions and priorities change –Responsiveness of each participant to the influence of the others B. Requirements for joint activity

25. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Football team Interpredictability – predict where team members will be on the field and who will be available Common ground – who is keeper, who defends, who attacks Directability – keeper tells others where to stand during a corner B. Requirements: example

26. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen C. Choreography of joint activity Three phases (functional breakdown) coordinating tasks is coordinating subtasks subtasks have an entry, body and exit phase Signaling transitions within and between phases Coordination devices serve to increase interpredictability examples are organizational structure, convention, communication protocol

27. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen C. Choreography: example Traffic Three phase actions -entering a road -being on the road -leaving a road Signaling transitions -turn signal Coordination devices -organizational structure, convention, communication protocol -to discuss…

28. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Coordination structures

29. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Organizational structure adapted from Bradshaw et al.

30. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Organizational structure in BW4T Notion of ‘role’ is central Possible roles in BW4T? searcher deliverer manager

31. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Notion of ‘role’ is central organization goals agents agent roles determine performed by Organizational perspective society agent roles agent goals determine performed within Agent perspective

32. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Conventions Coordination by convention solution

33. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen A group of agents Each agent is able to solve the BW4T task individually Agents do not take into account that there are other agents What happens? Conventions in BW4T

34. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Do not enter rooms that are occupied If you know about a block that can be delivered, deliver that block If someone intends to deliver a particular block, do not try to deliver that block as well Example of conventions in BW4T

35. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Conventions in organizational structure If a searcher finds a block, it must report that to the manager adapted from Bradshaw et al.

36. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen A group of agents Each agent is able to solve the BW4T task individually Agents take into account that there are other agents Agents do not communicate What happens? Communication in BW4T

37. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Implicit coordination in BW4T Room 1 Room 2 Robot 1 Robot 2 And what would happen here? Robots do NOT communicate. How do they choose a room?

38. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Specifying a MAS

39. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen MAS Specification In G OAL, a.mas file is a recipe for launching a multi-agent system, specifying: which environment will be launched, how many agents will be launched, the agent names for agent processes which agents are connected to environment which.goal file(s) is (are) used to create an agent.

40. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen MAS Specification A.mas file is a recipe for launching a MAS: which environment will be launched environment started by jar file, some of which may be initialized using the init command (check out environment doc). environment{ env = "blocksworld.jar". init = [configuration="bwconfigEx1.txt"]. }

41. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen MAS Specification A.mas file is a recipe for launching a MAS: which agent files will be used agent file defines a possible agent type. agentfiles { "coffeemaker.goal". "coffeegrinder.goal". }

42. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen MAS Specification A.mas file is a recipe for launching a MAS: which agent processes will be created a launch policy specifies when, which and how many agent processes will be created. launch maker:coffeemaker. launches one agent process called maker using the agent file referenced by coffeemaker (coffeemaker.goal). does not connect agent to an environment. launchpolicy { launch maker:coffeemaker. launch grinder:coffeegrinder. }

43. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen MAS Specification A.mas file is a recipe for launching a MAS: which agent processes will be created a conditional launch rule is triggered by the environment: whenever a controllable entity becomes available in the environment, an event is generated. the agent created by the launch part of the rule is connected to that entity. launchpolicy { when entity@env do launch elevator : file1. }

44. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen MAS Specification A.mas file is a recipe for launching a MAS: additional checks in launch rule this launch rule can only be applied if there is an available entity of type car, and the rule can be applied at most 3 times. Remark: launch rules are applied in linear order. launchpolicy { when [type=car,max=3]@env do launch elevator:file1. }

45. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen MAS Specification: Example A.mas file is a recipe for launching a MAS: environment section may be empty, in that case mas is run without an environment. agentfiles and launchpolicy must be non-empty. agentfiles { "coffeemaker.goal". "coffeegrinder.goal". } launchpolicy { launch maker:coffeemaker. launch grinder:coffeegrinder. }

46. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Agent Identity and Other Agents How does an agent know it is not alone? The names of all agents generated by G OAL are inserted in the belief base of all other agents. For example, agent(tarzan) is inserted in the belief base of agent jane when tarzan is created. How is an agent able to differentiate itself from other agents? I.e. how does agent jane know she is not tarzan? Upon creation, the fact me( ) where name is the agent’s name is inserted in the belief base of that agent.

47. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Communication in G OAL

48. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Communication: Primitives Communication primitives: –send(, ) –sendonce(, ) Example: Tarzan sends Jane a message: –send(jane, meeting(date(1,2,2010), time(12,0), duration(1,0), [jane, tarzan]))

49. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen GOAL Agent Architecture The basis of communication in GOAL is a simple mailbox semantics. Agent Environment Real or simulated world of events action percept Beliefs Goals Action Rules Process percepts message Process message

50. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Communication: Semantics Action: send(, ) ground send action can always be performed, i.e. precondition is true, assume agent sender sends message to recipients then GOAL puts sent(, ) in the mailbox of sender, agents that receive message put received(sender, ) in their mailbox. NB: keep in mind it may take some time before agent b receives the message.

51. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Communication: Semantics Action: sendonce(, ) same as send(, ), but sendonce action can only be performed if the mailbox of the sending agent does not contain: sent(, ) i.e. precondition is: not(sent(, )

52. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Using Messages in Mailbox The facts in the mailbox can be used like any other facts in the belief base of an agent. That is, sent(, ) and received(name, ) can be used in: –in conditions in action rules –c lauses in the knowledge or beliefs section

53. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Using Messages in Mailbox After receiving a message from tarzan, agent jane will have: received(tarzan, meeting(date(1,2,2010), time(12,0), duration(1,0), [jane,tarzan])) in her mailbox. Consequently, jane will believe: bel(received(tarzan, meeting(date(1,2,2010), …, …, [jane,tarzan])))

54. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Processing Message Pattern Inserting received information into belief base and removing received message: Removing received message cleans up the mailbox and makes sure that the action rule is not applied repeatedly (even after the received info is outdated). Using received messages in mental state conditions other than for processing the message, and using them in knowledge base, gives less readable code. if bel(received(A,Proposition)) then insert(Proposition) + delete(received(A,Proposition))

55. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Using Messages in Mailbox More complex conditions may be defined. For example: #define not_yet_invited(Person,Date,Time,Duration,Attendees) goal(meeting(Date, Time, Duration, Attendees)), bel(member(Person, Attendees), me(Me), not(Person = Me)), not(bel(sent(Person, meeting(Date,Time,Duration, Attendees)))). which informally means: I want to have a meeting with Attendees, Person is one of the Attendees (different from myself), and I did not send that person an invitation for the meeting I want yet.

56. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Message Types GOAL supports three message types, called moods: –declarative, typically used to inform Example: The time is 2 o'clock operator: ‘:’ : send(jane, :time(14,0)) –imperative, typically used to indicate a goal Example: I want the door to be closed! operator: ‘!’ : send(jane, !door(closed)) –interrogative, typically used to ask a question Example: How much milk is left? operator: ‘? : send(jane, ?amountMilk(_))

57. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Message Types Mood operators are translated to Prolog predicates in mailbox: –declarative: no predicate received(tarzan, time(14,0)) –imperative: imp received(tarzan, imp(door(closed))) –interrogative: int received(tarzan, int(amountMilk(_))) See programming guide, section 6.5.

58. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Agent Selectors First argument of a send action is called an agent selector, which can be: An agent name: send(jane, …) A variable (to be instantiated): send(Person, …) A list of selectors: send([jane,tarzan,Person], …) A quantor: –send(self, …) : message sent to agent itself –send(all, …) : message sent to all agents (incl. itself) –send(allother, …) : message sent to all other agents

59. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Sending messages to oneself Messages sent to oneself change the agent’s own mental state! Sending messages to oneself has special meaning. Changes are different for different moods: –send(self, :fact) inserts fact in belief base –send(self, !fact) adopts fact as goal and deletes fact from belief base. –send(self, ?fact) deletes fact from belief base N.B.: Do not use messages to self as an alternative way of updating the agent’s beliefs and goals!

60. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Selecting agents to send to Suppose agent wants to send to all other agents that are producers of milk that it wants to have milk. Q: Write an action rule which achieves this: –Use predicates have/1 and producerOf/2. A: if a-goal(have(milk)), bel(agent(X), producerOf(X, milk)) then send(X, !have(milk))

61. Koen Hindriks, Birna van Riemsdijk Multi-agent systemen Koen Hindriks, Birna van RiemsdijkMulti-agent systemen Organisation Tutorial this week: –Assignment 4: team agent for BW4T Exam: –Specification of material to study will be posted on blackboard. –Example exams will be made available.