1. peer-to-peer and agent-based computing Basic Theory of Agency (Contd)

2. 2 Imagine two environmental states e 1,e 2 Env, such that – e 1 e 2 and – see (e 1 ) = see (e 2 ) Such environments are different, but as far as agents are concerned they are indistinguishable! –The environments are mapped to the same percept –The agent would receive the same perceptual information from different environment states –Agents cannot tell them apart… Indistinguishable environments (1)

3. Environment states consisting of –Boolean variable temp set to true if the temperature of the room is above 35 o Celsius (otherwise temp is false) –Boolean variable smoke set to true if smoke has been detected in the environment (otherwise smoke is false) The full set of environment states is { temp, smoke, temp, smoke, temp, smoke, temp, smoke } The see function of a thermostat agent is: 3 Indistinguishable environments (2) e1e1 e2e2 e3e3 e4e4 see (e) = p 1 if e =e 1 or e =e 2 p 2 if e =e 3 or e =e 4

4. 4 Agents with State (1) Now, consider agents that record information about environment state and history:

5. Let I be the set of all internal states. The perception function see is unchanged: see : E Per The action selection function is now a mapping action : I Ac from internal states to actions. An additional function next is introduced, which maps an internal state and percept to an internal state: next : I Per I 5 Agents with State (2)

6. 6 Agent Control Loop 1.Agent starts in some initial internal state i 0 2. i cur = i 0 3.Agent observes the environment state e and generates a percept see (e ) 4.Agents internal state is updated via next function i next = next (i cur,see (e )) 5.Action selected by agent is action (next (i cur,see (e )) This action is then performed 6.Go to 3 with i cur = i next

7. 7 Tasks for Agents We build agents to carry out tasks for us. The task must be specified… However, we want to tell agents what to do without telling them how to do it. So, the question is: –how do we give an agent information about what states are better than others?

8. Possibility: associate utilities with individual states –Agents pursue states that maximise utility. A task specification is a function u : E which associates a real number with every state. 8 Utility Functions over States (1)

9. 9 Utility Functions over States (2) But what is the value of a run? Is it –The minimum utility of all states in a run? –The maximum utility of all states in a run? –The sum of utilities of all states in a run? –The average of utilities of all states in a run? Disadvantage: –It is difficult to specify a long term view when assigning utilities to individual states!

10. Another possibility – assign utilities to runs rather than individual states: u : R This takes an inherently long term view. Other variations: –Incorporate probabilities of different states emerging Difficulties with utility-based approaches: –Where do the numbers come from? –We dont think in terms of utilities! –Hard to formulate tasks in these terms… 10 Utilities over Runs

11. Simulated 2D grid environment with –Agents –Tiles –Obstacles and holes Agents can move up, down, left or right –If agent is next to a tile, agent can push tile. Holes have to be filled up with tiles by the agent to score points. Aim: score lots of points!! Holes appear and disappear at random… Utility function of a run r : u (r ) = 11 Utility in the TILEWORLD number of holes filled in r number of holes that appeared in r def

12. P (r |Ag,Env ) denotes probability that run r occurs when agent Ag is placed in environment Env : P (r |Ag,Env ) = 1 The optimal agent Ag opt in an environment Env is the one that maximises expected utility : Ag opt = arg max u (r ) P (r |Ag,Env)(1) r R(Ag,Env) Ag AG 12 Expected Utility & Optimal Agents r R(Ag,Env)

13. Some agents cannot be implemented on some computers; –E.g. a function Ag : R E Ac may need more memory than what is available to implement. AG m denotes those agents that can be implemented on machine (computer) m: AG m = {Ag | Ag AG and Ag can be implemented on m} We can replace equation (1) with the following, which defines the bounded optimal agent Ag opt : Ag opt = arg max u (r ) P (r |Ag,Env) (2) 13 r R(Ag,Env) Ag AG m Bounded Optimal Agents

14. A special case of assigning utilities to histories is to assign 0 (false) or 1 (true) to a run –If a run is assigned 1, then the agent succeeds on that run –Otherwise it fails. We call these predicate task specifications We denote a predicate task specification as : : R {0,1} 14 Predicate Task Specifications

15. A task environment is a pair Env,, where Env is an environment, and : R {0,1} is a predicate over runs. Let TE be the set of all task environments. A task environment specifies: –The properties of the system the agent will inhabit –The criteria by which an agent will be judged to have either failed or succeeded 15 Task Environments (1)

16. R (Ag, Env ) denotes the set of all runs of agent Ag in environment Env that satisfy : R (Ag, Env ) = {r | r R (Ag, Env ) (r ) = 1} We then say that an agent Ag in task environment Env, is successful if R (Ag, Env ) = R (Ag, Env ) 16 Task Environments (2)

17. Let P (r | Ag, Env ) denote the probability that run r occurs when agent Ag is placed in environment Env. The probability P ( |Ag,Env ) that is satisfied by Ag in Env would be: P ( |Ag,Env) = P (r |Ag,Env ) 17 The Probability of Success r R (Ag,Env )

18. 18 Achievement & Maintenance Tasks (1) Most common tasks in agent-based computing are – Achievement tasks: achieve state of affairs – Maintenance tasks: maintain state of affairs

19. 19 Achievement & Maintenance Tasks (2) An achievement task is specified by a set G of good or goal states G E : –An agent succeeds if it is guaranteed to bring about at least one of these states –We do not care which one – they are all considered equally good A maintenance goal is specified by a set B of bad states B E : –An agent succeeds in a particular environment if it manages to avoid all states in B, i.e., it never acts so that a state in B occurs.

20. 20 Agent Synthesis Agent synthesis is automatic programming: –The goal is to have a program that will take a task environment and –Automatically generate an agent that succeeds in this environment: syn : TE (Ag { }) (Think of as being like null in Java) Common agent synthesis mechanism: genetic programming –E.g., in the evolution of simple predator and prey agents.

21. A synthesis algorithm is sound –If it returns an agent, this agent succeeds in the task environment passed as input –I.e., the algorithm satisfies the condition: [syn ( Env, ) = Ag] [R (Ag, Env ) = R (Ag, Env )] A synthesis algorithm is complete –if it is guaranteed to return an agent whenever there exists an agent that will succeed in the task environment given as input; –I.e., the algorithm satisfies the condition: Ag AG.[R (Ag, Env ) = R (Ag, Env )] [syn ( Env, ) ] 21 Agent Synthesis (2)

22. Multi-Agent Systems (CS5562) – Wamberto Vasconcelos 22 Suggested Reading An Introduction to Multi-Agent Systems, M. Wooldridge, John Wiley & Sons, 2002. Chapter 2.