1. Koen HindriksMulti-Agent Systems 2012 Introduction Agent Programming Koen Hindriks Delft University of Technology, The Netherlands Learning to program teaches you how to think. Computer science is a liberal art. Steve Jobs Multi-Agent Systems Course

2. Koen Hindriks Multi-Agent Systems 2012 2 Koen HindriksMulti-Agent Systems 2013 2 Introducing Myself Dr. Koen Hindriks –PhD on Agent Programming Languages (Utrecht, 1996 - 2000) –Consultant at Accenture (Amsterdam, 2000 - 2005) –Assistant Professor Artificial Intelligence (Nijmegen, 2005 - 2006) –Assistant Professor MMI Group (Delft, 2006 - …) Main Research Interests –Rational agent programming and theory –Automated negotiation –Logic (with applications to agent modeling and NLP semantics) –Cognitive engineering and architectures

3. Koen Hindriks Multi-Agent Systems 2012 3 Koen HindriksMulti-Agent Systems 2013 3 Outline This lecture: –Blocks, Blocks, Blocks… Basic Blocks World example in the G OAL agent programming language –Actions –Rule-Based Action Selection

4. Koen Hindriks Multi-Agent Systems 2012 4 Koen HindriksMulti-Agent Systems 2012 4 This Part: BW4T Assignment Abstract simulated environment called: Blocks World for Teams (BW4T) Search and retrieval task Control robot by cognitive agent program Your task: implement decision-making

5. Koen HindriksMulti-Agent Systems 2012 The G OAL Agent Programming Language

6. Koen Hindriks Multi-Agent Systems 2012 6 Koen HindriksMulti-Agent Systems 2012 THE BLOCKS WORLD The Hello World example of Agent Programming

7. Koen Hindriks Multi-Agent Systems 2012 7 Koen HindriksMulti-Agent Systems 2012 7 The Blocks World Positioning of blocks on table is not relevant. A block can be moved only if it there is no other block on top of it. Objective: Move blocks in initial state such that result is goal state. A classic AI planning problem.

8. Koen Hindriks Multi-Agent Systems 2012 8 Koen HindriksMulti-Agent Systems 2012 8 Mental State of G OAL Agent knowledge{ block(X) :- on(X, _). clear(X) :- block(X), not(on(Y,X)). clear(table). tower([X]) :- on(X,table). tower([X,Y|T]) :- on(X,Y), tower([Y|T]). } beliefs{ on(a,b). on(b,c). on(c,table). on(d,e). on(e,table). on(f,g). on(g,table). } goals{ on(a,e), on(b,table), on(c,table), on(d,c), on(e,b), on(f,d), on(g,table). } The knowledge, belief, and goal sections together constitute the specification of the mental state of a G OAL Agent. Initial mental state of agent

9. Koen Hindriks Multi-Agent Systems 2012 9 Koen HindriksMulti-Agent Systems 2012 9 Previous Lecture Percepts Action events actionsgoals plans beliefs environment agent

10. Koen Hindriks Multi-Agent Systems 2012 10 Koen HindriksMulti-Agent Systems 2012 10 This Lecture Percepts Action events actionsgoals plans beliefs environment agent

11. Koen Hindriks Multi-Agent Systems 2012 11 Koen HindriksMulti-Agent Systems 2012 11 Next Lecture Percepts Action events actionsgoals plans beliefs environment agent

12. Koen Hindriks Multi-Agent Systems 2012 12 Koen HindriksMulti-Agent Systems 2013 12 Why “Hello World” Example? The Blocks World is: –Single agent: no other agents that can change the world –Fully observable: the agent can see “everything” –Static: only actions of agent can change the world  We do not need sensing / percepts.

13. Koen Hindriks Multi-Agent Systems 2012 13 Koen HindriksMulti-Agent Systems 2013 13 Agent Oriented Programming Agents provide a very effective way of building applications for dynamic and complex environments + Develop agents based on Belief-Desire-Intention agent metaphor, i.e. develop software components as if they have beliefs and goals, act to achieve these goals, and are able to interact with their environment and other agents.

14. Koen Hindriks Multi-Agent Systems 2012 14 Koen HindriksMulti-Agent Systems 2012 ACTIONS SPECIFICATIONS Changing Blocks World Configurations

15. Koen Hindriks Multi-Agent Systems 2012 15 Koen HindriksMulti-Agent Systems 2012 15 Actions Change the Environment… move(a,d)

16. Koen Hindriks Multi-Agent Systems 2012 16 Koen HindriksMulti-Agent Systems 2013 16 and Require Updating Mental States. To ensure adequate beliefs after performing an action the belief base needs to be updated (and possibly the goal base). –Add effects to belief base: insert on(a,d) after move(a,d). –Delete old beliefs: delete on(a,b) after move(a,d).

17. Koen Hindriks Multi-Agent Systems 2012 17 Koen HindriksMulti-Agent Systems 2013 17 and Require Updating Mental States. If a goal has been (believed to be) completely achieved, the goal is removed from the goal base. It is not rational to have a goal you believe to be achieved. Default update implements a blind commitment strategy. move(a,b) beliefs{ on(a,table), on(b,table). } goals{ on(a,b), on(b,table). } beliefs{ on(a,b), on(b,table). } goals{ }

18. Koen Hindriks Multi-Agent Systems 2012 18 Koen HindriksMulti-Agent Systems 2013 18 Action Specifications Actions in GOAL have preconditions and postconditions. Executing an action in GOAL means: –Preconditions are conditions that need to be true: Check preconditions on the belief base. –Postconditions (effects) are add/delete lists (STRIPS): Add positive literals in the postcondition Delete negative literals in the postcondition STRIPS-style specification move(X,Y){ pre { clear(X), clear(Y), on(X,Z), not( on(X,Y) ) } post { not(on(X,Z)), on(X,Y) } }

19. Koen Hindriks Multi-Agent Systems 2012 19 Koen HindriksMulti-Agent Systems 2013 19 move(X,Y){ pre { clear(X), clear(Y), on(X,Z), not( on(X,Y) ) } post { not(on(X,Z)), on(X,Y) } } Example: move(a,b) Check: clear(a), clear(b), on(a,Z), not( on(a,b) ) Remove: on(a,Z) Add: on(a,b) Note: first remove, then add. Actions Specifications table

20. Koen Hindriks Multi-Agent Systems 2012 20 Koen HindriksMulti-Agent Systems 2013 20 move(X,Y){ pre { clear(X), clear(Y), on(X,Z), not( on(X,Y) ) } post { not(on(X,Z)), on(X,Y) } } Example: move(a,b) Actions Specifications beliefs{ on(a,table). on(b,table). } beliefs{ on(b,table). on(a,b). }

21. Koen Hindriks Multi-Agent Systems 2012 21 Koen HindriksMulti-Agent Systems 2013 21 move(X,Y){ pre { clear(X), clear(Y), on(X,Z), not( on(X,Y) ) } post { not(on(X,Z)), on(X,Y) } } 1.Is it possible to perform move(a,b) ? 2.Is it possible to perform move(a,d) ? Actions Specifications EXERCISE: knowledge{ block(a), block(b), block(c), block(d), block(e), block(f), block(g), block(h), block(i). clear(X) :- block(X), not(on(Y,X)). clear(table). tower([X]) :- on(X,table). tower([X,Y|T]) :- on(X,Y), tower([Y|T]). } beliefs{ on(a,b). on(b,c). on(c,table). on(d,e). on(e,table). on(f,g). on(g,table). } No, clear(b) fails. Yes.

22. Koen Hindriks Multi-Agent Systems 2012 22 Koen HindriksMulti-Agent Systems 2013 22 move(X,Y){ pre { clear(X), clear(Y), on(X,Z), not( on(X,Y) ) } post { not(on(X,Z)), on(X,Y) } } We already check clear(Y). Why do we also have not(on(X,Y)) in the precondition? Actions Specifications EXERCISE: Since we always have clear(table). The condition prevents moving blocks that are already on the table. knowledge{ block(a), block(b), block(c), block(d), block(e), block(f), block(g), block(h), block(i). clear(X) :- block(X), not(on(Y,X)). clear(table). tower([X]) :- on(X,table). tower([X,Y|T]) :- on(X,Y), tower([Y|T]). } beliefs{ on(a,b). on(b,c). on(c,table). on(d,e). on(e,table). on(f,g). on(g,table). }

23. Koen Hindriks Multi-Agent Systems 2012 23 Koen HindriksMulti-Agent Systems 2012 ACTION RULES Selecting actions to perform

24. Koen Hindriks Multi-Agent Systems 2012 24 Koen HindriksMulti-Agent Systems 2013 24 Agent-Oriented Programming How do humans choose and/or explain actions? Examples: I believe it rains; so, I will take an umbrella with me. I go to the video store because I want to rent I-robot. I don’t believe busses run today so I take the train. Use intuitive common sense concepts: beliefs + goals => action See Chapter 1 of the Programming Guide

25. Koen Hindriks Multi-Agent Systems 2012 25 Koen HindriksMulti-Agent Systems 2013 25 Selecting Actions: Action Rules Action rules are used to define a strategy for action selection. Defining a strategy for blocks world: –If constructive move can be made, make it. –If block is misplaced, move it to table. program{ if bel(tower([Y|T])), a-goal(tower([X,Y|T])) then move(X,Y). if a-goal(tower([X|T])) then move(X,table). }

26. Koen Hindriks Multi-Agent Systems 2012 26 Koen HindriksMulti-Agent Systems 2013 26 Selecting Actions: Action Rules What happens? Check whether condition a-goal(tower([X|T]) can be derived from the current mental state of agent. If so, then apply rule and perform move(X,table). if a-goal(tower([X|T])) then move(X,table).

27. Koen Hindriks Multi-Agent Systems 2012 27 Koen HindriksMulti-Agent Systems 2013 27 Selecting Actions: Action Rules What happens? if a-goal(tower([X|T])) then move(X,table). knowledge{ block(X) :- on(X, _). clear(X) :- block(X), not(on(Y,X)). clear(table). tower([X]) :- on(X, table). tower([X,Y|T]) :- on(X,Y), tower([Y|T]). } beliefs{ on(a, b). on(c, d). } goals{ on(a, table), on(c, table). } d c b a EXERCISE: Random choice for either moving a or c.

28. Koen Hindriks Multi-Agent Systems 2012 28 Koen HindriksMulti-Agent Systems 2013 28 Order of Action Rules Action rules are executed by default in linear order. The first rule that fires is executed. Default order can be changed to random. Arbitrary rule that is able to fire may be selected. program{ if bel(tower([Y|T])), a-goal(tower([X,Y|T])) then move(X,Y). if a-goal(tower([X|T])) then move(X,table). } program[order=random]{ if bel(tower([Y|T])), a-goal(tower([X,Y|T])) then move(X,Y). if a-goal(tower([X|T])) then move(X,table). }

29. Koen Hindriks Multi-Agent Systems 2012 29 Koen HindriksMulti-Agent Systems 2013 29 Example Program: Action Rules Agent program may allow for multiple action choices d To table Random, arbitrary choice program[order=random]{ if bel(tower([Y|T])), a-goal(tower([X,Y|T])) then move(X,Y). if a-goal(tower([X|T])) then move(X,table). }

30. Koen Hindriks Multi-Agent Systems 2012 30 Koen HindriksMulti-Agent Systems 2013 30 The Sussman Anomaly (1/5) Non-interleaved planners typically separate the main goal, on(A,B),on(B,C) into 2 sub-goals: on(A,B) and on(B,C). Planning for these two sub-goals separately and combining the plans found does not work in this case, however. a c Initial state bc b a Goal state

31. Koen Hindriks Multi-Agent Systems 2012 31 Koen HindriksMulti-Agent Systems 2013 31 The Sussman Anomaly (2/5) Initially, all blocks are misplaced One constructive move can be made (c to table) Note: move(b,c) not enabled. Only action enabled: c to table (2x). Need to check conditions of action rules: if bel(tower([Y|T]),a-goal(tower([X,Y|T))then move(X,Y). if a-goal(tower([X|T))then move(X,table). We have bel(tower([c,a]) and a-goal(tower([c])). c b a Goal state a c Initial state b

32. Koen Hindriks Multi-Agent Systems 2012 32 Koen HindriksMulti-Agent Systems 2013 32 The Sussman Anomaly (3/5) Only constructive move enabled is –Move b onto c Need to check conditions of action rules: if bel(tower([Y|T]), a-goal(tower([X,Y|T))then move(X,Y). if a-goal(tower([X|T))then move(X,table). Note that we have: a-goal(on(a,b),on(b,c),on(c,table)), but not: a-goal(tower[c])). Current state c b a Goal state a c b

33. Koen Hindriks Multi-Agent Systems 2012 33 Koen HindriksMulti-Agent Systems 2013 33 The Sussman Anomaly (4/5) Again, only constructive move enabled –Move a onto b Need to check conditions of action rules: if bel(tower([Y|T]), a-goal(tower([X,Y|T))then move(X,Y). if a-goal(tower([X,T))then move(X,Y). Note that we have: a-goal(on(a,b),on(b,c),on(c,table)), but not: a-goal(tower[b,c]). c b a Goal state a c b Current state

34. Koen Hindriks Multi-Agent Systems 2012 34 Koen HindriksMulti-Agent Systems 2013 34 The Sussman Anomaly (5/5) Upon achieving a goal completely that goal is automatically removed. The idea is that no resources should be wasted on achieving the goal. In our case, goal(on(a,b),on(b,c),on(c,table)) has been achieved, and is dropped. The agent has no other goals and is ready. c b a Goal state a c b Current state

35. Koen Hindriks Multi-Agent Systems 2012 35 Koen HindriksMulti-Agent Systems 2013 35 Organisation Read Programming Guide Ch1-3 (+ User Manual) See: http://ii.tudelft.nl/trac/goal#Documentationhttp://ii.tudelft.nl/trac/goal#Documentation Tutorial: –Download G OAL : See http://ii.tudelft.nl/trac/goalhttp://ii.tudelft.nl/trac/goal –Practice exercises from Programming Guide –BW4T assignments 3 and 4 made available today Next lecture: –Sensing, perception, environments –Other types of rules & macros –Agent architecture