1 CO Games Development 2 Week 17 – Goal-Oriented Action Planning Gareth Bellaby

2 Introduction ●In AI literature tend to just use the word "planning". ●In games the phrase "Goal-Oriented Planning" (GOAP) is common. ●One way to think about a planning system is a version of a FSM. ●Like a FSM it uses states. However, the boxes represent actions whilst states are the input into, and output from, an action.

3 STRIPS ●STRIPS (Stanford Research Institute Problem Solver) ●Formal language developed by Richard Fikes and Nils Nilsson. ●"STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving" published in ●Assumes that all conditions not stated to be true are false.

4 Importance ●Used in a number of recent games. ●F.E.A.R. Demonstrated its commercial importance. The AI in F.E.A.R. is highly regarded. It was perceived to have aided the success of the game. ●F.E.A.R. 2: Project Origin. ●Fallout 3. ●Silent Hill: Homecoming. ●Empire: Total War. ●Not a comprehensive list.

5 Justification ●The search techniques we've looked at are not useful for certain types of problems because ●large branching factor ●problem of decomposing some problems into discrete states ●finding an efficient heuristic function

6 Planning ●Planning is the process of divising a sequence of actions to achieve a goal. ●Search techniques such as pathfinding are an example of planning. ●Could try to deal with problems as they arise but this is inefficient. Get stuck in blind alleys. Having to repeat actions. Having to backtrack. Planning turns out to be a far better approach.

7 Language ●Using actions, states and goals. ●Need a language for representing problems. ●The process is one of expressing the world as logical conditions. ●Propositional logic. ●Statements. ●First order representations of actions. ●Logical structure.

8 Example ●Consider the problem of moving from A to B. ●The start state is being at location A. ●The goal state is being at location B. ●The action is movement from A to B. ●The problem is described using logical statements. ●In order to move from A to be you would have to be at location A. ●If you have moved to location B you would now be at location B. ●How else to represent this?

9 State ●Expressed as logical statements, e.g. ●At( B ) ● Logical statements can be combined together, e.g. ●At( door ) AND holding( key ) ●If you look at literature you will see that it tends to use logical symbols: ●At( door ) ^ holding( key )

10 State ●Formally the state is a conjunction of literals. ●Propositional literals. ●First-order literals. ●The literals are function-free ●So can't have: ●At( open( door ) )

11 Goal ●A goal is expressed as a state. ●For example, ●At( B )

12 Actions ●An action is specified in terms of preconditions and effects ●For example, move from A to B ●Move(A, B) ●Preconditions: At(A) ●Postconditions: not At(A), At(B)

13 Actions ●Every literal not explicitly referred to in the effect remains unchanged. ●The postcondition has to state the negation of the precondition in order to remove it from the list. ●Uses a list

14 Reasoning ●Precondition is the entry state. ●Postcondition is the exit state. ●The exit state and entry state are logical statements. ●An action takes the system from an entry state to an exit state. An action is expressed in terms of a change in state of the system. ●The system as a whole answers a question.

15 Example ●Want to go from point A to point B. ●Unfortunately there is a closed and locked door in the way. ●There are three actions, each leading to the logical statement "The door is open". ●The door can be unlocked and opened if you have the key. ●The door can be picked open if you have the lockpicks. ●The door can be destroyed if you the explosives.

16 Example ●The goal state is ●At( B ) ●The Move action is: ●Move(A, B) ●Preconditions: At(A) ●Postconditions: not At(A) ^ At(B)

17 Unlocking the door ●Door is closed and locked. ●Player has a key ●Action is to unlock the door thus opening it ●Unlock(door) ●Preconditions: holding( key ) ^ At( A ) ^ door( closed ) ●Postconditions: door( open )

18 Picking the lock ●Door is closed and locked. ●Player has lockpicks ●Action is to Pick lock thus opening it. ●Pick(door) ●Preconditions: holding(lockpicks ) ^ At( A ) ^ door( closed ) ● Postconditions: door( open )

19 Destroying the door ●Player has explosives ●Action is to destroy the door thus opening it. ●BlowUp(door) ●Preconditions: holding(explosives ) ^ At( A ) ^ door( closed ) ● Postconditions: door( open )

20 3 ways to open the door

21 Opening the door

22 Goal state

23 Combining actions

24 Analysis ●Notice the way in which this is different from a FSM. There are no specified links. The system merely attaches preconditions to postconditions. ●Need to specific the representation of the system. ●Break the problem into a set of discrete actions and logical statements. ●Notice the way that I don't have to specify that the door is unlock (picked, destroyed) because it is implicit in the change of state from closed to open.

25 Scenario ●If the door was just open then the agent would simply go through it. ●If the door was closed then select the solution which is applicable. ●Obvious to see how it could be extended, e.g. the agent doesn't have the key but they ke is held by a guard, or the agent doesn't have the explosive but it can be found in the ammo safe. ●And further... the key can be obtained if the guard is killed... or unconscious... need a gun... or a the knockout gas... and so on.

26 Advantages ●Flexible. If the door was just open then the agent would simply go through it. Could supply one of the objects. Could chain with other actions to create multiple scenarios. In this way we can specify the language of moving from one location to another and of opening doors. ●Efficient. ●Multiple agents. ●Dynamic: system can change with the agent inside in it.

27 Forwards and backwards chaining ●There are multiple forms of branching. Could fan out, could fan in, multiple routes, etc. ●Multiple paths, some leading to dead ends. ●Multiple solutions (some more efficient than others). ●Need to employ backwards and forwards chaining algorithms.

28 Forwards and backwards chaining ●Forwards chaining - moving forward from the start state, chaining postconditions onto preconditions. ●Backwards chaining - moving backwards from the goal state, chaining preconditions onto postconditions.

29 Efficiency ●Multiple solutions. How to gauge which to use? ●Mentioned efficiency. ●Could simply count the number of steps in the solution (number of links). ●However, some actions could be difficult than others. ●Perhaps ascribe a complexity score to the actions. ●Employ a search algorithm.

30 Extensions ●Planning within the context of deterministic, observable, finite systems. ●Change occurs when the agent acts. ●There are extensions which bring in nondeterministic systems. ●The planning languages used have been systematised within a common syntax. This is called the Planning Domain Definition Language.

31 References ●The following URL has lots of relevant articles, including key ones by Fikes and Nilsson. ● ●Russell and Norvig, Atificial Intelligence, Chapter 11. ●"Command Hierarchies Using Goal-Oriented Action Planning", David Pittman, AI Game Programming Wisdom 4. ●"Applying Goal-Oriented Action Planning to Games", Jeff Orkin, AI Game Programming Wisdom 2. ●"Implementing Practical Planning for Game AI", Jamie Cheng, Game Programming Gems 5.

32 References ●Jeff Orkin ● ●Good collection of resources. Includes his own: ●"3 States and a Plan: The AI of F.E.A.R.", GDC ●"Agent Architecture Considerations for Real-Time Planning in Games", AIIDE ●"Symbolic Representation of Game World State: Toward Real-Time Planning in Games", AAAI Challenges in Game AI Workshop ●"Applying Goal-Oriented Planning for Games", draft for AI Game Programming Wisdom 2. ●Also other useful links.