1. Automated Planning & Computer Games: Perspectives and Applications
Hector Munoz-Avila

2. General-Purpose Planning: State & Goals
Initial state: (on A Table) (on C A) (on B Table) (clear B) (clear C)
Goals: (on C Table) (on B C) (on A B) (clear A)
Initial state
Goals C B A B C
(Ke Xu)

3. General-Purpose Planning: Operators
No block on top of ?x
No block on top of ?y nor ?x ?x ?y
transformation ?y ?x … …
On table
Operator: (Unstack ?x)
Preconditions: (on ?x ?y) (clear ?x)
Effects:
Add: (on ?x table) (clear ?y)
Delete: (on ?x ?y)

4. Planning: Search SpaceB C A B C B A B A C B A C B A B C C B A B A C A C A A B C B C C B A A B C
(Michael Moll)

5. Some Examples
Which of the following problems can be modeled as AI planning problems?
Route search: Find a route between Lehigh University and the Naval Research Laboratory
Project management: Construct a project plan for organizing an event (e.g., the Musikfest)
Military operations: Develop an air campaign
Information gathering: Find and reserve an airline ticket to travel from Newark to Miami
Game playing: plan the behavior of a computer controlled player
Resources control: Plan the stops of several of elevators in a skyscraper building.
Answer: ALL!

6. Classes of General-Purpose Planners
General purpose planners can be classified according to the space where the search is performed:
We are going to discuss these forms
Hierarchical
Disjunctive plans
state
plan
SAT

7. State- and Plan-Space Planning
State-space planners transform the state of the world. These planners search for a sequence of transformations linking the starting state and a final state
State of the world
(total order)
Plan-space planners transform the plans. These planners search for a a plan satisfying certain conditions
(partial-order, least-commitment)

8. Why Plan-Space Planning?
1. Motivation: “Sussman Anomaly”
Two subgoals to achieve:
(on A B) (on B C) A C B A B C

9. Why Plan-Space Planning?
Problem of state-space search:
Try (on A B) first:
put C on the Table, then put A on B
Accidentally wind up with A on B when B is still on the Table
We can not get B on C without taking A off B
Try to solve the first subgoal first appears to be mistaken A A B C A B C B C

10. But… Does Automated Planning has any use in Computer Games?
F.E.A.R. not!
Intro:
Combat sequence

11. Implementation Points in FEAR
Goal
Three categories of gaming goals
Relaxed
Investigative
Aggressive
Planner Search: state-space planning
World Representation: compiled in a data structure
Planner Optimization: compiled operators, heuristics (factoring the gaming goals)

12. Automated Planning: Uses in Games
AI paradigm used to implement opponent’s behavior in FEAR (see previous class)
Automated Planning is used to gain flexibility in designing Game AI behavior
An action:
Attack
positioned
Weapon ready
Enemy attacked
effects
preconditions

13. Example positioned Enemy attacked Attack Weapon is ready positioned
FEAR defines 30+ such actions. So could generate many plans.
How else could we use to create an opponent AI?
Attack
positioned
Weapon is ready
Enemy attacked
Stay-put
positioned
Take cover
Prepare Weapon
Ammo loaded
Holding weapon
Weapon is ready
How many plans can we generate to achieve Enemy attacked?
FSMs
(Project 1)

14. Optimization: Representation
Declared as a C++ procedure:
Action Attack()
{ if (positioned && WeaponISReady)
EnemyAttacked = true; }
Attack
positioned
Weapon is ready
Enemy attacked
Declared as a variable in standard C++:
Boolean EnemyAttacked;
Discussion points:
Complexity versus an standard AI planner
So why AI planners do not adopt this?

15. Optimization: Search C A B C A B C B A B A C B A C B A B C C B A B A C
(Michael Moll)

16. Implementation Continued

17. Implementation Continued
Operators are represented as code
Allows fast running time evaluation

18. FSM vs AI Planning (for details see Stanislav’s presentation) FSM:
Planning Operators
Patrol
Preconditions:
No Monster
Effects:
patrolled
Fight
Monster in sight
(for details see Stanislav’s presentation)
Patrol
Fight
Monster In Sight
No Monster
FSM:
A resulting plan:
Patrol
patrolled
Fight
No Monster
Monster in sight
Neither is more powerful than the other one

19. Planning Gives More Flexibility But Developer May Loose Control
“Separates implementation from data” --- Orkin
reasoning
knowledge
Many potential plans:
Patrol
Fight …
Planning Operators
Patrol
Preconditions:
No Monster
Effects:
patrolled
Fight
Monster in sight …
If conditions in the state change making the current plan unfeasible: replan!

20. Hierarchical (HTN) Planning
Principle: Complex tasks are decomposed into simpler tasks. The goal is to decompose all the tasks into primitive tasks, which define actions that change the world.
Travel(UMD, Lehigh)
Travel from UMD to Lehigh University
Fly(National, L.V. International)
Travel(L.V. Int’nal,Lehigh)
Travel(UMD,National)
Seats available
Travel by plane
Enough money for air
fare available
Travel by car
Enough money
for gasoline
Roads are passable
Taxi(UMD,UMD-Metro)
Metro(UMD-Metro,National)
alternative
methods
Taxi(L.V. Int’nal,Lehigh)

21. Application to Computer Bridge
Chess: better than all but the best humans
Bridge: worse than many good players
Why bridge is difficult for computers
It is an imperfect information game
Don’t know what cards the others have (except the dummy)
Many possible card distributions, so many possible moves
If we encode the additional moves as additional branches in the game tree, this increases the number of nodes exponentially
worst case: about 6x1044 leaf nodes
average case: about 1024 leaf nodes
Not enough time to search the game tree
(Dana S. Nau)

22. How to Reduce the Size of the Game Tree?
Bridge is a game of planning
Declarer plans how to play the hand by combining various strategies (ruffing, finessing, etc.)
If a move doesn’t fit into a sensible strategy, then it probably doesn’t need to be considered
HTN approach for declarer play
Use HTN planning to generate a game tree in which each move corresponds to a different strategy, not a different card
Reduces average game-tree size to about 26,000 leaf nodes
Bridge Baron: implements HTN planning
Won the 1997 World Bridge Computer Challenge
All commercial versions of Bridge Baron since 1997 have include an HTN planner (has sold many thousands of copies)
(Dana S. Nau)

23. Why “Classical”? Classical planning makes a number of assumptions:
Symbolic information (i.e., non numerical)
Actions always succeed
The “Strips” assumption: only changes that takes place are those indicated by the operators
Despite these (admittedly unrealistic) assumptions some work-around can be made (and have been made!) to apply the principles of classical planning to games
Neoclassical planning removes some of these assumptions