1. Exam #2 statistics (total = 100pt) u CS480: 12 registered, 9 took exam #2  Average: 80.89  Max: 100 (2)  Min: 68 u CS580: 8 registered, 8 took exam #2  Average: 83.13  Max: 98 (1)  Min: 60 u CS796: 20 registered, 20 took exam #2  Average: 87.4  Max: 100 (4)  Min: 53 1

2. 2 CS480 CS580 CS796

3. Planning AIMA: 10.1, 10.2, 10.3 Follow slides and use textbook as reference

4. Early final exam 12/9/2010? Term paper for CS796 due 12/1/2010 4

5. “During the Persian Gulf crisis of 1991, U.S. forces deployed a Dynamic Analysis and Replanning Tool, DART (Cross and Walker, 1994), to do automated logistics planning and scheduling for transportation. This involved up to 50,000 vehicles, cargo, and people at a time, and had to account for starting points, destinations, routes, and conflict resolution among all parameters. The AI planning techniques generated in hours a plan that would have taken weeks with older methods. The Defense Advanced Research Project Agency (DARPA) stated that this single application more than paid back DARPA's 30-year investment in Al.” 5

6. Progression: An action A can be applied to state S iff the preconditions are satisfied in the current state The resulting state S’ is computed as follows: --every variable that occurs in the actions effects gets the value that the action said it should have --every other variable gets the value it had in the state S where the action is applied holding(A) ~Clear(A) ~Ontable(A) Ontable(B), Clear(B) ~handempty Ontable(A) Ontable(B), Clear(A) Clear(B) hand-empty Pickup(A) Pickup(B) holding(B) ~Clear(B) ~Ontable(B) Ontable(A), Clear(A) ~handempty Pickup(x) Prec: hand-empty,clear(x),ontable(x) eff: holding(x),~ontable(x),~hand-empty,~Clear(x) STRIPS ASSUMPTION: ONLY variables that have been changed by the action are mentioned in the effect Precond Effect S S’

7. Regression: A state S can be regressed over an action A (or A is applied in the backward direction to S) Iff: --There is no variable v such that v is given different values by the effects of A and the state S --There is at least one variable v’ such that v’ is given the same value by the effects of A as well as state S The resulting state S’ is computed as follows: -- every variable that occurs in S, and does not occur in the effects of A will be copied over to S’ with its value as in S -- every variable that occurs in the precondition list of A will be copied over to S’ with the value it has in in the precondition list ~clear(B) hand-empty Putdown(A) Stack(A,B) ~clear(B) holding(A) clear(B) Putdown(B)?? Termination test: Stop when the state s’ is entailed by the initial state s I Putdown(x) Prec: holding(x) eff: Ontable(x), hand-empty, clear(x), ~holding(x) Stack(x,y) Prec: holding(x), clear(y) eff: on(x,y), ~clear(y), ~holding(x), hand-empty S S’

8. Progression vs. Regression The never ending war Progression has higher branching factor Progression searches in the space of complete (and consistent) states Regression has lower branching factor Regression searches in the space of partial states –There are 3 n partial states (as against 2 n complete states) You can also do bidirectional search stop when a (leaf) state in the progression tree entails a (leaf) state in the regression tree

9. Regression vs. Reversibility Notice that regression doesn’t require that the actions are reversible in the real world –We only think of actions in the reverse direction during simulation Normal blocks world is reversible (if you don’t like the effects of stack(A,B), you can do unstack(A,B)). However, if the blocks world has a “bomb” the table action, then normally, there won’t be a way to reverse the effects of that action. –But even with that action we can do regression –For example we can reason that the best way to make table go- away is to add “Bomb” action into the plan as the last action

10. Goal state is partial –if only m of the k state variables are mentioned in a goal specification, then upto 2 k-m complete state of the world can satisfy our goals! –Sometimes a more complete goal state may provide hints to the agent as to what the plan should be In the blocks world example, if we also state that On(A,B) as part of the goal (in addition to ~Clear(B)&hand-empty) then it would be quite easy to see what the plan should be. Initial state is complete –If initial state is partial, then we have “partial observability” (i.e., the agent doesn’t know where it is!) Because of the asymmetry between init and goal states, progression is in the space of complete states, while regression is in the space of “partial” states. Specifically, for k state variables, there are 2 k complete states and 3 k “partial” states –(a state variable may be present positively, present negatively or not present at all in the goal specification!) Init: Ontable(A),Ontable(B), Clear(A), Clear(B), hand-empty Goal: ~clear(B), hand-empty Progression vs. Regression

11. Planning vs. Search: What is the difference? Search assumes that there is a child-generator and goal-test functions which know how to make sense of the states and generate new states Planning makes the additional assumption that the states can be represented in terms of state variables and their values –Initial and goal states are specified in terms of assignments over state variables Which means goal-test doesn’t have to be a blackbox procedure –That the actions modify these state variable values The preconditions and effects of the actions are in terms of partial assignments over state variables –Given these assumptions certain generic goal-test and child-generator functions can be written Specifically, we discussed one Child-generator called “Progression”, another called “Regression” Notice that the additional assumptions made by planning do not change the search algorithms (A*, IDDFS etc)—they only change the child-generator and goal-test functions –In particular, search still happens in terms of search nodes that have parent pointers etc. The “state” part of the search node will correspond to –“Complete state variable assignments” in the case of progression –“Partial state variable assignments” in the case of regression

12. Relevance, Reachability & Heuristics Reachability: Given a problem [I,G], a (partial) state S is called reachable if there is a sequence [a 1,a 2,…,a k ] of actions, which when executed from state I will lead to a state where S holds Relevance: Given a problem [I,G], a state S is called relevant if there is a sequence [a 1,a 2,…,a k ] of actions, which when executed from state S will lead to a state satisfying G (Relevance is Reachability from goal state) Initial State I Goal state G S S Reachable states Relevant states

13. Progression takes “applicability” of actions into account –Specifically, it guarantees that every state in its search queue is reachable..but has no idea whether the states are relevant (constitute progress towards top-level goals) SO, heuristics for progression need to help it estimate the “relevance” of the states in the search queue Regression takes “relevance” of actions into account –Specifically, it makes sure that every state in its search queue is relevant.. But has not idea whether the states in its search queue are reachable SO, heuristics for regression need to help it estimate the “reachability” of the states in the search queue Relevance, Reachability & Heuristics States that are both reachable and relevant are useful in planning I G

14. Subgoal interactions Suppose we have a set of subgoals G 1,….G n Suppose the length of the shortest plan for achieving the subgoals in isolation is l 1,….l n We want to know what is the length of the shortest plan for achieving the n subgoals together, l 1…n If subgoals are independent: l 1..n = l 1 +l 2 +…+l n If subgoals have + interactions alone: l 1..n < l 1 +l 2 +…+l n If subgoals have - interactions alone: l 1..n > l 1 +l 2 +…+l n If you made “independence” assumption, and added up the individual costs of subgoals, then your resultant heuristic will be  perfect if the goals are actually independent  inadmissible (over-estimating) if the goals have positive interactions  admissible if the goals have negative interactions

15. h set-difference hChC hPhP h*h* h0h0 Cost of computing the heuristic Cost of searching with the heuristic Total cost incurred in search Not always clear where the total minimum occurs Old wisdom was that the global min was closer to cheaper heuristics Current insights are that it may well be far from the cheaper heuristics for many problems E.g. Pattern databases for 8-puzzle Plan graph heuristics for planning Scalability came from sophisticated reachability heuristics based on planning graphs....and not from any hand-coded domain-specific control knoweldge

16. Planning Graph and Projection Envelope of Progression Tree (Relaxed Progression) –Proposition lists: Union of states at k th level –Mutex: Subsets of literals that cannot be part of any legal state Lowerbound reachability information [Blum&Furst, 1995] [ECP, 1997][AI Mag, 2007] p pq pr ps pqr pq pqs psq ps pst A1 A2 A3 A2 A1 A3 A1 A3 A4 p pqrspqrs pqrstpqrst A1 A2 A3 A1 A2 A3 A4 Planning Graphs can be used as the basis for heuristics!

17. Planning Graph Basics –Envelope of Progression Tree (Relaxed Progression) Linear vs. Exponential Growth –Reachable states correspond to subsets of proposition lists –BUT not all subsets are states Can be used for estimating non- reachability –If a state S is not a subset of k th level prop list, then it is definitely not reachable in k steps p pq pr ps pqr pq pqs p psq ps pst pqrspqrs pqrstpqrst A1 A2 A3 A2 A1 A3 A1 A3 A4 A1 A2 A3 A1 A2 A3 A4

18. Don’t look at curved lines for now… Have(cake) ~eaten(cake) ~Have(cake) eaten(cake) Eat No-op Have(cake) eaten(cake) bake ~Have(cake) eaten(cake) Have(cake) ~eaten(cake) Eat No-op Have(cake) ~eaten(cake) Graph has leveled off, when the prop list has not changed from the previous iteration The note that the graph has leveled off now since the last two Prop lists are the same (we could actually have stopped at the Previous level since we already have all possible literals by step 2)

19. Blocks world State variables: Ontable(x) On(x,y) Clear(x) hand-empty holding(x) Stack(x,y) Prec: holding(x), clear(y) eff: on(x,y), ~cl(y), ~holding(x), hand-empty Unstack(x,y) Prec: on(x,y),hand-empty,cl(x) eff: holding(x),~clear(x),clear(y),~hand-empty Pickup(x) Prec: hand-empty,clear(x),ontable(x) eff: holding(x),~ontable(x),~hand-empty,~Clear(x) Putdown(x) Prec: holding(x) eff: Ontable(x), hand-empty,clear(x),~holding(x) Initial state: Complete specification of T/F values to state variables --By convention, variables with F values are omitted Goal state: A partial specification of the desired state variable/value combinations --desired values can be both positive and negative Init: Ontable(A),Ontable(B), Clear(A), Clear(B), hand-empty Goal: ~clear(B), hand-empty A B

20. onT-A onT-B cl-A cl-B he Pick-A Pick-B onT-A onT-B cl-A cl-B he h-A h-B ~cl-A ~cl-B ~he

21. onT-A onT-B cl-A cl-B he Pick-A Pick-B onT-A onT-B cl-A cl-B he h-A h-B ~cl-A ~cl-B ~he St-A-B St-B-A Ptdn-A Ptdn-B Pick-A onT-A onT-B cl-A cl-B he h-A h-B ~cl-A ~cl-B ~he on-A-B on-B-A Pick-B

22. Estimating the cost of achieving individual literals (subgoals) Idea: Unfold a data structure called “planning graph” as follows: 1. Start with the initial state. This is called the zeroth level proposition list 2. In the next level, called first level action list, put all the actions whose preconditions are true in the initial state -- Have links between actions and their preconditions 3. In the next level, called first level proposition list, put: Note: A literal appears at most once in a proposition list. 3.1. All the effects of all the actions in the previous level. Links the effects to the respective actions. (If multiple actions give a particular effect, have multiple links to that effect from all those actions) 3.2. All the conditions in the previous proposition list (in this case zeroth proposition list). Put persistence links between the corresponding literals in the previous proposition list and the current proposition list. *4. Repeat steps 2 and 3 until there is no difference between two consecutive proposition lists. At that point the graph is said to have “leveled off” The next 2 slides show this expansion upto two levels

23. Using the planning graph to estimate the cost of single literals: 1. We can say that the cost of a single literal is the index of the first proposition level in which it appears. --If the literal does not appear in any of the levels in the currently expanded planning graph, then the cost of that literal is: -- l+1 if the graph has been expanded to l levels, but has not yet leveled off -- Infinity, if the graph has been expanded (basically, the literal cannot be achieved from the current initial state) Examples: h({~he}) = 1 h ({On(A,B)}) = 2 h({he})= 0 How about sets of literals?  see next slide

24. Estimating reachability of sets We can estimate cost of a set of literals in three ways: Make independence assumption h sum (p,q,r)= h(p)+h(q)+h(r) Define the cost of a set of literals in terms of the level where they appear together h-lev({p,q,r})= The index of the first level of the PG where p,q,r appear together so, h({~he,h-A}) = 1 Compute the length of a “relaxed plan” to supporting all the literals in the set S, and use it as the heuristic: h relax

25. Neither h lev nor h sum work well always p1p1 p2p2 p3p3 p 99 p 100 B1 q B2 B3 B99 B100 q P1P1 A0A0 P0P0 p1p1 p2p2 p3p3 p 99 p 100 q B* q P1P1 A0A0 P0P0 True cost of {p 1 …p 100 } is 100 (needs 100 actions to reach) H lev says the cost is 1 H sum says the cost is 100 H sum better than H lev True cost of {p 1 …p 100 } is 1 (needs just one action reach) H lev says the cost is 1 H sum says the cost is 100 H lev better than H sum

26. h-sum; h-lev; H-lev is admissible H-sum in not admissible H-sum is larger than or equal to H-lev

27. Goal Interactions To better account for - interactions, we need to start looking into feasibility of subsets of literals actually being true together in a proposition level. Specifically,in each proposition level, we want to mark not just which individual literals are feasible, –but also which pairs, which triples, which quadruples, and which n-tuples are feasible. (It is quite possible that two literals are independently feasible in level k, but not feasible together in that level) The idea then is to say that the cost of a set of S literals is the index of the first level of the planning graph, where no subset of S is marked infeasible The full scale mark-up is very costly, and makes the cost of planning graph construction equal the cost of enumerating the full progres sion search tree. –Since we only want estimates, it is okay if talk of feasibility of upto k-tuples For the special case of feasibility of k=2 (2-sized subsets), there are some very efficient marking and propagation procedures. –This is the idea of marking and propagating mutual exclusion relations.

28. Level-off definition? When neither propositions nor mutexes change between levels

29. Two actions a1 and a2 are mutex if any of the following is true: (a) Inconsistent effects: one action negates the effect of the other. (b)Interference: one of the effects of one action is the negation of a prediction of the other (c)Competing needs: one of the predictions of one action is mutually exclusive with a prediction of the other Two propositions P1 and P2 are marked mutex if: all actions supporting P1 are pair-wise mutex with all actions supporting P2. Mutex Propagation Rules

30. onT-A onT-B cl-A cl-B he Pick-A Pick-B onT-A onT-B cl-A cl-B he h-A h-B ~cl-A ~cl-B ~he

31. onT-A onT-B cl-A cl-B he Pick-A Pick-B onT-A onT-B cl-A cl-B he h-A h-B ~cl-A ~cl-B ~he St-A-B St-B-A Ptdn-A Ptdn-B Pick-A onT-A onT-B cl-A cl-B he h-A h-B ~cl-A ~cl-B ~he on-A-B on-B-A Pick-B

32. Level-based heuristics on planning graph with mutex relations h lev ({p 1, …p n })= The index of the first level of the PG where p 1, …p n appear together and no pair of them are marked mutex. (If there is no such level, then h lev is set to l+1 if the PG is expanded to l levels, and to infinity, if it has been expanded until it leveled off) We now modify the h lev heuristic as follows This heuristic is admissible. With this heuristic, we have a much better handle on both + and - interactions. In our example, this heuristic gives the following reasonable costs: h({~he, cl-A}) = 1 h({~cl-B,he}) = 2 h({he, h-A}) = infinity (because they will be marked mutex even in the final level of the leveled PG) Works very well in practice H({have(cake),eaten(cake)}) = 2

33. How lazy can we be in marking mutexes? We noticed that h lev is already admissible even without taking negative interactions into account If we mark mutexes, then h lev can only become more informed –So, being lazy about marking mutexes cannot affect admissibility –However, being over-eager about marking mutexes (i.e., marking non-mutex actions mutex) does lead to loss of admissibility

34. Some observations about the structure of the PG 1. If an action a is present in level l, it will be present in all subsequent levels. 2. If a literal p is present in level l, it will be present in all subsequent levels. 3. If two literals p,q are not mutex in level l, they will never be mutex in subsequent levels --Mutex relations relax monotonically as we grow PG

35. Summary Planning and search Progression Regression Planning graph and heuristics Goal interactions and mutual exclusion