1. © Daniel S. Weld 1 Logistics Travel Wed class led by Mausam Week’s reading R&N ch17 Project meetings

2. © Daniel S. Weld 2 Weekly Course Outline 1.Overview, agents, problem spaces 2. 3. 4. 5.Learning 6.Planning (and CSPs) 7.Uncertainty 8.Planning under uncertainty 9.Reinforcement learning 10.Topics 11.Project Presentations & Contest Search Knowledge Representation }

3. © Daniel S. Weld 3 Previously Constraint satisfaction Backtracking policies & var-ordering heuristics The planning problem Simplifying assumptions Searching world states Regression Compilation to SAT, CSP, ILP, BDD Graphplan Basics …

4. © Daniel S. Weld 4 Graphplan Phase 1 - Graph Expansion Necessary (insufficient) conditions for plan existence Local consistency of plan-as-CSP Phase 2 - Solution Extraction Variables action execution at a time point Constraints goals, subgoals achieved no side-effects between actions

5. © Daniel S. Weld 5 The Plan Graph … … … level 0level 2level 4level 6 level 1level 3level 5 propositions actions propositions actions

6. © Daniel S. Weld 6 Mutual Exclusion Actions A,B exclusive (at a level) if A deletes B’s precond, or B deletes A’s precond, or A & B have inconsistent preconds Propositions P,Q inconsistent (at a level) if all ways to achieve P exclude all ways to achieve Q

7. © Daniel S. Weld 7 Searching for a solution If goals are present & non-mutex: Choose action to achieve each goal Add preconditions to next goal set Recurse!

8. © Daniel S. Weld 8 Immediate Outline Graphplan Example Relation to CSP Reachability analysis & heuristic generation Do you need the whole planning graph? Temporal planning Planning under uncertainty

9. © Daniel S. Weld 9 Dinner Date Initial Conditions: (:and (cleanHands) (quiet)) Goal: (:and (noGarbage) (dinner) (present)) Actions: (:operator carry :precondition :effect (:and (noGarbage) (:not (cleanHands))) (:operator dolly :precondition :effect (:and (noGarbage) (:not (quiet))) (:operator cook :precondition (cleanHands) :effect (dinner)) (:operator wrap :precondition (quiet) :effect (present))

10. © Daniel S. Weld 10 Planning Graph noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 0 Prop 1 Action 2 Prop 3 Action 4 Prop

11. © Daniel S. Weld 11 Are there any exclusions? noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 0 Prop 1 Action 2 Prop 3 Action 4 Prop

12. © Daniel S. Weld 12 Do we have a solution? noGarb cleanH quiet dinner present carry dolly cook wrap cleanH quiet 0 Prop 1 Action 2 Prop 3 Action 4 Prop

13. © Daniel S. Weld 13 Extend the Planning Graph noGarb cleanH quiet dinner present carry dolly cook wrap carry dolly cook wrap cleanH quiet noGarb cleanH quiet dinner present 0 Prop 1 Action 2 Prop 3 Action 4 Prop

14. © Daniel S. Weld 14 Searching Backwards noGarb cleanH quiet dinner present carry dolly cook wrap carry dolly cook wrap cleanH quiet noGarb cleanH quiet dinner present 0 Prop 1 Action 2 Prop 3 Action 4 Prop

15. © Daniel S. Weld 15 One (of 4) Possibilities noGarb cleanH quiet dinner present carry dolly cook wrap carry dolly cook wrap cleanH quiet noGarb cleanH quiet dinner present 0 Prop 1 Action 2 Prop 3 Action 4 Prop

16. © Daniel S. Weld 16 One (of 4) possibilities noGarb cleanH quiet dinner present carry dolly cook wrap carry dolly cook wrap cleanH quiet noGarb cleanH quiet dinner present 0 Prop 1 Action 2 Prop 3 Action 4 Prop

17. © Daniel S. Weld 17 Graphplan Solution Extraction as a Constraint Network Present NoGarb Dinner Not carry & cook Not dolly & wrap

18. © Daniel S. Weld 18 Speed / Memory Ratios Do & Kambhampati http://citeseer.nj.nec.com/do00solving.html

19. © Daniel S. Weld 19 Variable-Ordering Consider normal GP solution extraction Can you think of a good VO heuristic? P Q

20. © Daniel S. Weld 20 Level of Proposition DVO What if same number of supporting actions? Prefer props added latest Most constrained P Q

21. © Daniel S. Weld 21 Immediate Outline Graphplan Example Relation to CSP Reachability analysis & heuristic generation Do you need the whole planning graph? Temporal planning Planning under uncertainty

22. © Daniel S. Weld 22 Heuristics for World-Space Search Any ideas?

23. © Daniel S. Weld 23 Optimistic Projection of Achievability At(0,0) Key(0,1) Prop list Level 0 At(0,1) At(1,0) noop Action list Level 0 Move(0,0,0,1) Move(0,0,1,0) x At(0,0) key(0,1) Prop list Level 1 x At(0,0) Key(0,1) noop x Action list Level 1 x Prop list Level 2 Move(0,1,1,1) At(1,1) At(1,0) At(0,1) Move(1,0,1,1) noop x x x x x x …... x Pick_key(0,1) Have_key ~Key(0,1)x x x x x Mutexes Initial state 0 1 2 Goal state Grid Problem Serial PG: PG where any pair of non-noop actions are marked mutex lev(S): index of the first level where all props in S appear non- mutexed. If there is no such level, then If the graph is grown to level off, then  Else k+1 (k is the current length of the graph)

24. © Daniel S. Weld 24 Cost of a Set of Literals lev(p) : index of the first level at which p comes into the planning graph lev(S): index of the first level where all props in S appear non- mutexed. If there is no such level, then If the graph is grown to level off, then  Else k+1 (k is the current length of the graph) SumSet-Level Partition-kAdjusted SumCombo Set-Level with memos h(S) =  p  S lev({p}) h(S) = lev(S) Admissible

25. © Daniel S. Weld 25 Adjusting the Sum Heuristic Start with Sum heuristic and adjust it to take subgoal interactions into account Negative interactions in terms of “degree of interaction” Positive interactions in terms of co-achievement links Ignore negative interactions when accounting for positive interactions (and vice versa) [Kambhampati AAAI 2000] HAdjSum2M(S) = length(RelaxedPlan(S)) + max p,q  S  (p,q) Where  (p,q) = lev({p,q}) - max{lev(p), lev(q)} /*Degree of –ve Interaction */

26. © Daniel S. Weld 26 McDermott’s Grid World

27. © Daniel S. Weld 27 Immediate Outline Graphplan Example Relation to CSP Reachability analysis & heuristic generation Do you need the whole planning graph? Temporal planning Planning under uncertainty

28. © Daniel S. Weld 28 Observation 1 Propositions monotonically increase (always carried forward by no-ops) p ¬q ¬r p q ¬q ¬r p q ¬q r ¬r p q ¬q r ¬r A A B A B

29. © Daniel S. Weld 29 Observation 2 Actions monotonically increase p ¬q ¬r p q ¬q ¬r p q ¬q r ¬r p q ¬q r ¬r A A B A B

30. © Daniel S. Weld 30 Observation 3 Proposition mutex relationships monotonically decrease pqr…pqr… A pqr…pqr… pqr…pqr…

31. © Daniel S. Weld 31 Observation 4 Action mutex relationships monotonically decrease pq…pq… B pqrs…pqrs… pqrs…pqrs… A C B C A pqrs…pqrs… B C A

32. © Daniel S. Weld 32 New Representation Propositions Actions pqrs…pqrs… ABCD…ABCD… 0 2 4 4 1 3 3 5

33. © Daniel S. Weld 33 Mutex Relationships pqrs…pqrs… ABCD…ABCD… 0 2 4 4 1 3 3 5  7 8 Propositions Actions

34. © Daniel S. Weld 34 Plan Graph pqrs…pqrs… ABCD…ABCD… 0 2 4 4 1 3 3 5  7 6 Props & actions: start level  start time Mutex relations: end level  end time

35. © Daniel S. Weld 35 TGP: Doing Time Actions Real duration Parallel Effects occur at end Preconditions hold throughout Affected propositions undefined during A eff1 eff2 pre cond1 [Smith & Weld IJCAI-99]

36. © Daniel S. Weld 36 TGP Graph Expansion New Proposition New Action New Support Add New Prop Mutex Add New Action Mutex Terminate Prop Mutex Terminate Action Mutex + + + pqrs…pqrs… ABCD…ABCD… 0 2 4 4 1 3 3 5  7 8

37. © Daniel S. Weld 37 Temporal Planning Graphplan too complex Can we modify state-space search? Suppose regression

38. © Daniel S. Weld 38 TP4: Search Space The initial state S 0 =? E={p i } p i is required to be true at t F= {(a i,  i )} action a i is planned to start at t-  i t: time stamp S 0 = (G, NULL) The final state S ? {S= (E, NULL)|E  I p } S = ( E, F, t) [Haslum & Geffner ECP 2001]

39. © Daniel S. Weld 39 TP4: Action Application S=(E,{(a j,  j )},t) A set of actions, A, is applicable in S if -- every action in A is compatible with each other; --  p  E,  a  A, p  add(a), a is compatible with  b  F E={p i } p i is required to be true at t F= {(a i,  i )} action a i is planned to start at t-  i t: time stamp

40. © Daniel S. Weld 40 TP4: branching rule Choose a set of actions SE applicable in S, define: F new = { (a, dur(a)| a  SE }  adv = min{ , a!=no-op} then  ’ = {pre(a) | (a,  )  F  F new } F ’ = { (a,  adv )| (a,  )  F  F new,  adv } S ’ = (E ’, F ’ )