1. Lirong Xia Planning Tuesday, April 22, 2014

2. The kernel trick Neuron network Clustering (k-means) 1 Last time

3. Kernelized Perceptron 2 If we had a black box (kernel) which told us the dot product of two examples x and y: –Could work entirely with the dual representation –No need to ever take dot products (kernel trick) Downside: slow if many examples get nonzero alpha

4. Non-Linear Separators 3 General idea: the original feature space can always be mapped to some higher-dimensional feature space where the training set is separable:

5. Some Kernels 4 Kernels implicitly map original vectors to higher dimensional spaces, take the dot product there, and hand the result back Linear kernel: Quadratic kernel: RBF: infinite dimensional representation Mercers theorem: if the matrix A with A i,j =K(x i,x j ) is positive definite, then K can be represented as the inner product of a higher dimensional space

6. Neuron = Perceptron with a non-linear activation function a i =g(Σ j W j,i a j ) 5 Neuron

7. Feed-forward networks –single-layer (perceptrons) –multi-layer (perceptrons) can represent any boolean function –Learning Minimize squared error of prediction Gradient descent: backpropagation algorithm Recurrent networks –Hopfield networks have symmetric weights W i,j =W j,i –Boltzmann machines use stochastic activation functions 6 Neural networks

8. a 5 = g 5 ( W 3,5 × a 3 + W 4,5 × a 4 ) = g 5 ( W 3,5 × g 3 ( W 1,3 × a 1 + W 2,3 × a 2 )+ W 4,5 × g 4 ( W 1,4 × a 1 + W 2,4 × a 2 )) 7 Feed-forward example

9. K-Means 8 An iterative clustering algorithm –Pick K random points as cluster centers (means) –Alternate: Assign data instances to closest mean Assign each mean to the average of its assigned points –Stop when no points assignments change

10. K-Means as Optimization 9 Consider the total distance to the means: Each iteration reduces phi Two states each iteration: –Update assignments: fix means c, change assignments a –Update means: fix assignments a, change means c points assignments means

11. Today: Planning A brief review of logic Planning problems Search algorithms

12. Atomic variables (atoms) –can be true or false –IsRaining –Literal: an atomic variable or its negation Operators –Not(x), And(x,y), Or(x,y) –applied to atomic variables or formulas Semantics –meaning w.r.t. true/false valuation Syntax –purely symbolic inference rules 11 Language of propositional logic

13. Terms –objects –can be combined by functions –e.g. NextInteger(x) Relation symbols –functions that takes terms as inputs –can take true or false –E.g. IsRainyToday (0-ary), IsHuman(x) (1-ary), AtPosition(x,y) (2-ary) –Literal: a relation symbol or its negation Operators –Not(x), And(x,y), Or(x,y) –Quantifiers: and –applied to relation symbols or formulas, but not to terms Semantics –meaning w.r.t. true/false valuation of relation symbols –Note: a term cannot be true or false Syntax –purely symbolic inference rules 12 Language in first-order logic

14. Caution: deliberately confusing semantics and syntax for illustration Syllogism –All men are mortal –Socrates is a man –Therefore, Socrates is mortal Propositional logic –SocratesIsManSocratesIsMortal –SocratesIsMan = true –Therefore, SocratesIsMortal = true First-order logic – x (IsMan(x) IsMortal(x)) –IsMan(Socrates) = true –IsMortal(Socrates) = true 13 Example of semantics

15. Today: Planning A brief review of logic Planning problems Search algorithms

16. Search Problems 15 A search problem consists of: –A state space …… –A successor function (with actions, costs) –A start state and a goal test A solution is a sequence of actions (a plan) which transforms the start state to a goal state Algorithms –BFS, DFS, UCS –Heuristics: best first, A*

17. State space –compactly represented by logic Actions –compactly represented by preconditions: activation rule effects: consequences A solution is a sequence of actions 16 Planning as a search problem

18. State of the world (STRIPS language) Stanford Research Institute Problem Solver State of the world = conjunction of positive, ground, function-free literals –At(Home) AND IsAt (Umbrella, Home) AND CanBeCarried(Umbrella) AND IsUmbrella(Umbrella) AND HandEmpty AND Dry –Not OK as part of the state: NOT(At(Home)) (negative) At(x) (not ground) At(Bedroom(Home)) (uses the function Bedroom) Any literal not mentioned is assumed false –Other languages make different assumptions, e.g., negative literals part of state, unmentioned literals unknown

19. An action: TakeObject TakeObject(location, x) Preconditions: –HandEmpty –CanBeCarried(x) –At(location) –IsAt(x, location) Effects (NOT something means that that something should be removed from state): –Holding(x) –NOT(HandEmpty) –NOT(IsAt(x, location))

20. Another action WalkWithUmbrella(location1, location2, umbr) Preconditions: –At(location1) –Holding(umbr) –IsUmbrella(umbr) Effects: –At(location2) –NOT(At(location1))

21. Yet another action WalkWithoutUmbrella(location1, location2) Preconditions: –At(location1) Effects: –At(location2) –NOT(At(location1)) –NOT(Dry)

22. A goal and a plan Goal: At(Work) AND Dry Recall initial state: –At(Home) AND IsAt(Umbrella, Home) AND CanBeCarried(Umbrella) AND IsUmbrella(Umbrella) AND HandEmpty AND Dry TakeObject(Home, Umbrella) –At(Home) AND CanBeCarried(Umbrella) AND IsUmbrella(Umbrella) AND Dry AND Holding(Umbrella) WalkWithUmbrella(Home, Work, Umbrella) –At(Work) AND CanBeCarried(Umbrella) AND IsUmbrella(Umbrella) AND Dry AND Holding(Umbrella)

23. Planning to write a paper Suppose your goal is to be a co-author on an AI paper with both theorems and experiments, within a year LearnAbout(x,y) Preconditions: HasTimeForStudy(x) Effects: Knows(x,y), NOT(HasTimeForStudy(x)) HaveNewIdea(x) Preconditions: Knows(x,AI), Creative(x) Effects: Idea, Contributed(x) ProveTheorems(x) Preconditions: Knows(x,AI), Knows(x,Math), Idea Effect: Theorems, Contributed(x) PerformExperiments(x) Preconditions: Knows(x,AI), Knows(x,Coding), Idea Effect: Experiments, Contributed(x) WritePaper(x) Preconditions: Knows(x,AI), Knows(x,Writing), Idea, Theorems, Experiments Effect: Paper, Contributed(x) Goal: Paper AND Contributed(You) FindExistingOpenProblem(x) Preconditions: Knows(x,AI) Effects: Idea Name a few things that are missing/unrealistic…

24. Some start states Start1: HasTimeForStudy(You) AND Knows(You,Math) AND Knows(You,Coding) AND Knows(You,Writing) Start2: HasTimeForStudy(You) AND Creative(You) AND Knows(Advisor,AI) AND Knows(Advisor,Math) AND Knows(Advisor,Coding) AND Knows(Advisor,Writing) (Good luck with that plan…) Start3: Knows(You,AI) AND Knows(You,Coding) AND Knows(OfficeMate,Math) AND HasTimeForStudy(OfficeMate) AND Knows(Advisor,AI) AND Knows(Advisor,Writing) Start4: HasTimeForStudy(You) AND Knows(Advisor,AI) AND Knows(Advisor,Math) AND Knows(Advisor,Coding) AND Knows(Advisor,Writing)

25. Today: Planning A brief review of logic Planning problems Search algorithms

26. Forward state-space search (progression planning) Successors: all states that can be reached with an action whose preconditions are satisfied in current state At(Home) IsAt(Umbrella, Home) CanBeCarried(Umbrella) IsUmbrella(Umbrella) HandEmpty Dry At(Home) Holding(Umbrella) CanBeCarried(Umbrella) IsUmbrella(Umbrella) Dry TakeObject(Home, Umbrella) At(Work) IsAt(Umbrella, Home) CanBeCarried(Umbrella) IsUmbrella(Umbrella) HandEmpty WalkWithoutUm brella(Home, Work) WalkWithUmbrella( Home, Work, Umbrella) At(Work) Holding(Umbrella) CanBeCarried(Umbrella) IsUmbrella(Umbrella) Dry WalkWithout Umbrella(Wor k, Home) At(Home) IsAt(Umbrella, Home) CanBeCarried(Umbrella) IsUmbrella(Umbrella) HandEmpty GOAL! WalkWithoutUmbrella( Home, Umbrella) (!) WalkWithoutUm brella(Home, Work)

27. Backward state-space search (regression planning) Predecessors: for every action that accomplishes one of the literals (and does not undo another literal), remove that literal and add all the preconditions At(location1) At(location2) IsAt(umbr, location2) CanBeCarried(umbr) IsUmbrella(umbr) HandEmpty Dry At(location1) Holding(umbr) IsUmbrella(umbr) Dry TakeObject(location2, umbr) This is accomplished in the start state, by substituting location1=location2=Home, umbr=Umbrella WalkWithUmbrella( location1, Work, umbr) At(Work) Dry GOAL WalkWithUmbrella(location2, location1) WalkWithoutUmbrella can never be used, because it undoes Dry (this is good)

28. Heuristics for state-space search Cost of a plan: (usually) number of actions Heuristic 1: plan for each subgoal (literal) separately, sum costs of plans –Does this ever underestimate? Overestimate? Heuristic 2: solve a relaxed planning problem in which actions never delete literals (empty-delete- list heuristic) –Does this ever underestimate? Overestimate? –Very effective, even though requires solution to (easy) planning problem Progression planners with empty-delete-list heuristic perform well

29. Blocks world On(B, A), On(A, Table), On(D, C), On(C, Table), Clear(B), Clear(D) A B C D

30. Blocks world: Move action Move(x,y,z) Preconditions: –On(x,y), Clear(x), Clear(z) Effects: –On(x,z), Clear(y), NOT(On(x,y)), NOT(Clear(z)) A B C D

31. Blocks world: MoveToTable action MoveToTable(x,y) Preconditions: –On(x,y), Clear(x) Effects: –On(x,Table), Clear(y), NOT(On(x,y)) A B C D

32. Blocks world example Goal: On(A,B) AND Clear(A) AND On(C,D) AND Clear(C) A plan: MoveToTable(B, A), MoveToTable(D, C), Move(C, Table, D), Move(A, Table, B) Really two separate problem instances A B C D

33. A partial-order plan Goal: On(A,B) AND Clear(A) AND On(C,D) AND Clear(C) A B C D Start MoveToTable( B,A) MoveToTable( D,C) Move(A, Table, B) Move(C, Table, D) Finish Any total order on the actions consistent with this partial order will work

34. A partial-order plan (with more detail) Start MoveToTable(B, A) MoveToTable(D, C) Move(A,T able, B) Move(C,T able, D) Finish On(B,A) Clear(B)On(D,C)Clear(D) Clear(A)Clear(B)On(A, Table)Clear(D)Clear(C)On(C, Table) On(A, B)On(C, D)Clear(A)Clear(C) On(B,A) Clear(B)On(D,C)Clear(D)On(A, Table)On(C, Table)

35. Not everything decomposes into multiple problems: Sussman Anomaly Goal: On(A,B) AND On(B,C) Sequentially pursuing goals does not work Optimal plan: MoveToTable(C,A), Move(B,Table,C), Move(A,Table,B) AB C

36. An incorrect partial order plan for the Sussman Anomaly Start Finish On(A, B) On(A, Table) MoveToTable(C, A) Move(B,T able,C) Move(A, Table,B) On(B, C) On(B, Table)Clear(B)On(C, A)Clear(C) On(C, A)Clear(C) Clear(A)On(A, Table)Clear(B) Clear(C)On(B, Table) Move(B,Table,C) must be after MoveToTable(C,A), otherwise it will ruin Clear(C) Move(A,Table,B) must be after Move(B,Table,C), otherwise it will ruin Clear(B)

37. Corrected partial order plan for the Sussman Anomaly Start Finish On(A, B) On(A, Table) MoveToTable(C, A) Move(B,T able, C) Move(A, Table, B) On(B, C) On(B, Table)Clear(B)On(C, A)Clear(C) On(C, A)Clear(C) Clear(A)On(A, Table)Clear(B) Clear(C)On(B, Table) No more flexibility in the order due to protection of causal links

38. Searching for a partial-order plan Start WalkWithoutUmbrella(H ome, Work) Finish At(Work)Dry At(Home)IsAt(Umbrella,Home)CanBeCarried(Umbrella)IsUmbrella(Umbrella)HandEmptyDry no way to resolve conflict! WalkWithUmbrella(Home, Work, Umbrella) At(Home) Holding(Umbrella)IsUmbrella(Umbrella) TakeObject(Home, Umbrella) HandEmptyCanBeCarried(Umbrella)At(Home)IsAt(Umbrella,Home)

39. Searching for partial-order plans Somewhat similar to constraint satisfaction Search state = partially completed partial order plan –Not to be confused with states of the world –Contains actions, ordering constraints on actions, causal links, some open preconditions Search works as follows: –Choose one open precondition p, –Consider all actions that achieve p (including ones already in the plan), –For each such action, consider each way of resolving conflicts using ordering constraints

40. Planning graphs Each level has literals that could be true at that level Mutex (mutual exclusion) relations indicate incompatible actions/literals On(C, A) On(A, Table) Clear(C) MoveToTable(C,A) Move(C,A,B) On(B, Table) Clear(B) Move(B,Table,C) On(C, A) On(A, Table) Clear(C) On(B, Table) Clear(B) On(C, Table) On(C, B) On(B, C) Clear(A) … continued on board

41. Reasons for mutex relations… … between actions: –Inconsistent effects: one action negates effect of the other –Interference: one action negates precondition of the other –Competing needs: the actions have preconditions that are mutex … between literals: –Inconsistent support: any pair of actions that can achieve these literals is mutex

42. A problematic case for planning graphs FeedWith(x, y) –Preconditions: Edible(y) –Effects: NOT(Edible(y)), Fed(x) Start: Edible(Bread1), Edible(Bread2) Goal: Fed(Person1), Fed(Person2), Fed(Person3)

43. Planning graph for feeding Any two of these could simultaneously be true at time 1, so no mutex relations Really need 3-way mutex relations, but experimentally this is computationally not worthwhile Edible(Bread1) FeedWith(Person1, Bread1) Edible(Bread2) FeedWith(Person2, Bread1) FeedWith(Person3, Bread1) FeedWith(Person1, Bread2) FeedWith(Person2, Bread2) FeedWith(Person3, Bread2) Edible(Bread1) Edible(Bread2) Fed(Person1) Fed(Person2) Fed(Person3)

44. Uses of planning graphs If the goal literals do not all appear at a level (or have mutex relations) then we know we need another level –Converse does not hold Useful heuristic: first time that all goal literals appear at a level, with no mutex relations Graphplan algorithm: once all goal literals appear, try to extract solution from graph –Can use CSP techniques by labeling each action as in the plan or out of the plan –In case of failure, generate another level

45. Example Fast-Forward planner… –http://members.deri.at/~joergh/ff.htmlhttp://members.deri.at/~joergh/ff.html … with towers of Hanoi example… –http://www.tempastic.org/vhpop/http://www.tempastic.org/vhpop/ … in course directory:./ff -o hanoi-domain.pddl -f hanoi-3.pddl Btw., why is towers of Hanoi solvable with any number of disks?