Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 CS 2710, ISSP 2610 Chapter 3, Part 3 Heuristic Search Chapter 4 Local Search and Optimization.

Similar presentations


Presentation on theme: "1 CS 2710, ISSP 2610 Chapter 3, Part 3 Heuristic Search Chapter 4 Local Search and Optimization."— Presentation transcript:

1 1 CS 2710, ISSP 2610 Chapter 3, Part 3 Heuristic Search Chapter 4 Local Search and Optimization

2 2 Beam Search Cheap, unpredictable search For problems with many solutions, it may be worthwhile to discard unpromising paths Greedy best first search that keeps a fixed number of nodes on the fringe

3 3 Beam Search def beamSearch(fringe,beamwidth): while len(fringe) > 0: cur = fringe[0] fringe = fringe[1:] if goalp(cur): return cur newnodes = makeNodes(cur, successors(cur)) for s in newnodes: fringe = insertByH(s, fringe) fringe = fringe[:beamwidth] return []

4 4 Beam Search Optimal? Complete? Hardly! Space? O(b) (generates the successors) Often useful

5 Creating Heuristics 5

6 6 Combining Heuristics If you have lots of heuristics and none dominates the others and all are admissible… Use them all! H(n) = max(h1(n), …, hm(n))

7 Relaxed Heuristic Relaxed problem A problem with fewer restrictions on the actions The cost of an optimal solution to a relaxed problem is an admissible heuristic for the original problem. 7

8 8 Relaxed Problems Exact solutions to different (relaxed) problems H1 (# of misplaced tiles) is perfectly accurate if a tile could move to any square H2 (sum of Manhattan distances) is perfectly accurate if a tile could move 1 square in any direction

9 9 Relaxed Problems If problem is defined formally as a set of constraints, relaxed problems can be generated automatically Absolver (Prieditis, 1993) –Discovered a better heuristic for 8 puzzle and the first useful heuristic for Rubiks cube

10 Systematic Relaxation Precondition List –A conjunction of predicates that must hold true before the action can be applied Add List –A list of predicates that are to be added to the description of the world-state as a result of applying the action Delete List –A list of predicates that are no longer true once the action is applied and should, therefore, be deleted from the state description Primitive Predicates –ON(x, y):tile x is on cell y –CLEAR(y):cell y is clear of tiles –ADJ(y, z):cell y is adjacent to cell z 10

11 Here is the full definition of s move for the n- puzzle Move(x, y, z): precondition list ON(x, y), CLEAR(z), ADJ(y, z) add list ON(x, z), CLEAR(y) delete list ON(x, y), CLEAR(z) 11

12 (1) Removing CLEAR(z) and ADJ(y, z) gives # tiles out of place. Misplaced distance is 1+1=2 moves

13 (2) Removing CLEAR(z) gives Manhattan distance. Manhattan distance is 6+3=9 moves

14 Pattern Database Heuristics The idea behind pattern database heuristics is to store exact solution costs for every possible sub-problem instance. The idea behind pattern database heuristics is to store exact solution costs for every possible sub-problem instance. 14

15 Solve part of the problem, ignoring the other tiles

16 16 Pattern Databases optimal solution cost of the subproblem <= optimal solution cost of the full problem. Run exhaustive search to find optimal solutions for every possible configuration of 3, 7, 11, 12, 13, 14, 15, and store the results Do the same for the other tiles (maybe in two 4-tile subsets) Do this once before any problem solving is performed. Expensive, but can be worth it, if the search will be applied to many problem instances (deployed)

17 Pattern Databases Recall, in our example, we have three subproblems (subsets of 7, 4, and 4 tiles) State S has specific configurations of those subsets h(s)? 17

18 h(s)? Look up the exact costs for ss configurations of the 7, 4, and 4 tiles in the database Take the max! The max of a set of admissible heuristics is admissible What if it isnt feasible to have entries for all possibilities? …. 18

19 What if it isnt feasible to have entries for all possibilities? …. Take the max of: –The exact costs we do have, and the Manhattan distance for those we dont have 19

20 Sums of admissible heurstics We would like to take the sum rather than the max, since the result is more informed In general, adding two admissible heuristics might not be admissible For example, moves that solve one subproblem might help another subproblem But we can choose patterns that are disjoint, so we can sum them 20

21 Disjoint Pattern Database Heuristics Patterns that have no tiles in common. (As in our example) When calculating costs for a pattern, only count moves of the tiles in the pattern Patterns that have no tiles in common. (As in our example) When calculating costs for a pattern, only count moves of the tiles in the pattern Add together the heuristic values for the individual patterns. Add together the heuristic values for the individual patterns. The sum is admissible and more informed than taking the max The sum is admissible and more informed than taking the max 21

22 Examples for Disjoint Pattern Database Heuristics moves needed to solve red tiles 25 moves needed to solve blue tiles Overall heuristic is sum, or 20+25=45 moves 22

23 Overall heuristic is sum of the Manhattan Distance of each tile which is 39 moves. A trivial example of disjoint pattern database heuristics is Manhattan Distance in the case that we view every tile as a single pattern database 23

24 For your interest lab.org/bib/abstracts/Koen07g.htmlhttp://idm- lab.org/bib/abstracts/Koen07g.html P. Haslum, A. Botea, M. Helmert, B. Bonet and S. Koenig. Domain- Independent Construction of Pattern Database Heuristics for Cost-Optimal Planning. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pages ,

25 Def. Linear Conflict Heuristic --Two tiles t j and t k are in a linear conflict if t j and t k are the same line, the goal positions of t j and t k are both in that line, t j is to the right of t k, and goal position of t j is to the left of the goal position of t k. Linear Conflict Heuristic Function 25

26 Linear Conflict Example 1331 Manhattan distance is 2+2=4 moves 26

27 1331 Manhattan distance is 2+2=4 moves Linear Conflict Example 27

28 133 1 Manhattan distance is 2+2=4 moves Linear Conflict Example 28

29 133 1 Manhattan distance is 2+2=4 moves Linear Conflict Example 29

30 133 1 Manhattan distance is 2+2=4 moves Linear Conflict Example 30

31 1331 Manhattan distance is 2+2=4 moves Linear Conflict Example 31

32 1331 Manhattan distance is 2+2=4 moves Add a penalty for each linear conflict Linear Conflict Example 32

33 33 Other Sources of Heuristics Ad-hoc, informal, rules of thumb (guesswork) Approximate solutions to problems (algorithms course) Learn from experience (solving lots of 8-puzzles). –Each optimal solution is a learning example (node,actual cost to goal) –Learn heuristic function, E.G. H(n) = c1x1(n) + c2x2(n) x1 = #misplaced tiles; x2 = #adj tiles also adj in the goal state. c1 & c2 learned (best fit to the training data)

34 34 Search Types –Backtracking state-space search –Local Search and Optimization –Constraint satisfaction search –Adversarial search

35 35 Local Search and Optimization Previous searches: keep paths in memory, and remember alternatives so search can backtrack. Solution is a path to a goal. Path may be irrelevant, if the final configuration only is needed (8- queens, IC design, network optimization, …)

36 36 Local Search Use a single current state and move only to neighbors. Use little space Can find reasonable solutions in large or infinite (continuous) state spaces for which the other algorithms are not suitable

37 37 Optimization Local search is often suitable for optimization problems. Search for best state by optimizing an objective function.

38 38 Visualization States are laid out in a landscape Height corresponds to the objective function value Move around the landscape to find the highest (or lowest) peak Only keep track of the current states and immediate neighbors

39 39 Local Search Alogorithms Two strategies for choosing the state to visit next –Hill climbing –Simulated annealing Then, an extension to multiple current states: –Genetic algorithms

40 40 Hillclimbing (Greedy Local Search) Generate nearby successor states to the current state Pick the best and replace the current state with that one. Loop

41 41 Hill-climbing search problems Local maximum: a peak that is lower than the highest peak, so a bad solution is returned Plateau: the evaluation function is flat, resulting in a random walk Ridges: slopes very gently toward a peak, so the search may oscillate from side to side Local maximum Plateau Ridge

42 42 Random restart hill-climbing Start different hill-climbing searches from random starting positions stopping when a goal is found Save the best result from any search so far If all states have equal probability of being generated, it is complete with probability approaching 1 (a goal state will eventually be generated). Finding an optimal solution becomes the question of sufficient number of restarts Surprisingly effective, if there arent too many local maxima or plateaux

43 43 Simulated Annealing Based on a metallurgical metaphor –Start with a temperature set very high and slowly reduce it. –Run hillclimbing with the twist that you can occasionally replace the current state with a worse state based on the current temperature and how much worse the new state is.

44 44 Simulated Annealing Annealing: harden metals and glass by heating them to a high temperature and then gradually cooling them At the start, make lots of moves and then gradually slow down

45 45 Simulated Annealing More formally… –Generate a random new neighbor from current state. –If its better take it. –If its worse then take it with some probability proportional to the temperature and the delta between the new and old states.

46 46 Simulated annealing Probability of a move decreases with the amount ΔE by which the evaluation is worsened A second parameter T is also used to determine the probability: high T allows more worse moves, T close to zero results in few or no bad moves Schedule input determines the value of T as a function of the completed cycles

47 47 function Simulated-Annealing(problem, schedule) returns a solution state inputs:problem, a problem schedule, a mapping from time to temperature current Make-Node(Initial-State[problem]) for t 1 to do T schedule[t] if T=0 then return current next a randomly selected successor of current ΔE Value[next] – Value[current] if ΔE > 0 then current next else current next only with probability e ΔE/T

48 Intuitions Hill-climbing is incomplete Pure random walk, keeping track of the best state found so far, is complete but very inefficient Combine the ideas: add some randomness to hill-climbing to allow the possibility of escape from a local optimum 48

49 Intuitions the algorithm wanders around during the early parts of the search, hopefully toward a good general region of the state space Toward the end, the algorithm does a more focused search, making few bad moves 49

50 Theoretical Completeness There is a proof that if the schedule lowers T slowly enough, simulated annealing will find a global optimum with probability approaching 1 In practice, that may be way too many iterations In practice, though, SA can be effective at finding good solutions 50

51 51 Local Beam Search Keep track of k states rather than just one, as in hill climbing In comparison to beam search we saw earlier, this algorithm is state-based rather than node-based.

52 52 Local Beam Search Begins with k randomly generated states At each step, all successors of all k states are generated If any one is a goal, alg halts Otherwise, selects best k successors from the complete list, and repeats

53 53 Local Beam Search Successors can become concentrated in a small part of state space Stochastic beam search: choose k successors, with probability of choosing a given successor increasing with value Like natural selection: successors (offspring) of a state (organism) populate the next generation according to its value (fitness)

54 54 Genetic Algorithms Variant of stochastic beam search Combine two parent states to generate successors

55 55 function GA (pop, fitness-fn) Repeat new-pop = {} for i from 1 to size(pop): x = rand-sel(pop,fitness-fn) y = rand-sel(pop,fitness-fn) child = reproduce(x,y) if (small rand prob): child mutate(child) add child to new-pop pop = new-pop Until an indiv is fit enough, or out of time Return best indiv in pop, according to fitness-fn

56 56 function reproduce(x,y) n = len(x) c = random num from 1 to n return: append(substr(x,1,c),substr(y,c+1,n)

57 57 Example: n-queens Put n queens on an n × n board with no two queens on the same row, column, or diagonal

58 58 Genetic Algorithms Notes Representation of individuals –Classic approach: individual is a string over a finite alphabet with each element in the string called a gene –Usually binary instead of AGTC as in real DNA Selection strategy –Random –Selection probability proportional to fitness –Selection is done with replacement to make a very fit individual reproduce several times Reproduction –Random pairing of selected individuals –Random selection of cross-over points –Each gene can be altered by a random mutation

59 59 Genetic Algorithms When to use them? Genetic algorithms are easy to apply Results can be good on some problems, but bad on other problems Genetic algorithms are not well understood


Download ppt "1 CS 2710, ISSP 2610 Chapter 3, Part 3 Heuristic Search Chapter 4 Local Search and Optimization."

Similar presentations


Ads by Google