KU NLP Heuristic Search Heuristic Search and Expert Systems (1) q An interesting approach to implementing heuristics is the use of confidence measures by expert systems to weigh the results of a rule. q Expert systems employ confidence measures to select the conclusions with the highest likelihood of success. q Games are ideal vehicles for exploring the design and behavior of heuristic search algorithm. 1. Search spaces are large enough to require heuristic pruning 2. Most games are complex enough to suggest a rich variety of heuristic evaluations for comparison and analysis. 3. Games generally do not involve complex representational issues. 4. Because each node of the state space has a common representation(e.g. a board description), a single heuristic may be applied throughout the search space.

KU NLP Heuristic Search Heuristic Search and Expert Systems (2) q More realistic problems(such as those found in expert systems applications, planning, intelligent control, and machine learning) complicate the implementation and analysis of heuristic search by requiring multiple heuristics to deal with different situations in the problem space.  A single heuristic may not apply to each state in these domains. Instead, situation specific problem-solving heuristics are encoded in the syntax and content of individual problem solving operators.

KU NLP Heuristic Search32 Example The Financial Advisor (1) q So far, the knowledge base has been treated as a set of logical implications, whose conclusions are either true or false, depending on the truth value of the premises. savings_account(adequate)  income(adequate)  investment(stocks) q In reality, it is possible that such an individual may prefer the added security of a combination strategy or even that of placing all investment money in savings  The problem solver should try to account for this uncertainty.  Expert systems have attached a numeric weight (confidence measure or certainty factor) to the conclusion of each rule.  Savings_account(adequate)  income(adequate)  investment(stocks) with confidence=0.8

KU NLP Heuristic Search33 Example The Financial Advisor (2)  Savings_account(adequate)  income(adequate)  investment(combination) with confidence=0.5  Savings_account(adequate)  income(adequate)  investment(savings) with confidence=0.1 q Heuristic search algorithms can use certainty factors in a number of ways  The results of all applicable rules could be produced with varying certainties placed on multiple conclusions.  The program might return only the result with the strongest confidence value.

KU NLP Heuristic Search Using Heuristics in Games (1) q Games are good application area for heuristic algorithm. q Tow-person games are more complicated than simple puzzles.  Hostility  maximize own advantage while minimize opponent’s opportunity of win.  Unpredictable opponent  different knowledge of games.  Credit assignment is difficult.

KU NLP Heuristic Search Using Heuristics in Games (2) q Various games  Checker  Samuel’s program had an interesting learning component which allowed its performance to improve with experience. Ultimately, the program was able to beat its author. Evaluate all states at a level with a evaluation polynomial. (C 1 *piece advantage + C 2 *advancement + C 3 *center control + C 4 *fork treat + C 5 *mobility …) If the evaluation polynomial led to a losing series of moves, the program adjusted its coefficients to improve performance.  Limitations No notion of global strategy, it may lead the program into local maxima. The learning component of the program is vulnerable to inconsistencies in the opponent play.  Go : Go is a very difficult game to play by machine since the average branching factor of the game tree is very high.  Backgammon : a backgammon program must choose its moves with incomplete information about what may happen.  Othello, Chess: achieved world-championship level.

KU NLP Heuristic Search Minimax Procedure on Exhaustively Searchable Graphs (1) q If a game whose state space is small enough to be exhaustively searched, then the problem is systematically searching the space of possible moves and counter moves by the opponent.  A game of “nim” (Fig 4.13, p145, tp37)  To predict opponent’s behavior, we assume that our opponent uses the same knowledge of the state space as we use. q MAX represents the player trying to win, or to maximize the advantage. MIN is the opponent who attempts to minimize MAX’s score.

KU NLP Heuristic Search Minimax Procedure on Exhaustively Searchable Graphs (2)

KU NLP Heuristic Search Minimax Procedure on Exhaustively Searchable Graphs (3) q In implementing minimax, label each level MIN or MAX in the search space according to whose move it is at that point in the game (Fig 4.14, p146)  If the parent state is a MAX node, give it the maximum value among its children.  If the parent is a MIN node, give the minimum value of its children.  The value assigned to each state indicates the value of the best state that this player can hope to achieve.

KU NLP Heuristic Search Minimax Procedure on Exhaustively Searchable Graphs (4)

KU NLP Heuristic Search Minimaxing to Fixed Ply Depth q In applying MINIMAX to complicated games, the state space is searched to a predefined number of levels(N- move look-ahead)  The value propagated back to the root node is simply the heuristic value of the best state that can be reached in N moves. q Many game heuristics measure the advantage of one player over another.  simple heuristic take the difference in the number of pieces belonging to MAX and MIN, and try to maximize the difference.  more sophisticated strategy might assign different values to the pieces (e.g. queen, pawn, king, checker) or location on the board. q Game playing programs typically look ahead a fixed ply depth. The states on that ply are measured heuristically and the values are propagated back up the graph using MINIMAX.

KU NLP Heuristic Search41 Minimax Search (1) q One-ply search q two-ply search C A BD (8)(3)(-2) (9) C A BD EFGHIJK (-6)(0) (-2)(-4)(-3)

KU NLP Heuristic Search42 Minimax Search (2) q Two-ply search (continued) C A BD EFGHIJK (9)(-6)(0) (-2)(-4)(-3) (-4)(-2)(-6) (-2) Maximizing ply Minimizing ply

KU NLP Heuristic Search43 Minimax Search (3)

KU NLP Heuristic Search44 Minimax Procedure (1) q When a heuristic applied with a limited look-ahead, it is possible that the depth of the look-ahead may not detect that a heuristically promising path leads to a bad situation later in the game.  Horizon effect: selection of the state may cause the entire game to be lost.  Selective deepening of search : searching several plies deeper from states that look exceptionally good. q The evaluations that take place very deep in the space can be biased by their very depth. q A desirable heuristic can be one that attempts to measure the conflict in the game.  A heuristic applied to tic-tac-toe (Fig 4.16, tp45)

KU NLP Heuristic Search45 Minimax Procedure (2)

KU NLP Heuristic Search46 Minimax Procedure (3)

KU NLP Heuristic Search47 Minimax Procedure (4)

KU NLP Heuristic Search Alpha-Beta Procedure (1) q Minimax procedure is a depth-first process, its efficiency can be improved by using branch-and- bound techniques in which partial solutions that are clearly worse than known solutions can be abandoned early. q It requires the maintenance of two threshold values (alpha and beta)  one representing a lower bound on the value that a maximizing node may be assigned  one representing an upper bound on the value that a minimizing node may be assigned.

KU NLP Heuristic Search Alpha-Beta Procedure (2) q At maximizing levels, rule out a move early if it becomes clear that its value will be less than the current threshold. q At minimizing levels, search will be terminated if values that are greater than the current threshold are discovered.

KU NLP Heuristic Search Alpha-Beta Procedure (3) q An Alpha Cutoff Maximizing ply Minimizing ply A BC DEFG (  3) (3) (5)(-5) (  -5)

KU NLP Heuristic Search Alpha-Beta Procedure (4) q Alpha and Beta Cutoffs Maximizing ply Minimizing ply Maximizing ply Minimizing ply A BC DEFG (3) (5) (  0) H IJMN KL (0  ) (0) (5)

KU NLP Heuristic Search Alpha-Beta Procedure (5) q Alpha and Beta Cutoffs (continued) A BC DEFG (5) (  5) (3)(5) H IJMN KL (0  ) (0) (5) (  7) (4) (7) Maximizing ply Minimizing ply Maximizing ply Minimizing ply

KU NLP Heuristic Search Alpha-Beta Procedure (6)