# 1 KNOWLEDGE REPRESENTATION & REASONING Constraint Satisfaction Problems & Constraint Programming Representing Knowledge with Constraints x5x5 x6x6 x1x1.

## Presentation on theme: "1 KNOWLEDGE REPRESENTATION & REASONING Constraint Satisfaction Problems & Constraint Programming Representing Knowledge with Constraints x5x5 x6x6 x1x1."— Presentation transcript:

1 KNOWLEDGE REPRESENTATION & REASONING Constraint Satisfaction Problems & Constraint Programming Representing Knowledge with Constraints x5x5 x6x6 x1x1 x2x2 x3x3 x4x4

2 KNOWLEDGE REPRESENTATION & REASONING Properties  Computationally difficult  Technical and modeling expertise needed  Experimental in nature  Important (\$\$\$) in practice Many solution techniques  (Mixed) integer programming  Specialized methods  Local search/metaheuristics  Constraint programming KR&R for Combinatorial Problems Many, many practical applications  Resource allocation, scheduling, routing, frequency assignment, timetabling, vehicle routing, etc. Sports scheduling example  In sports league scheduling we try to build the schedule of matches between teams (e.g. football teams). There are various constraints:  Each team must play each other exactly twice (once home and once away)  No team can play more than two consecutive home or away matches  The number of times that a team plays two consecutive home or away matches must be minimum  Teams that use the same stadium cannot play home games at the same date  Games between top teams must occur at certain dates (due to TV coverage)  Etc.

3 KNOWLEDGE REPRESENTATION & REASONING Quotations “Constraint programming represents one of the closest approaches computer science has yet made to the Holy Grail of programming: the user states the problem, the computer solves it.” Eugene C. Freuder, Constraints, April 1997 “Were you to ask me which programming paradigm is likely to gain most in commercial significance over the next 5 years I’d have to pick Constraint Programming, even though it’s perhaps currently one of the least known and understood.” Dick Pountain, BYTE, February 1995

4 KNOWLEDGE REPRESENTATION & REASONING Constraint Programming Constraint programming is the continuous dream of programming  State the constraints  The solver will find a solution Knowledge representation with constraints is the preferred model in many domains  Scheduling  Vehicle routing  Resource allocation  Temporal Reasoning  … x5x5 x6x6 x1x1 x2x2 x3x3 x4x4

5 KNOWLEDGE REPRESENTATION & REASONING What is Constraint Programming? Broad Answer:  Programming where the use of constraints plays a central role  alternative to logic programming, functional programming, object- oriented programming  there are constraint programming languages that support this What is a constraint?  Let X 1,X 2,...,X n be a finite sequence of variables, each associated with a domain, D 1,D 2,...,D n.  A constraint on X 1,X 2,...,X n is a relation D 1 × D 2 × · · · × D n  can be defined explicitly/extensionally or implicitly/intentionally

6 KNOWLEDGE REPRESENTATION & REASONING What is Constraint Programming? Narrow Answer:  A more specific answer is obtained by programming with constraints in a particular manner.  Constraint programming involves solving a problem by:  Modelling: Formulate the problem as a finite set of constraints (a Constraint Satisfaction Problem).  Solving: Solve the CSP, perhaps by using a constraint programming language  Mapping: Map the solution to the CSP to a solution to the original problem

7 KNOWLEDGE REPRESENTATION & REASONING What is a Constraint? What is a constraint?  Let X 1,X 2,...,X n be a finite sequence of variables, each associated with a domain, D 1,D 2,...,D n.  A constraint on X 1,X 2,...,X n is a relation D 1 × D 2 × · · · × D n  can be defined explicitly/extensionally or implicitly/intentionally There are constraints everywhere!  Room Myrto is occupied from 12:00 until 15:00  Traffic in the web < 100 Gbytes/sec  Salary < 15k Euro  Train 1 must leave 20 minutes before train 2 arrives  Exams for 1st semester must be at least 2 days apart x+y>z (0,1,0) (1,0,0) (1,1,0) (1,1,1) explicit implicit

8 KNOWLEDGE REPRESENTATION & REASONING Constraint Programming (CP) Began in late 1970s – early 1980s from AI world  Prolog III (Marseilles, France)  CLP(R)  CHIP (ECRC, Germany) Application areas  Scheduling, sequencing, resource and personnel allocation, etc. etc. Active research area  Specialized conferences (CP, CP/AI-OR, …)  Journal (Constraints)  Companies (ILOG, COSYTEC,..) AI Software Engineering Logic Programming OR CP Discrete Mathematics These days… Let’s look at this

9 KNOWLEDGE REPRESENTATION & REASONING Constraint Programming Two main contributions  A new solving approach to combinatorial problems  AI search methods and heuristics  Orthogonal and complementary to standard OR methods  A new language for representing combinatorial problems  Rich language for constraints  Much closer to the real problem than OR  Language for search procedures  Easily extensible

10 KNOWLEDGE REPRESENTATION & REASONING The Origins Artificial Intelligence  Scene Labelling (Waltz) Interactive Graphics  Sketchpad (Sutherland)  ThingLab (Borning) Logic Programming  unification --> constraint solving Operations Research  NP-hard combinatorial problems Let’s look at this

11 KNOWLEDGE REPRESENTATION & REASONING Integer Programming (IP) Consider the manufacture of television sets. A linear programming model might give a production plan of 205.7 sets per week.  No trouble stating that production should be 205 sets per week (or even ``roughly 200 sets per week''). Suppose we were buying warehouses to store finished goods. A model that suggests we buy 0.7 warehouse at some location and 0.6 somewhere else would be of little value.  Warehouses come in integer quantities, and we would like our model to reflect that fact. This integrality restriction has far reaching effects. Modeling with integer variables has turned out to be useful far beyond restrictions to integral production quantities. With integer variables, one can model logical requirements, fixed costs, sequencing and scheduling requirements, and many other problem aspects.

12 KNOWLEDGE REPRESENTATION & REASONING Integer Programming (IP) The trouble with all this modeling power, however, is that problems with as few as 40 variables can be beyond the abilities of even the most sophisticated computers.  Most real problems with more than 100 or so variables are not possible to solve unless they show specific exploitable structure.  Despite the possibility (or even likelihood) of enormous computing times, there are methods that can be applied to solving integer programs. An IP problem in which all variables are required to be integer is called a pure integer programming problem.  If some variables are restricted to be integer and some are not then the problem is a mixed integer programming problem (MIP).  The case where the integer variables are restricted to be 0 or 1 is called pure (mixed) 0-1 programming problems or pure (mixed) binary integer programming problems.

13 KNOWLEDGE REPRESENTATION & REASONING Relationship to Linear Programming Given an integer program There is an associated linear program called the linear relaxation formed by dropping the integrality restrictions: Since (LR) is less constrained than (IP), the following are immediate: If (IP) is a minimization, the optimal objective value for (LR) is less than or equal to the optimal objective for (IP). If (IP) is a maximization, the optimal objective value for (LR) is greater than or equal to that of (IP). If (LR) is infeasible, then so is (IP). feasible solution Solving (LR) does give some information: it gives a bound on the optimal value, and, if we are lucky, may give the optimal solution to IP. But for some problems it is very difficult to even get a feasible solution!

14 KNOWLEDGE REPRESENTATION & REASONING Branch and Bound We will explain branch and bound by using this model: Maximize 8X 1 + 11X 2 + 6X 3 + 4X 4 Subject to 5X 1 + 7X 2 + 4X 3 + 3X 4 ≤ 14 X j {0,1} j = 1,…4. The linear relaxation solution is X 1 =1, X 2 = 1,X 3 = 0.5, X 4 =0 with a value of 22. We know that no integer solution will have value more than 22. Unfortunately, since X 3 is not integer, we do not have an integer solution yet. We want to force X 3 to be integer. To do so, we branch on X 3, creating two new problems. In one, we will add the constraint X 3 =0. In the other, we add the constraint X 3 = 1. Note that any optimal solution to the overall problem must be feasible to one of the subproblems. If we solve the linear relaxations of the subproblems, we get the following solutions: X 3 = 0: objective 21.65, X 1 = 1, X 2 = 1, X 3 = 0,X 4 =0.677; X 3 = 1: objective 21.85, X 1 = 1, X 2 = 0.714, X 3 = 1, X 4 = 0. At this point we know that the optimal integer solution is no more than 21.85, but we still do not have any feasible integer solution. So, we will take a subproblem and branch on one of its variables. In general, we will choose the subproblem as follows: We will choose an active subproblem, which so far only means one we have not chosen before, and we will choose the subproblem with the highest solution value (for maximization) (lowest for minimization). In this case, we will choose the subproblem with X 3 = 1, and branch on X 2. After solving the resulting subproblems, we have the branch and bound tree in Figure 2. Figure 1 Figure 2

15 KNOWLEDGE REPRESENTATION & REASONING Branch and Bound The solutions are: X 3 = 1,X 2 = 0: objective 18, X 1 = 1, X 2 =0, X 3 =1,X 4 =1; X 3 = 1, X 2 =1: objective 21.8, X 1 = 0.6, X 2 =1, X 3 =1, X 4 =0. We now have a feasible integer solution with value 18. Furthermore, since the X 3 =1, X 2 =0 problem gave an integer solution, no further branching on that problem is necessary. It is not active due to integrality of solution. There are still active subproblems that might give values more than 18. Using our rules, we will branch on problem X 3 =1, X 2 =1 by branching on X 1 to get Figure 3. The solutions are: X 3 =1, X 2 =1, X 1 =0: objective 21, X 1 =0, X 2 =1, X 3 = 1, X 4 =1 X 3 =1, X 2 =1, X 1 =1: infeasible. Our best integer solution now has value 21. The subproblem that generates that is not active due to integrality of solution. The other subproblem generated is not active due to infeasibility. There is still a subproblem that is active. It is the subproblem with solution value 21.65. There is no better integer solution for this subproblem than 21. But we already have a solution with value 21. It is not useful to search for another such solution. Therefore, we can mark this subproblem it not active. There are no longer any active subproblems, so the optimal solution value is 21. Figure 3

16 KNOWLEDGE REPRESENTATION & REASONING Constraint Satisfaction Problems At the core of Constraint Programming

17 KNOWLEDGE REPRESENTATION & REASONING What is a Constraint Satisfaction Problem? A constraint satisfaction problem (CSP) is defined by:  A set of variables X 1,…,X n  Each variable Χ i has a domain D i with its possible values  A set of constraints C 1,…,C m  Each constraint involves a subset of the variables it specifies the allowed combinations of values for this subset  A k-ary constraint C on a set of variables X 1,…,X k is a subset of the Cartesian product D 1 x…x D k  The set of variables in a constraint is called the constraint scope Binary and non-binary (or n-ary) constraint satisfaction problems x1x1 x2x2  {0,…,5}{0,…,3}

18 KNOWLEDGE REPRESENTATION & REASONING Constraint Satisfaction Problems Solution of a CSP  Assignment of a value to each variable so that all constraints are satisfied Goals:  Find one solution (feasibility problem)  Find all solutions  Find a solution that maximizes (or minimizes) some quantity  constraint optimization problem  Find an approximate “solution” All these tasks are NP-hard! (except perhaps one of them)

19 KNOWLEDGE REPRESENTATION & REASONING Constraint Graphs & Hypergraphs x5x5 x6x6 x1x1 x2x2 x3x3 x4x4 variables – nodes binary constraints – edges the label of an edge specifies the constraint x5x5 x6x6 x1x1 x2x2 x3x3 x4x4 variables – nodes n-ary constraints – hyperedges

20 KNOWLEDGE REPRESENTATION & REASONING Example – Map Coloring We want to color each area in the map with a different color We have three colors red, green, blue

21 KNOWLEDGE REPRESENTATION & REASONING Example – Map Coloring Formal Definition: Variables  WA, NT, SA, Q, NSW, V, T Domains (the same for all variables)  {red, green, blue} Constraints  C(WA,NT) = {(red, green), (red, blue), (green, red), (green, blue), (blue,red), (blue, green)}  C(WA,SA) = …

22 KNOWLEDGE REPRESENTATION & REASONING Constraint Graph WA SA NT Q NSW V T All constraints are binary Two unconnected components

23 KNOWLEDGE REPRESENTATION & REASONING Example – 8 Queens problem We want to place 8 queens on the chessboard so they can’t attack each other

24 KNOWLEDGE REPRESENTATION & REASONING Example– 8 Queens problem Formal Definition: Variables  Each variable X i (i=1,…,8) represents the column where there is the i-th queen (i.e. the queen in the i-th row) Domains  If the columns are represented by numbers from 1 to 8 then the domain of each variable X i is D i = {1,2,…,8}

25 KNOWLEDGE REPRESENTATION & REASONING Παράδειγμα – 8 Queens problem Constraints  There is a binary constraint C(X i, X j ) for each pair of variables. These constraints can be defined as follows:  For all variables X i and X j, X i  X j  For all variables X i and X j, if X i = a and X j = b then i – j  a – b and i – j  b – a

26 KNOWLEDGE REPRESENTATION & REASONING Example – Cryptoarithmetics FTU W RO X3X3 X2X2 X1X1 T W O + T W O F O U R

27 KNOWLEDGE REPRESENTATION & REASONING Example – Cryptoarithmetics Formal Definition: Variables and Domains  F, T, U, W, R, O  {0,1,2,3,4,5,6,7,8,9}  X 1, X 2, X 3  {0,1} Constraints  alldifferent(F, T, U, W, R, O)  O + O = R + 10 X 1  X 1 + W + W = U + 10 X 2  X 2 + T + T = O + 10 X 3  X 3 = F T W O + T W O F O U R

28 KNOWLEDGE REPRESENTATION & REASONING Example: Crossword puzzle 123 4 5

29 KNOWLEDGE REPRESENTATION & REASONING Crossword puzzle as a CSP Variables and their domains  X1 is 1 acrossD1 consists of all 5-letter words in the dictionary  X2 is 2 downD2 consists of all 4-letter words in the dictionary  X3 is 3 downD3 consists of all 3-letter words in the dictionary  X4 is 4 acrossD4 consists of all 4-letter words in the dictionary  X5 is 5 acrossD5 consists of all 2-letter words in the dictionary Constraints (implicit/intensional)  C12 is “the 3rd letter of X1 must equal the 1st letter of X2”  C13 is “the 5th letter of X1 must equal the 1st letter of X3”  C24 is …  C25 is …  C34 is...

30 KNOWLEDGE REPRESENTATION & REASONING 123 4 5 Variables: X1 X2 X3 X4 X5 Domains: D1 = {astar, happy, hello, hoses} D2 = {live, load, peal, peel, save, talk} D3 = {ant, oak, old} D4 = {live, load, peal, peel, save, talk} Constraints (explicit/extensional): C12 = {(astar, talk), (happy, peal), (happy, peel), (hello, live) …} C13 =... X1 X4 X2 X3 Crossword puzzle as a CSP

31 KNOWLEDGE REPRESENTATION & REASONING Real Constraint Satisfaction Problems puzzles (not really practical applications, but they are fun)  N-queens, Zebra (five house puzzle), crossword puzzle, cryptoarithmetics (SEND+MORE=MONEY), mastermind graph coloring analysis and synthesis of analog circuits option trading analysis cutting stock DNA sequencing crew scheduling chemical hypothetical reasoning warehouse location patient treatment scheduling airport counter allocation (Cathay Pacific Airways Ltd) crew rostering problem (Italian Railway Company) well activity scheduling (Saga Petroleum a.s.)

32 KNOWLEDGE REPRESENTATION & REASONING Early Commercial Applications (90s) Lufthansa: Short-term staff planning. Hongkong Container Harbor: Resource planning. Renault: Short-term production planning. Nokia: Software configuration for mobile phones. Airbus: Cabin layout. Siemens: Circuit verification. Caisse d’epargne: Portfolio management.

33 KNOWLEDGE REPRESENTATION & REASONING Applications in Research Artificial Intelligence  Machine Vision  Natural Language Understanding  Temporal and Spatial Reasoning  Theorem Proving  Qualitative Reasoning  Robotics  Agents  Planning  Timetabling  Scheduling  Vehicle Routing  Resource allocation  Frequency Assignment

34 KNOWLEDGE REPRESENTATION & REASONING Applications in Research Computer Science:  Program Analysis, Robotics, Agents Molecular Biology, Biochemestry, Bioinformatics:  Protein Folding, Genomic Sequencing Economics:  Scheduling, Stock Investment Planning Linguistics:  Parsing Medicine:  Decision Support Physics : System Modeling

35 KNOWLEDGE REPRESENTATION & REASONING CSP Technology : Practical & Successful Constraint satisfaction technology is one of the most successful examples of practical AI There are many successful companies which build and trade CSP technology  ILOG  Cosytec  Parc Technologies  i2 Technologies  IQ Software  …

36 KNOWLEDGE REPRESENTATION & REASONING CSP Technology : Practical & Successful! AKL ALE Amulet and Garnet B-Prolog Bertrand Brandeis Interval Arithmetic Constraint Solver Brandeis Interval Arithmetic Constraint Solver CHIP CIAL CLAIRE CLP CONFLEX' CPLEX Cassowary Contax Cooldraw, Deltablue, Skyblue, ThinglabII Cooldraw, Deltablue, Skyblue, ThinglabII ECLiPSe FSQP/CFSQP Goedel GNU-Prolog ICE InC++ library IF/Prolog ILOG Numerica, ILOG Schedule, ILOG Solver Interval Solver for Microsoft Excel JSolver LIFE MAC Newton Nicolog Omega Oz ProFIT Prolog III, Prolog IV Pulsar QUAD-CLP(R) Quantum Leap RISC-CLP(Real) SEL ICStus Screamer StarFLIP++ Steeles constraint system' TOY Toupie Trilogy Unicalc cu-Prolog opbdp

37 KNOWLEDGE REPRESENTATION & REASONING A real CSP – Job-shop scheduling Examples of job shop scheduling problems include  factory scheduling problems, in which some operations have to be performed within one or several shifts  spacecraft mission scheduling problems, in which time windows are determined by astronomical events over which we have no control  patient treatment scheduling problems, in which a number of patients need to receive treatment that requires certain equipment within certain time windows, etc. When solving a job shop CSP, the objective is to find as quickly as possible a feasible schedule, namely a schedule where each operation is performed within one of its legal time windows and no resource is oversubscribed.

38 KNOWLEDGE REPRESENTATION & REASONING Job-shop scheduling problem (JSSP) A JSSP requires scheduling a set of jobs J={ j 1,..., j n } on a set of physical resources RES={R 1,...,R m }  Each job j consists of a set of operations O ={O 1,...,O n } to be scheduled according to a process routing that specifies a partial ordering among these operations (e.g. O i BEFORE O j ). O 1 O 4 O 6 O 2 O 5 O 7 O 3 O 8 Job 1 O 1 O 2 O 3 Job 2

39 KNOWLEDGE REPRESENTATION & REASONING Job-shop scheduling problem (JSSP) Each job j has a release date rd j and a due date (or deadline) dd j between which all its operations have to be performed. Each operation O i has a fixed duration du i and a start time st i whose value has to be selected. The domain of possible start times of each operation is initially constrained by the release and due dates of the job to which the operation belongs.  there can be additional unary constraints that further restrict the set of admissible start times of each operation, thereby defining one or several time windows within which an operation has to be carried out  e.g. a specific shift in factory scheduling In order to be successfully executed, each operation O i requires p i different resources (e.g. a machine) R ij (1  j  p i )

40 KNOWLEDGE REPRESENTATION & REASONING The JSSP as a CSP Variables  A set of variables is associated with each operation, O i, which consists of  the operation start time, st i  its resource requirements, R ij Constraints  Precedence constraints defined by the process routings translate into linear inequalities of the type: st i +du i  st j (i.e. O i BEFORE O j )  Capacity constraints that restrict the use of each resource to only one operation at a time translate into disjunctive constraints of the form: (  p where O i,O j require R p ) st i +du i  st j  st j +du j  st i. These constraints simply express that, unless they use different resources, two operations O i and O j cannot overlap.

41 KNOWLEDGE REPRESENTATION & REASONING The JSSP as a CSP A job shop problem with 4 jobs Each node is labeled by the operation that it represents and the resource required by this operation. Each operation has a single resource requirement with a single possible value. Operation start times are the only variables.

42 KNOWLEDGE REPRESENTATION & REASONING A real CSP – The car sequencing problem In a car production scenario, cars are placed on conveyor belts which move through different work areas. A production line is normally required to produce cars of different models. The number of cars required for each model is called the production requirement. Each work area is constrained by its resource constraint or Capacity constraint. variable – one for every position in the conveyor belt (i.e. if there are n cars to be scheduled, the problem consists of n variables). domain - the set of car models, for example from model A to D. The task - to assign a value (a car model) to each variable (a position in the conveyor belt), satisfying both the production requirements and capacity constraints.

43 KNOWLEDGE REPRESENTATION & REASONING The car sequencing problem

44 KNOWLEDGE REPRESENTATION & REASONING Constraints and Databases There are close links between CSPs and relational database theory Constraint terminology CSP Variable Domain Constraint Constraint scope Constraint tuples Set of solutions Database terminology Database Attribute Attribute domain Table Table schema Table instance Join of all tables

45 KNOWLEDGE REPRESENTATION & REASONING Constraints and Databases – Example Consider the following CSP  A set of variables X = {x 0,…,x 9 }  All variables have the domain D = {0,1,2}  There are constraints with the following scopes and allowed tuples:  c 1 = {x 0,x 1,x 3 } – {(0,0,0), (0,1,0), (1,0,1), (1,1,1), (0,1,2)}  c 2 = {x 1,x 2,x 3 } – {(0,0,0), (0,0,1), (1,1,0), (1,0,1), (0,1,2)}  c 3 = {x 1,x 4 } – {(0,0), (1,1)}  c 4 = {x 3,x 6 } – {(0,0), (1,1), (1,0), (2,0)}  c 5 = {x 4,x 5,x 6 } – {(0,0,0), (0,0,1), (1,1,1), (1,0,2)}  c 6 = {x 4,x 7 } – {(0,1), (1,0)}  c 7 = {x 5,x 8 } – {(0,1), (1,0), (1,1)}  c 8 = {x 6,x 9 } – {(0,0), (1,1)}

46 KNOWLEDGE REPRESENTATION & REASONING Constraints and Databases – Example c1c1 c2c2 c3c3 c4c4 c5c5 c6c6 c7c7 c8c8 x0x0 x1x1 x3x3 x1x1 x2x2 x3x3 x1x1 x4x4 x3x3 x6x6 x4x4 x5x5 x6x6 x4x4 x7x7 x5x5 x8x8 x6x6 x9x9 0000000000 0100011110etc. 10111011 11110120 012012 The constraints as a relational database

47 KNOWLEDGE REPRESENTATION & REASONING Solving CSPs Assuming we have expressed knowledge about a problem as a CSP  how can we reason with it?  how can we find a solution (if one exists)?  how can we find all solutions?  how can we infer new knowledge? Generate and test Backtracking search algorithms Approximation algorithms Constraint propagation algorithms

48 KNOWLEDGE REPRESENTATION & REASONING Solving CSPs There are two general approaches to solving CSPs that are used in practice  Systematic Search  Explore systematically the space of all assignments  systematic = every valuation will be explored sometime  extends partial assignments  Local Search  explore the search space by small steps  start with an initial complete assignment  repairs complete assignments

49 KNOWLEDGE REPRESENTATION & REASONING First of all: Generate & Test Probably the most general problem solving method Algorithm: generate labelling test satisfaction Drawbacks: Improvements: blind generatorsmart generator --> local search late discovery oftesting within generator inconsistencies--> backtracking

50 KNOWLEDGE REPRESENTATION & REASONING Generate and test: Crossword Puzzle Try each possible combination until you find one that works:  astar – live – ant – live  astar – live – ant – load  astar – live – ant – peal  … Doesn’t check constraints until all variables have been instantiated Very inefficient way to explore the space of possibilities (4*6*3*6 = 432 for this trivial problem, most inconsistent) 123 4 5

51 KNOWLEDGE REPRESENTATION & REASONING Search Algorithms for CSPs A general search algorithm for CSPs Initial State  No value has been assigned to any variable Actions  Assign a value from D i to an unassigned variable X i Goal Test  All variables have been assigned and all constraints are satisfied The order in which the actions are executed does not matter  We can take advantage of this!

52 KNOWLEDGE REPRESENTATION & REASONING The Search Space of CSPs The search space is finite The depth of the search tree is specified  Equal to the number of variables Solutions are always at the leaves of the search tree Leaves

53 KNOWLEDGE REPRESENTATION & REASONING Search Algorithms for CSPs Which generic AI search algorithm looks suitable for CSPs? Breadth-First Search ?  No! BFS will be inefficient because solutions are always at the leaves Depth-First Search ?  Better than BFS. But it will frequently waste time searching while constraints are already violated Hill Climbing ?  Minimize conflicts

54 KNOWLEDGE REPRESENTATION & REASONING Search Algorithms for CSPs We will study variations of DFS especially for CSPs.  These algorithms are based on backtracking search Simple (or Chronological) Backtracking (BT) Backjumping (BJ) Forward Checking (FC) FC with Conflict-based Backjumping (FC-CBJ) Maintaining Arc Consistency (MAC) Also two variations of hill climbing  Min-conflicts  Min-conflicts with Random Walk

55 KNOWLEDGE REPRESENTATION & REASONING Chronological Backtracking (ΒΤ) The basic idea in all systematic backtracking-based algorithms is to start with a partial solution (i.e. assignments of a subset of the variables) and continue assigning variables until we reach a complete solution BT follows this technique  Consider the variables in some order  Pick an unassigned variable and give it a provisional value such that it is consistent with all of the constraints  If no such assignment can be made, we’ve reached a dead end and need to backtrack to the previous variable and try its next value  Continue this process until a solution is found or we backtrack to the initial variable and have exhausted all possible valaues

56 KNOWLEDGE REPRESENTATION & REASONING Chronological Backtracking (ΒΤ) solution variable 0 variable 1 variable 2 (current variable) variable 3 variable 4 current assignment Previous variables { { Future variables a b a a a a a aa a b b bb b b b b

57 KNOWLEDGE REPRESENTATION & REASONING Chronological Backtracking (ΒΤ) procedure CHRONOLOGICAL_BACKTRACKING (vars,doms,cons) solution  BT (vars,Ø,doms,cons) function BT (unlabelled,compound_label,doms,cons) returns a solution or NIL if unlabelled = Ø then return compound_label else pick a variable x from unlabelled repeat pick a value v from D x ; delete v from D x if compound_label + {(x,v)} violates no constraints then result  BT(unlabelled - {x}, compound_label + {(x,v)}, doms,cons) if result  NIL then return result end until D x = Ø return NIL end

58 KNOWLEDGE REPRESENTATION & REASONING Chronological Backtracking (in action) WA = redWA = blueWA = green WA = red NT = green WA = red NT = blue WA = red NT = green Q = red WA = red NT = green Q = blue WA = red NT = red WA = red NT = green Q = green

59 KNOWLEDGE REPRESENTATION & REASONING Backtracking: Crossword Puzzle 1 2 3 4 5  X1=astarX1=happy X2=load X2=talk X2=live … … X3=ant X3=oakX3=old astar u n a l k

60 KNOWLEDGE REPRESENTATION & REASONING Chronological Backtracking (ΒΤ) Evaluation Complete and Sound ?  Yes and Yes Time complexity: Ο(d n e)  where d is the maximum domain size, n the number of variables, and e the number of constraints Χώρος: Ο(nd)  the space required to store the domains of all variables The complexities hold under the assumption that all constraint checks are performed in constant time and constraints are stored in constant space

61 KNOWLEDGE REPRESENTATION & REASONING GT & BT – Example 1 Problem: X::{1,2}, Y::{1,2}, Z::{1,2} X = Y, X  Z, Y > Z generate & test backtracking

62 KNOWLEDGE REPRESENTATION & REASONING GT & BT 4-queen problem Place 4 queens so that no two queens are in attack. 1 2 3 4 Q1Q1 Q2Q2 Q3Q3 Q4Q4 Q i : line number of queen in column i, for 1  i  4 Q 1, Q 2, Q 3, Q 4  Q 1  Q 2, Q 1  Q 3, Q 1  Q 4, Q 2  Q 3, Q 2  Q 4, Q 3  Q 4, Q 1  Q 2 -1, Q 1  Q 2 +1, Q 1  Q 3 -2, Q 1  Q 3 +2, Q 1  Q 4 -3, Q 1  Q 4 +3, Q 2  Q 3 -1, Q 2  Q 3 +1, Q 2  Q 4 -2, Q 2  Q 4 +2, Q 3  Q 4 -1, Q 3  Q 4 +1

63 KNOWLEDGE REPRESENTATION & REASONING 4-queen problem first solution 1 2 3 4 Q1Q1 Q2Q2 Q3Q3 Q4Q4 There is a total of 256 valuations GT algorithm will generate 64valuations with Q 1 =1; +48valuations with Q 1 =2, 1  Q 2  3; +3valuations with Q 1 =2, Q 2 =4, Q 3 =1; =115valuations to find first solution

64 KNOWLEDGE REPRESENTATION & REASONING 4-queen problem, BT algorithm 1 2 3 4 Q1Q1 Q2Q2 Q3Q3 Q4Q4 1 2 3 4 Q1Q1 Q2Q2 Q3Q3 Q4Q4 1 2 3 4 Q1Q1 Q2Q2 Q3Q3 Q4Q4 1 2 3 4 Q1Q1 Q2Q2 Q3Q3 Q4Q4 1 2 3 4 Q1Q1 Q2Q2 Q3Q3 Q4Q4

65 KNOWLEDGE REPRESENTATION & REASONING Advantages declarative nature with procedural capabilities (when needed)  focus on describing the problem to be solved, and choosing the algorithm to solve it co-operative problem solving  unified framework for integration of variety of special-purpose algorithms semantic foundation  amazingly clean and elegant languages  roots in logic programming applications  proven success

66 KNOWLEDGE REPRESENTATION & REASONING Limitations NP-hard problems & tractability unpredictable behaviour ad-hoc modelling too much expertise required  new constraints, solvers, heuristics, modelling non-incremental (rescheduling) awkward handling of optimization  solvers tuned to finding first solution weak solver collaboration  with OR engines for example

67 KNOWLEDGE REPRESENTATION & REASONING Useful Links On-line guide to Constraint Programming  http://kti.ms.mff.cuni.cz/%7Ebartak/constraints/ http://kti.ms.mff.cuni.cz/%7Ebartak/constraints/ Constraints Archive  http://www.cs.unh.edu/ccc/archive/ http://www.cs.unh.edu/ccc/archive/ CSPLib : a problem library for constraints  http://4c.ucc.ie/~tw/csplib/ http://4c.ucc.ie/~tw/csplib/ Course on Theory and Practice of Constraint Satisfaction  http://www.cse.unl.edu/~choueiry/CSCE990-05/schedule.htmhttp://www.cse.unl.edu/~choueiry/CSCE990-05/schedule.htm

Download ppt "1 KNOWLEDGE REPRESENTATION & REASONING Constraint Satisfaction Problems & Constraint Programming Representing Knowledge with Constraints x5x5 x6x6 x1x1."

Similar presentations