Chapter 3: Finite Constraint Domains

Chapter 3: Finite Constraint Domains
Where we meet the simplest and yet the most difficult constraints, and some clever and not so clever ways to solve them. CSC Finite Constraint Domains

CSC5240 - Finite Constraint Domains
Chapter Overview FD Constraint Satisfaction Problems A Backtracking Solver Node and Arc Consistency Combining Consistency and Searching Bounds and Generalized Consistency Optimization for Arithmetic CSPs Stochastic Search Methods CSC Finite Constraint Domains

Finite Constraint Domains
An important class of constraint domains Modeling constraint problems involving choice: e.g. scheduling, routing and timetabling The greatest industrial impact of constraint programming has been on these problems CSC Finite Constraint Domains

Finite Domain CSPs A FD CSP (or simply CSP hereafter) is a triple áZ,D,Cñ Z is a finite set of variables {x1,x2,…,xn} D is a function that maps each variable x to its domain D(x), a finite set of objects C is a constraint, each primitive constraint of which on an arbitrary subsets of Z It is understood as the constraint CSC Finite Constraint Domains

Map Colouring A classic CSP is the problem of coloring a map so that no adjacent regions have the same color Can the map of Australia be colored with 3 colors ? CSC Finite Constraint Domains

Four Queens Problem- 1 Place 4 queens on a 4 x 4 chessboard so that none can take another. Q1 Q2 Q3 Q4 1 2 3 4 Four variables Q1, Q2, Q3, Q4 representing the row of the queen in each column. Domain of each variable is {1,2,3,4} One solution!  CSC Finite Constraint Domains

Four Queens Problem- 2 The constraints: Not same row Not diagonally up Not diagonally down CSC Finite Constraint Domains

Cryptarithmetic Puzzle
SEND + MORE = MONEY Variables: {S,E,N,D,M,O,R,Y} Domains: D(S) = D(M) = {1,…,9} and D(V) = {0,…,9} for each V {E,N,D,O,R,Y} Constraint: alldifferent([S,E,N,D,M,O,R,Y]) 1000S+100E+10N+D+1000M+100O+10R+E= 10000M+1000O+100N+10E+Y CSC Finite Constraint Domains

Smugglers Knapsack A smuggler, with a knapsack of capacity 9, needs to choose items to smuggle to make a profit of at least 30 What should the domains of the variables be? CSC Finite Constraint Domains

Binary CSPs A binary CSP is a CSP with unary or binary constraints only A CSP with more than unary and binary constraints are general CSPs Theoretically speaking, every general CSP can be transformed to an “equivalent” binary CSP How?? CSC Finite Constraint Domains

Graph-Related Concepts
A graph (directed or undirected) is a pair (V,U), where V is a set of nodes and U (VV) is a set of arcs, each of which is a pair of adjacent nodes For undirected graphs, (j,k) and (k,j) denote the same arc for every pair of adjacent nodes j and k CSC Finite Constraint Domains

Graph-Related Concepts (cont’d)
A hypergraph is a pair (V,U), where V is a set of nodes and U is a set of hyper-arcs, each of which is a set of nodes A constraint hypergraph of a CSP áZ,D,Cñ is a hypergraph in which each node denote a variable in Z, and each hyper-arc denote a primitive constraint in C CSC Finite Constraint Domains

Graph-Related Concepts (cont’d)
A path in a graph (or hypergraph) is a sequence of nodes drawn from the graph, where every pair of adjacent nodes in this sequence forms an arc (or hyper-arc) A path of length n is a path which goes thru n+1 (not necessarily distinct) nodes Draw the constraint hypergraphs of previous CSP examples CSC Finite Constraint Domains

CSP Solution Techniques
A CSP-solving algorithm is sound if every answer returned by the algorithm is indeed a solution of the CSP A CSP-solving algorithm is complete if every solution can be found by the algorithm Soundness and completeness are desirable properties of CSP-solving algorithms CSC Finite Constraint Domains

CSP Solution Techniques (cont’d)
CSPs are NP-complete in general In some real-life problems, an incomplete (and sometimes even unsound) but efficient algorithm is acceptable CSC Finite Constraint Domains

Domain Specific .vs. General
Encoding domain specific knowledge can gain efficiency: e.g. the N-Queen problem But … Tailor-made algorithms are costly Tailor-made algorithms are not adaptable in (even slight) change of problem specification General algorithms can often form the basis of specialized algorithms CSPs are NP-complete anyway!!! CSC Finite Constraint Domains

Three Classes of Techniques
Generate-and-Test (not really a technique) Searching Problem reduction Solution Synthesis CSC Finite Constraint Domains

Generate-and-Test Systematically generating all combinations of values from domains of variables For each generated valuation , where var() = Z, test whether  satisfies all primitive constraints in C Highly combinatorial and impractical even for small problems CSC Finite Constraint Domains

Searching - 1 Searching is fundamental in almost all areas of computer science, including AI Chronological backtracking search Labeling a variable Pick a variable x Pick an available value v from D(x), making sure that is compatible with the current valuation Consider an alternative available value in D(x) if current variable labeling violates some constraints CSC Finite Constraint Domains

Searching - 2 If all the variables are labelled, then a solution is found If, at any stage, no available value can be assigned to a variable, the label that was last picked is revised Repeat until either a solution is found or all possible combinations of labels have been tried CSC Finite Constraint Domains

A Simple Backtracking Solver
The backtracking solver: enumerates values for one variable at a time checks that no primitive constraint is false at each stage Assume satisfiable(c) returns false when primitive constraint c with no variables is unsatisfiable; and true otherwise CSC Finite Constraint Domains

Partial Satisfiability
Check if a constraint is made unsatisfiable by a primitive constraint with no variables partial_satisfiable(C) for each primitive constraint c in C if vars(c) is empty then if satisfiable(c) = false then return false return true CSC Finite Constraint Domains

Backtracking Solve - 1 back_solve(C,D) if vars(C) is empty return partial_satisfiable(C) choose x in vars(C) for each value d in D(x) let C1 be C with x replaced by d if partial_satisfiable(C1) then if back_solve(C1,D) then return true return false CSC Finite Constraint Domains

Backtracking Solve - 2 CSC Finite Constraint Domains

Backtracking Solve - 2 Choose var X domain {1,2} CSC Finite Constraint Domains

Backtracking Solve - 2 Choose var Y domain {1,2} CSC Finite Constraint Domains

Backtracking Solve - 2 partial_satisfiable false CSC Finite Constraint Domains

Backtracking Solve - 2 Choose var Z domain {1,2} CSC Finite Constraint Domains

Backtracking Solve - 2 No variables, and false CSC Finite Constraint Domains

Backtracking Solve - 2 Choose var Z domain {1,2} CSC Finite Constraint Domains

Backtracking Solve - 2 No variables, and false CSC Finite Constraint Domains

Backtracking Solve - 2 Choose var X domain {1,2} CSC Finite Constraint Domains

Searching - 3 Complexities of chronological backtracking n: number of variables e: the number of contraints a: the size of the largest domain b: the size of the largest constraint Time complexity: O(aneb) Space complexity: O(na+eb) CSC Finite Constraint Domains

Search Space - 1 Z={x,y,z}, D(x)={a,b,c,d}, D(y)={e,f,g}, D(Z)={p,q} CSC Finite Constraint Domains

Search Space - 2 Fixed variable ordering: x, y, z CSC Finite Constraint Domains

Search Space - 3 Fixed variable ordering: z, y, x CSC Finite Constraint Domains

Characteristics of Search Space
The size of search space is finite # of leaves: # of internal nodes: Variable ordering is important. Why??? Problem still dominated by The depth of the tree is fixed The number of variables with ordering Twice the number of variables without ordering CSC Finite Constraint Domains

More Characteristics Subtrees are similar With fixed variable ordering, subtrees under each branch of the same level are identical in topology Experience in searching one subtree may be useful in subsequently searching its siblings CSC Finite Constraint Domains

Problem Reduction - 1 Idea: transform a CSP into another which is hopefully easier to solve or recognizable as insoluble. For example: 3X1 + 4X2 = 5 3X1 + 4X2 = 5  4X1 - 2X2 = 7 11X1 = 19 CSC Finite Constraint Domains

Problem Reduction - 2 A CSP P = áZ,D,Cñ is reduced to P’ =áZ’,D’,C’ñ if P and P’ are equivalent x  Z (or Z’)  D’(x)  D(x) CS  C  C’S  C’  C’S  CS CSC Finite Constraint Domains

Problem Reduction - 3 Problem reduction amounts to identifying and removing redundant or infeasible values from domains and constraints A value in a domain is redundant if it is not part of any solution A tuple in a constraint is redundant if it is not a projection of any solution CSC Finite Constraint Domains

Problem Reduction - 4 Problem reduction results in domains of smaller sizes and stronger constraints If the domain of any variable becomes empty, then one can conclude that the CSP is unsatisfiable How to identify redundant values?? X{1,2,3}>=Y{2,4,5}  D(X)={2,3}  D(Y)={2} CSC Finite Constraint Domains

Problem Reduction - 5 A CSP is minimal if no domain contains any redundant values and no constraint contains any redundant tuples When a CSP áZ,D,Cñ with CZ  C is reduced to a minimal one, then CZ contains nothing but solution Every CSP áZ,D,Cñ can have CZ  C by creating dummy constraints. How??? CSC Finite Constraint Domains

Problem Reduction - 6 Impractical trying to reduce a CSP to its minimal counterpart since the process is NP-hard in general Remove only those redundant values and tuples that are recognizable relatively easily Never (almost) use problem reduction alone Problem reduction is usually coupled with searching!! CSC Finite Constraint Domains

Consistency Concepts The term “consistency” has a different meaning from that used in logic A concept defined wrt a certain property Each type of consistency is defined in such a way that if the presence of a value (or tuple) in a domain (or constraint) falsifies them, then it can be concluded to be redundant CSC Finite Constraint Domains

Node and Arc Consistency
Basic idea: find an equivalent CSP to the original one with smaller variable domains Key: examine 1 prim. constraint c at a time Node consistency: (vars(c)={x}) remove any values from the domain of x that falsify c Arc consistency: (vars(c)={x,y}) remove any values from D(x) for which there is no value in D(y) that satisfies c and vice versa CSC Finite Constraint Domains

Node consistency Primitive constraint c is node consistent with domain D if |vars(c)|  1 or if vars(c) = {x} then for each d in D(x) x assigned d is a solution of c A CSP is node consistent if each primitive constraint in the CSP is node consistent CSC Finite Constraint Domains

Node consistency Example
The following CSP is not node consistent (see Z) This CSP is node consistent The map coloring and 4-queens CSPs are node consistent. Why? CSC Finite Constraint Domains

Achieving Node consistency
node_consistent(C,D) for each prim. constraint c in C D := node_consistent_primitive(c, D) return D node_consistent_primitive(c, D) if |vars(c)| =1 then let {x} = vars(c) return D CSC Finite Constraint Domains

Arc Consistency A primitive constraint c is arc consistent with domain D if |vars{c}|  2 or vars(c) = {x,y} and for each d in D(x) there exists e in D(y) such that and similarly for y A CSP is arc consistent if each primitive constraint in the CSP is arc consistent CSC Finite Constraint Domains

Arc Consistency Examples
This CSP is node consistent but not arc consistent For example the value 4 for X and X < Y. The following equivalent CSP is arc consistent The map coloring and 4-queens CSPs are also arc consistent. CSC Finite Constraint Domains

Achieving Arc Consistency - 1
ac_revise(c, D) if |vars(c)| = 2 then return D Removes values which are not arc consistent with c CSC Finite Constraint Domains

arc_consistent_1(C,D) repeat W := D for each primitive constraint c in C D := ac_revise(c,D) until W = D return D A very naive version (there are much better) CSC Finite Constraint Domains

arc_consistent_3(C,D) Q := the set of primitive constraints from C while (Q not empty) do W := D remove a primitive constraint c from Q if ((D := ac_revise(c,D))  W) then Q := Q  {c’| vars(c)  vars(c’)  } endwhile return D CSC Finite Constraint Domains

More Consistency Algorithms
AC-4, AC-5, AC-6, AC-7, … Parallel/Distributed consistency algorithms CSC Finite Constraint Domains

Using Node and Arc Consistency
We can build constraint solvers using the consistency methods Two important kinds of domain false domain: some variable has empty domain valuation domain: each variable has a singleton domain Extend satisfiable to CSP with valuation domain CSC Finite Constraint Domains

NC and AC Solvers D := node_consistent(C,D) D := arc_consistent(C,D) if D is a false domain then return false if D is a valuation domain then return satisfiable(C,D) return unknown CSC Finite Constraint Domains

NC and AC Solver Example
Colouring Australia: with constraints WA NT SA Q NSW V T Node consistency CSC Finite Constraint Domains

Colouring Australia: with constraints WA NT SA Q NSW V T Arc consistency CSC Finite Constraint Domains

Colouring Australia: with constraints WA NT SA Q NSW V T Arc consistency Answer: unknown CSC Finite Constraint Domains

Problem Reduction + Search
Efficiency of searching can be improved by pruning off search space that contains no solution Problem reduction helps Reducing the domain size of a variable is effectively the same as pruning off branches Tightening constraints helps us to reduce search space at a later stage of search CSC Finite Constraint Domains

Striking Off a Balance CSC Finite Constraint Domains

Search Space and Basic Search
. CSC Finite Constraint Domains

Some Thoughts on Basic Search
What is wrong with basic search? Blind search: redundant values are all tried mindlessly! CSC Finite Constraint Domains

Some Thoughts on Labeling
Labeling a variable can be viewed as creating another CSP, not necessarily equivalent but usually “smaller”, from the original CSP CSC Finite Constraint Domains

Lookahead Strategies Idea: spend a bit more time at each node of the search tree to look ahead to check if any subtrees under the node can be pruned Basic strategy Commit to a label at a time, and reduce the new problem at each step to some form of consistency or to detect unsatisfiability In case of unsatisfiability, backtrack to the next available value for the last committed label CSC Finite Constraint Domains

Forward Checking (FC) Maintains the invariance that for every unlabeled variable, there exists at least one value in its domain which is compatible with the committed variable assignments When a variable assignment  is committed to, FC will remove values from the domains of the unlabeled variables which are incompatible with  CSC Finite Constraint Domains

Backtracking Consistency Solver
We can combine consistency with the backtracking solver Apply node and arc consistency before starting the backtracking solver and after each variable is given a value CSC Finite Constraint Domains

Backtrking Consist. Solver Eg
Q1 Q2 Q3 Q4 1 There is no possible value for variable Q3! Therefore, we need to choose another value for Q2. No value can be assigned to Q3 in this case! 2 3 4 CSC Finite Constraint Domains

Q1 Q2 Q3 Q4 1 We cannot find any possible value for Q4 in this case! backtracking, Find another value of Q1? Yes, Q1 = 2 backtracking, Find another value of Q2? No! Backtracking… Find another value for Q3? No! 2 3 4 CSC Finite Constraint Domains

Q1 Q2 Q3 Q4 1 2 3 4 CSC Finite Constraint Domains

Search Space with FC CSC Finite Constraint Domains

Search Space with AC CSC Finite Constraint Domains

Choices in Search Which variable to look at next? Affect the shape of the search space Which value to try next? Affect the ordering of branches Which constraint to examine next? Constraint checking can be expensive For problems requiring only a single solution tuple, heuristics can help! CSC Finite Constraint Domains

Variable-Ordering CSC Finite Constraint Domains

Value-Ordering CSC Finite Constraint Domains

Colouring Australia: with constraints WA NT SA Q NSW V T Backtracking enumeration Select a variable with domain of more than 1, T Add constraint Apply consistency Answer: true CSC Finite Constraint Domains

Hyper-arc Consistency - 1
What about primitive constraints with more than 2 variables? Hyper-arc consistency: extending arc consistency to arbitrary number of variables CSC Finite Constraint Domains

A primitive constraint c is hyper-arc consistent with domain D if |vars{c}|  2 or vars(c) = {x1,…, xn} and for each d1  D(x1),  di  D(xi) for all i  {2,…,n} such that and similarly for xi, i  {2,…,n} A CSP is hyper-arc consistent if each primitive constraint in the CSP is hyper-arc consistent CSC Finite Constraint Domains

More Consistencies Path-consistency 1-consistency, …, k-consistency Give a satisfiable CSP which is not NC (AC) Give an unsatisfiable CSP which is NC (AC) Give an NC CSP which is not AC, and vice versa CSC Finite Constraint Domains

Unfortunately determining hyper-arc consistency is NP-hard (so it is probably exponential) What can we do? CSC Finite Constraint Domains

Bounds Consistency - 2 Arithmetic CSP: constraints are on integers Range: [l..u] represents the set of integers {l, l+1, ..., u} Idea: use real number consistency and only examine the endpoints (upper and lower bounds) of the domain of each variable Define min(D,x) as minimum element in domain of x, similarly for max(D,x) CSC Finite Constraint Domains

Bounds Consistency - 3 Suppose xi has domain [li..ui] A primitive constraint c is bounds consistent with domain D if for each var x in vars(c) exist real numbers d1  [l1,u1], ..., dk  [lk,uk] for remaining vars x1, ..., xk such that is a solution of c and similarly for An arithmetic CSP is bounds consistent if all its primitive constraints are CSC Finite Constraint Domains

Bounds Consistency Example
Not bounds consistent, consider Z=2, then X-3Y=10 But the domain below is bounds consistent Compare with the hyper-arc consistent domain CSC Finite Constraint Domains

Achieving Bounds Consistency - 1
Given a current domain D we wish to modify the endpoints of domains so the result is bounds consistent Propagation rules do this CSC Finite Constraint Domains

Consider the primitive constraint X = Y + Z which is equivalent to the three forms Reasoning about minimum and maximum values: Propagation rules for the constraint X = Y + Z CSC Finite Constraint Domains

The propagation rules determine that: Hence the domains can be reduced to CSC Finite Constraint Domains

More propagation rules
Given initial domain: We determine that new domain: CSC Finite Constraint Domains

Disequations Y  Z Disequations of the form Y  Z give weak propagation rules. Only when one side takes a fixed value that equals the minimum or maximum of the other is there propagation CSC Finite Constraint Domains

Multiplication X = Y  Z - 1
If all variables are positive its simple enough Example: becomes: But what if variables can be 0 or negative? CSC Finite Constraint Domains

Calculate X bounds by examining extreme values Similarly for upper bound on X using maximum BUT this does not work for Y and Z? As long as min(D,Z) <0 and max(D,Z)>0 there is usually no bounds restriction on Y using the propagation rules Recall we are using real numbers (e.g. 4/d) CSC Finite Constraint Domains

We can wait until the range of Z is non-negative or non-positive and then use rules like division by 0: CSC Finite Constraint Domains

Bounds Consistency Algorithm
Repeatedly apply the propagation rules for each primitive constraint until there is no change in the domain We do not need to examine a primitive constraint until the domains of the variables involve are modified CSC Finite Constraint Domains

Smugglers knapsack problem (no whiskey available) CSC Finite Constraint Domains

Smugglers knapsack problem (no whiskey available) Continuing there is no further change Note how we had to reexamine the profit constraint CSC Finite Constraint Domains

Bounds Consistency Solver
D := bounds_consistent(C,D) if D is a false domain return false if D is a valuation domain return satisfiable(C,D) return unknown CSC Finite Constraint Domains

Backtrking Bounds Cons. Solver
Apply bounds consistency before starting the backtracking solver and after each variable is given a value CSC Finite Constraint Domains

Backtrking Bounds Solver Eg
Smugglers knapsack problem (whiskey available) Current domain: CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 P = 1 CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 P = 1 (0,1,3) CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 Solution Found: return true P = 1 (0,1,3) CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Backtrack Initial bounds consistency W = 0 P = 1 (0,1,3) CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 P = 1 (0,1,3) CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 P = 1 P = 2 (0,1,3) CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 P = 1 P = 2 (0,1,3) false CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Backtrack Initial bounds consistency W = 0 P = 1 P = 2 (0,1,3) false CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 P = 1 P = 2 (0,1,3) false CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 P = 1 P = 2 P = 3 (0,1,3) false CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 P = 1 P = 2 P = 3 (0,1,3) false (0,3,0) CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 W = 1 P = 1 P = 2 P = 3 (0,1,3) false (0,3,0) CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 W = 1 P = 1 P = 2 P = 3 (1,1,1) (0,1,3) false (0,3,0) CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 W = 1 W = 2 P = 1 P = 2 P = 3 (1,1,1) (0,1,3) false (0,3,0) CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 W = 1 W = 2 P = 1 P = 2 P = 3 (1,1,1) (2,0,0) (0,1,3) false (0,3,0) CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 W = 1 W = 2 P = 1 P = 2 P = 3 (1,1,1) (2,0,0) No more solutions (0,1,3) false (0,3,0) CSC Finite Constraint Domains

Generalized Consistency
Can use any consistency method with any other communicating through the domain node consistency : prim constraints with 1 var arc consistency: prim constraints with 2 vars bounds consistency: other prim. constraints Sometimes we can get more information by using complex constraints and special consistency methods CSC Finite Constraint Domains

The alldifferent Constraint
alldifferent({V1,...,Vn}) holds when each variable V1,..,Vn takes a different value alldifferent({X, Y, Z}) is equivalent to Arc consistent with domain BUT there is no solution! Identifiable by specialized consistency for alldifferent CSC Finite Constraint Domains

The alldifferent Consistency
let c be of the form alldifferent(V) while exists v in V where D(v) = {d} V := V - {v} for each v’ in V D(v’) := D(v’) - {d} DV := union of all D(v) for v in V if |DV| < |V| then return false domain return D CSC Finite Constraint Domains

Examples for alldifferent
DV = {1,2}, V={X,Y,Z} hence detect unsatisfiability DV = {1,2,3,4,5}, V={X,Y,Z,T} don’t detect unsat Maximal matching based consistency could CSC Finite Constraint Domains

Other Complex Constraints
schedule n tasks with start times Si and durations Di needing resources Ri where L resources are available at each moment array access if I = i, then X = Vi and if X != Vi then I != i CSC Finite Constraint Domains

Solution Synthesis - 1 As searching with multiple branches explored simultaneously As problem reduction since constraint for the set of all variables is created and reduced, eventually to contain solution tuples only At each stage, a partial solution (a compound label) is extended by adding one label to it at a time CSC Finite Constraint Domains

Solution Synthesis - 2 Goal is to collect the sets of legal labels for larger and larger sets of variables PROCEDURE Naïve_synthesis(Z,D,C) { Order the variables in Z as x1,x2,…,xn; part_soln[0] = {()}; FOR k = 1 to n DO part_soln[k]  { ps + <xk,vk> | ps  part_soln[k-1]  vk  D(xk)  ps + <xk,vk> satisfies all constraints on var(ps)  xk }; RETURN(part_soln[n]) } CSC Finite Constraint Domains

Optimization for FD CSPs
Domains are finite We can use a solver to build a straightforward optimizer retry_int_opt(C, D, f, best) D2 := int_solv(C,D) if D2 is a false domain then return best let sol be the solution corresponding to D2 return retry_int_opt(C /\ f < sol(f), D, f, sol) CSC Finite Constraint Domains

Retry Optimization Example
Smugglers knapsack problem (optimize profit) First solution found: Corresponding solution CSC Finite Constraint Domains

Smugglers knapsack problem (optimize profit) Next solution found: Corresponding solution CSC Finite Constraint Domains

Smugglers knapsack problem (optimize profit) No next solution! Corresponding solution CSC Finite Constraint Domains

Smugglers knapsack problem (optimize profit) No next solution! Return best solution CSC Finite Constraint Domains

Backtracking Optimization
Since the solver may use backtrack search anyway combine it with the optimization At each step in backtracking search, if best is the best solution so far add the constraint f < best(f) CSC Finite Constraint Domains

Backtracking Optimization Eg
Smugglers knapsack problem (whiskey available) Current domain: CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 P = 1 CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Current domain: Initial bounds consistency W = 0 Solution Found: add constraint P = 1 (0,1,3) CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Initial bounds consistency W = 0 P = 1 (0,1,3) CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Initial bounds consistency W = 0 P = 1 P = 2 (0,1,3) false CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Initial bounds consistency W = 0 P = 1 P = 2 P = 3 (0,1,3) false false CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Initial bounds consistency W = 0 W = 1 P = 1 P = 2 P = 3 (1,1,1) (0,1,3) false false CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Initial bounds consistency W = 0 W = 1 P = 1 P = 2 P = 3 (1,1,1) (0,1,3) false false Modify constraint CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Initial bounds consistency W = 0 W = 1 W = 2 P = 1 P = 2 P = 3 (1,1,1) false (0,1,3) false false Modify constraint CSC Finite Constraint Domains

Smugglers knapsack problem (whiskey available) Initial bounds consistency W = 0 W = 1 W = 2 P = 1 P = 2 P = 3 (1,1,1) false (0,1,3) false false Return last sol (1,1,1) CSC Finite Constraint Domains

Branch and Bound Optimization
Unlike Simplex, the previous methods don’t use the objective function to direct search Branch and bound optimization for (C,f) use simplex to find a real optimal, if solution is integer stop otherwise choose a var x with non-integer opt value d and examine the problems use the current best solution to constrain prob CSC Finite Constraint Domains

Branch and Bound Example
Smugglers knapsack problem CSC Finite Constraint Domains

Smugglers knapsack problem Solution (2,0,0) = 30 CSC Finite Constraint Domains

Smugglers knapsack problem Solution (2,0,0) = 30 false CSC Finite Constraint Domains

Smugglers knapsack problem Solution (2,0,0) = 30 false false CSC Finite Constraint Domains

Smugglers knapsack problem Solution (2,0,0) = 30 false false false CSC Finite Constraint Domains

Smugglers knapsack problem Solution (2,0,0) = 30 false false false false CSC Finite Constraint Domains

Smugglers knapsack problem Solution (2,0,0) = 30 Solution (1,1,1) = 32 false false false false CSC Finite Constraint Domains

Smugglers knapsack problem Solution (2,0,0) = 30 Solution (1,1,1) = 32 false Worse than best sol false false false CSC Finite Constraint Domains

Smugglers knapsack problem false Solution (2,0,0) = 30 Solution (1,1,1) = 32 false Worse than best sol false false false CSC Finite Constraint Domains

Smugglers knapsack problem false false Solution (2,0,0) = 30 Solution (1,1,1) = 32 false Worse than best sol false false false CSC Finite Constraint Domains

Stochastic Search Methods - 1
CSPs are combinatorial in general but many applications require timely responses Stochastic search employs heuristics and nondeterminism in the traversal of the otherwise exponential search space Stochastic search methods are usually incomplete but have been shown to be efficient in solving certain large-scale and computationally hard CSPs CSC Finite Constraint Domains

Stochastic Search Methods - 2
An important class of stochastic search is local search Initialized randomly at a certain state Explore only local neighborhood states Determine the next move based on some heuristical criteria Can be trapped in local minima or plateaus Mechanism for escaping local minima CSC Finite Constraint Domains

GENET A neural network model for solving binary CSPs Network architecture A network representation of binary CSPs Convergence procedure Solution searching CSC Finite Constraint Domains

Network Architecture - 1
Consider a CSP u1 u3 u2 {1, 2, 3} |u1 - u2| = 2 u2 < u3 CSC Finite Constraint Domains

Each variable i  Z is represented by a cluster of label nodes i, j, one for each value j in its domain u2 1 2 3 u1 1 2 3 u3 1 2 3 u1 u2 u3 {1, 2, 3} |u1 - u2| = 2 u2 < u3 CSC Finite Constraint Domains

Each label node i, j is associated with an output Vi, j Vi, j = 1, if j is assigned to i (the node is on) Vi, j = 0, otherwise (the node is off) u1 1 2 3 u2 u3 u1 u2 u3 {1, 2, 3} |u1 - u2| = 2 u2 < u3 on off CSC Finite Constraint Domains

Binary constraints are represented by weighted connections between incompatible label nodes All weights Wi, jk,l are set to -1 initially u1 1 2 3 u2 u3 u1 u2 u3 {1, 2, 3} |u1 - u2| = 2 u2 < u3 |u1 - u2| = 2 CSC Finite Constraint Domains

Binary constraints are represented by weighted connections between incompatible label nodes All weights Wi, jk,l are set to -1 initially u1 1 2 3 u2 u3 u1 u2 u3 {1, 2, 3} |u1 - u2| = 2 u2 < u3 CSC Finite Constraint Domains

The input Ii, j to a label node i, j is u1 1 2 3 u2 u3 u1 u2 u3 {1, 2, 3} |u1 - u2| = 2 u2 < u3 -1 CSC Finite Constraint Domains

Convergence Procedure - 1
Initialize the network to a random valid state loop for each cluster in parallel do change the node with max input to on end for if all nodes’ output unchanged then if input to all on nodes is zero then terminate and return the solution else update connection weight: end if end loop u1 1 2 3 u2 u3 CSC Finite Constraint Domains

Initialize the network to a random valid state loop for each cluster in parallel do change the node with max input to on end for if all nodes’ output unchanged then if input to all on nodes is zero then terminate and return the solution else update connection weight: end if end loop u1 1 2 3 u2 u3 -1 -2 -1 -1 CSC Finite Constraint Domains

Initialize the network to a random valid state loop for each cluster in parallel do change the node with max input to on end for if all nodes’ output unchanged then if input to all on nodes is zero then terminate and return the solution else update connection weight: end if end loop u1 1 2 3 u2 u3 -1 -2 -2 CSC Finite Constraint Domains

The energy E(N,S) of a state S of a GENET network N is E(N,S) is always non-positive with negative weights E(N,S0) of a solution state S0 is always zero, a global maximum CSC Finite Constraint Domains

The GENET convergence procedure carries out an optimization process for energy E(N,S) GENET is useful only for finding an arbitrary solution of a CSP No way to find all solutions of a CSP Not possible to perform optimization CSC Finite Constraint Domains

Systematic Search Backtracking tree search as backbone Consistency algorithms for pruning search space Variable- and Value-ordering heuristics Sound and complete Successfully solve many real life problems Not so good with large scale and hard CSPs CSC Finite Constraint Domains

Systematic Search: a Summary
Systematic search extends a partial assignment gradually to obtain a complete assignment, ensuring that no constraints are violated during search CSC Finite Constraint Domains

Local Search Repair-based: always dealing with complete assignments and constraints can be violated Heuristics and non-determinism in the traversal of the otherwise exponential search space Effective in solving certain large-scale and computationally hard CSPs, but incomplete Can be trapped in local minima Optionally a mechanism for escaping local minima, such as random restart or breakout CSC Finite Constraint Domains

Speeding Things Up - 1 Specific variable and value ordering heuristics Redundant constraints but little is known Exploiting specific problem characteristics Structure of constraint graph Properties of constraints and domains Decomposability and symmetry CSC Finite Constraint Domains

Speeding Things Up - 2 Intelligent backtracking techniques Hybrid solvers Cooperating constraint solvers Cooperating models CSC Finite Constraint Domains

Chapter Summary - 1 Finite domain CSPs form an important class of problems Solving of FD CSPs is essentially based on backtracking search Reduce the search using consistency methods node, arc, bounds, generalized CSC Finite Constraint Domains

Chapter Summary - 2 Variable and value orderings are also important Optimization is based on repeated solving or using a real optimizer to guide the search Stochastic search, although incomplete, are useful when (1) only an arbitrary solution is needed and (2) the application is time critical CSC Finite Constraint Domains

Homework Read the entire Chapter 3 of the textbook, including Section 3.10 (very useful) Read the articles by David McAllester and Vipin Kumar Visit the Webpage of Roman Bartak The following would take at most 2 hours Exercises: 3.3, 3.5, 3.8, 3.10, 3.13 Practical exercises: 3.2, 3.3, 3.4, 3.7 CSC Finite Constraint Domains

Acknowledgement With the consent of Peter Stuckey, I have shamelessly adopted and adapted a number of slides produced by him for use in this presentation I have also copied a few diagrams from Edward Tsang’s book I am solely responsible, however, for any errors therein CSC Finite Constraint Domains

Chapter 3: Finite Constraint Domains

Similar presentations

Presentation on theme: "Chapter 3: Finite Constraint Domains"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Chapter 3: Finite Constraint Domains

Similar presentations

Presentation on theme: "Chapter 3: Finite Constraint Domains"— Presentation transcript:

Similar presentations

About project

Feedback