Constraint Satisfaction CSC361 AI CSC361: Constraint Satisfaction

Constraint Satisfaction Constraint Satisfaction Problems (CSP) Backtracking search for CSPs AI CSC361: Constraint Satisfaction

Constraint Satisfaction Problems (CSPs) Standard search problem: defined by state, actions, goal-test and f(n). CSP: state is defined by variables X1, X2, ..,Xn with values from domain D1,D2, ..,Dn. goal test is a set of constraints specifying allowable combinations of values for subsets of variables CSP solution: find a complete and consistent assignment of values to variables. AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction CSP: Example 1 Map Colouring: Map of Australia to be coloured so that no neighbouring regions have the same colour. 3 colours available: Red, Green, Blue AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction CSP: Example 1 How can this be formulated as a CSP? Variables: WA, NT, Q, NSW, V, SA, and T Domain for each variable: {R, G, B} Constraints: No neighbouring regions should have the same colours e.g. WA and NT have the following constraint between them: {(R,G),(R,B),(G,R),(G,B),(B,R),(B,G)} Solution: {WA=R, NT=G, Q=R, NSW=G, V=R, SA=B, T=R}  a complete & consistent assignment AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction CSP: Example 1 AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Constraint Graph CSP can be represented as a constraint graph or network: nodes are variables, arcs are constraints. Binary CSP. AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction CSP: Example 2 8 queens problem as CSP. AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction CSP: Example 2 How can this be formulated as a CSP? Variables: V1, V2, V3, V4, V5, V6, V7, V8  Vi = row occupied by queen_i in column i. Domain for each variable: {1,2,3,…,7,8} Constraints: No attack constraint on Vi and Vj : {(1,3),(1,4)(1,5),…,(2,4)(2,5),…}  rules out (1,2) …. i.e. 22 out of 64 possible combinations Solution: {…}  a complete & consistent assignment AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction CSP: Types Discrete variables finite domains: n variables, domain size d  O(dn) complete assignments e.g., Boolean CSPs, incl.~Boolean satisfiability (NP-complete) infinite domains: integers, strings, etc. e.g., job scheduling, variables are start/end days for each job Continuous variables e.g., start/end times for Hubble Space Telescope observations linear constraints solvable in polynomial time by linear programming AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction CSP: Constraint Types Unary constraints involve a single variable, e.g., SA ≠ green Binary constraints involve pairs of variables, e.g., SA ≠ WA Higher-order constraints involve 3 or more variables, e.g., cryptarithmetic column constraints AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction CSP: Example 3 Variables: F T U W R O X1 X2 X3 Domains: {0,1,2,3,4,5,6,7,8,9} Constraints: Alldiff (F,T,U,W,R,O) O + O = R + 10 · X1 X1 + W + W = U + 10 · X2 X2 + T + T = O + 10 · X3 X3 = F, T ≠ 0, F ≠ 0 CRYPTARITHMETIC AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Real World CSPs Assignment problems e.g., who teaches what class Timetabling problems e.g., which class is offered when and where? Transportation scheduling Factory scheduling Job Shop Scheduling Notice that many real-world problems involve real-valued variables AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Solving CSPs We can use the search algos to solve CSPs. Initial state: the empty assignment { } Actions as Successor function: assign a value to an unassigned variable that does not conflict with current assignment  fail if no legal assignments Goal test: the current assignment is complete Path cost: constant cost (= 1) for each assignment action. All solutions have same cost (= n), so order of assignment does not matter but .. Using BFS: at level 1 branch factor = n*d, level 2 = (n-1)*d and at level n = n!*dn AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Backtracking Search Actions or Variable assignments are commutative, i.e., [WA = red then NT = green ] same as [ NT = green then WA = red ] Therefore, we need to consider assignments to a single variable at each node  b = d and there are dn leaves. Depth-first search for CSPs with single-variable assignments is called backtracking search Backtracking search is the basic uninformed algorithm for CSPs. AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Backtracking Search AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Backtracking Search AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Backtracking Search AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Backtracking Search AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Backtracking Search AI CSC361: Constraint Satisfaction

Improving Backtracking Search Simple backtracking can be improved by using some general purpose methods. 1. Which variable should be assigned a value next? In what order should its values be tried? 2. How does the current variable assignment affect the other unassigned variables? 3. When a path fails (i.e. dead-end is reached) can search avoid repeating this failure in other paths? AI CSC361: Constraint Satisfaction

Improving Backtracking Search Idea behind all schemes : backtracking should trace smaller tree during search. Variable & Value Ordering Look-Ahead Schemes Forward Checking Constraint Propagation Look-Back Schemes AI CSC361: Constraint Satisfaction

Variable & Value Ordering By default next un-assigned variable selected next for assignment  this may not lead to a small tree. Consider v1 = {a,b,c,d} v2 = {e,f}. Assigning v2 first and then v1 is more efficient!! AI CSC361: Constraint Satisfaction

Variable & Value Ordering Different schemes are used to order variables and values. Minimum Remaining Values (MRV)  Idea: choose a variable with fewest legal values to assign first, so that failure occurs quickly and search tree is pruned. Does not help in Australia example, because all variables have three colours. AI CSC361: Constraint Satisfaction

Variable & Value Ordering Degree Heuristic or Most Constrained Variable heuristic  Idea: assign a variable involved in largest number of constraints. This reduces the branching factor of unassigned variables and prunes search tree. Australia example: SW variable with highest degree = 5, could be chosen first. AI CSC361: Constraint Satisfaction

Variable & Value Ordering Least-constraining value: which values to try first?  Idea: choose a value that rules out the fewest choices for the neighbouring variables in the constraint graph. How should Q be assigned? AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Look-Ahead Schemes Forward Checking: whenever a varaible X is assigned, the forward checking process looks at each unassigned variable Y, connected to X, and deletes from Y’s domain any value inconsistent with the value chosen for X.  reduces the branch factor of future variables. AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Look-Ahead Schemes AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Look-Ahead Schemes AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Look-Ahead Schemes AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Look-Ahead Schemes AI CSC361: Constraint Satisfaction

Forward Checking: Example 4-Queens problem X1 {1,2,3,4} X3 X4 X2 1 3 2 4 AI CSC361: Constraint Satisfaction

Forward Checking: Example 4-Queens problem X1 {1,2,3,4} X3 X4 X2 1 3 2 4 AI CSC361: Constraint Satisfaction

Forward Checking: Example 4-Queens problem X1 {1,2,3,4} X3 { ,2, ,4} X4 { ,2,3, } X2 { , ,3,4} 1 3 2 4 AI CSC361: Constraint Satisfaction

Forward Checking: Example 4-Queens problem X1 {1,2,3,4} X3 { ,2, ,4} X4 { ,2,3, } X2 { , ,3,4} 1 3 2 4 AI CSC361: Constraint Satisfaction

Forward Checking: Example 4-Queens problem X1 {1,2,3,4} X3 { , , , } X4 { ,2,3, } X2 { , ,3,4} 1 3 2 4 DOMAIN EMPTY AI CSC361: Constraint Satisfaction

Forward Checking: Example 4-Queens problem X1 { ,2,3,4} X3 {1,2,3,4} X4 X2 1 3 2 4 AI CSC361: Constraint Satisfaction

Forward Checking: Example 4-Queens problem X1 { ,2,3,4} X3 {1, ,3, } X4 {1, ,3,4} X2 { , , ,4} 1 3 2 4 AI CSC361: Constraint Satisfaction

Forward Checking: Example 4-Queens problem X1 { ,2,3,4} X3 {1, ,3, } X4 {1, ,3,4} X2 { , , ,4} 1 3 2 4 AI CSC361: Constraint Satisfaction

Forward Checking: Example 4-Queens problem X1 { ,2,3,4} X3 {1, , , } X4 {1, ,3, } X2 { , , ,4} 1 3 2 4 AI CSC361: Constraint Satisfaction

Forward Checking: Example 4-Queens problem X1 { ,2,3,4} X3 {1, , , } X4 {1, ,3, } X2 { , , ,4} 1 3 2 4 AI CSC361: Constraint Satisfaction

Forward Checking: Example 4-Queens problem X1 { ,2,3,4} X3 {1, , , } X4 { , ,3, } X2 { , , ,4} 1 3 2 4 AI CSC361: Constraint Satisfaction

Forward Checking: Example 4-Queens problem X1 { ,2,3,4} X3 {1, , , } X4 { , ,3, } X2 { , , ,4} 1 3 2 4 AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Look-Ahead Schemes FC propagates information from assigned to unassigned variables, but doesn't provide early detection for all failures: NT and SA cannot both be blue! Constraint propagation repeatedly enforces constraints locally AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Arc-Consistency Simplest form of constraint propagation – arc consistency. X Y is consistent iff for every value x of X there is some allowed y; SA  NSW: SA=B & NSW=R AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Arc-Consistency NSW  SA is consistent iff: NSW=red and SA=blue; NSW=blue and SA=??? Arc can be made consistent by removing blue from NSW AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Arc-Consistency If X loses a value, neighbors of X need to be rechecked Remove Red from V. AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Arc-Consistency If X loses a value, neighbors of X need to be rechecked Arc-consistency detects failures early than forward checking AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Arc-Consistency function AC-3(csp) return the CSP, possibly with reduced domains inputs: csp, a binary csp with variables {X1, X2, …, Xn} local variables: queue, a queue of arcs initially the arcs in csp while queue is not empty do (Xi, Xj)  REMOVE-FIRST(queue) if REMOVE-INCONSISTENT-VALUES(Xi, Xj) then for each Xk in NEIGHBORS[Xi ] do add (Xi, Xj) to queue function REMOVE-INCONSISTENT-VALUES(Xi, Xj) return true iff we remove a value removed  false for each x in DOMAIN[Xi] do if no value y in DOMAIN[Xi] allows (x,y) to satisfy the constraints between Xi and Xj then delete x from DOMAIN[Xi]; removed  true return removed AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Look-Back Schemes Intelligent backtracking Standard form is chronological backtracking i.e. try different value for preceding variable. More intelligent, backtrack to conflict set. Set of variables that caused the failure or set of previously assigned variables that are connected to X by constraints. Backjumping moves back to most recent element of the conflict set. Forward checking can be used to determine conflict set. AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Local Search for CSPs Hill-climbing, simulated annealing typically work with "complete" states, i.e., all variables assigned To apply to CSPs: allow states with unsatisfied constraints operators reassign variable values Variable selection: randomly select any conflicted variable Value selection by min-conflicts heuristic: choose value that violates the fewest constraints i.e., hill-climb with h(n) = total number of violated constraints AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Local Search for CSPs function MIN-CONFLICTS(csp, max_steps) return solution or failure inputs: csp, a constraint satisfaction problem max_steps, the number of steps allowed before giving up current  an initial complete assignment for csp for i = 1 to max_steps do if current is a solution for csp then return current; var  a randomly chosen, conflicted variable from VARIABLES[csp]; value  the value v for var that minimizes CONFLICTS(var,v,current,csp); set var = value in current; return faiilure AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Local Search for CSPs States: 4 queens in 4 columns (44 = 256 states) Actions: move queen in column Goal test: no attacks Evaluation: h(n) = number of attacks Given random initial state, can solve n-queens in almost constant time for arbitrary n with high probability (e.g., n = 10,000,000) AI CSC361: Constraint Satisfaction

AI CSC361: Constraint Satisfaction Summary Formulating problems as CSPs Backtracking – algorithm for solving CSPs Various schemes to improve on the basic backtracking algorithm. Local Search for CSPs. AI CSC361: Constraint Satisfaction