1. Constraint Satisfaction Problems

2. Constraint Satisfaction Problems (CSP) Backtracking search for CSPs
Outline
Constraint Satisfaction Problems (CSP)
Backtracking search for CSPs
Local search for CSPs

3. Constraint Satisfaction Problems
CSPs
So far, states evaluated by heuristics and goal
CSP = factored representation of state
set of variables with a value
allows for more efficient algorithms
We want to find any solution or that there’s none

4. CSPs X , a set of variables, {X1, . . . , Xn}
Formulation
X , a set of variables, {X1, , Xn}
D, a set of domains for each X . {D1, , Dn}
C, a set of constraints
Goal: find a complete, consistent assignment

5. Constraint Satisfaction Problems
Example
Color each region either red,green or blue
No adjacent region can have the same color

6. Constraint Satisfaction Probelm
Map Coloring
Color each region either red,green or blue
No adjacent region can have the same color
X = {SA, NSW , NT , Q, WA, V , T }
D = {red , blue, green} for each Xi ∈ X }
C = {((∀Xi , Xj such that Xi touches Xj ), (Color (Xi )!= Color (Xj )))}

7. Example: Map Coloring Variables: Domains:
Constraints: adjacent regions must have different colors
Solutions are assignments satisfying all constraints, e.g.:
Implicit:
Explicit:

8. CSPs Scheduling Classes
Formulating Problems Example
Scheduling Classes
Can you formulate it in terms of variables, domains and constraints?
X =?
D =?
C =?

9. CSPs
Scheduling Classes
Can you formulate it in terms of variables, domains and constraints?
X = {CS235, CS355, CS101, etc.}
D = {Mon9am, Mon11am, Mon1pm Fri4pm, Fri6pm}
C = ∀i, j if xi , xj are co-requisites, then Ti != Tj

10. Example: N-Queens
Formulation 1:
Variables:
Domains:
Constraints

12. CSPs Constraint kind: AllDiff 27 AllDiff s
Sudoku
all different constraint forces every decision variable in a given group to assume a value different from the value of every other decision variable in that group
Constraint kind: AllDiff
27 AllDiff s
To play:
See :

13. Example: Cryptarithmetic
Variables: F T U W R O X1 X2 X3
Domains: {0,1,2,3,4,5,6,7,8,9}
Constraints: Alldiff (F,T,U,W,R,O)
X2 + T + T = O + 10 · X3
X3 = F, T ≠ 0, F ≠ 0
The goal here is to assign each letter a digit from 0 to 9 so that the arithmetic works out correctly. The rules are that all occurrences of a letter must be assigned the same digit, and no digit can be assigned to more than one letter.
E.g., setting F = 1, O = 4, R = 8, T = 7, W = 3, U = 6 gives =1468

14. Varieties of constraints
Unary constraints involve a single variable,
e.g., SA ≠ green
Binary constraints involve pairs of variables,
e.g., SA ≠ WA
Higher-order constraints involve 3 or more variables,
e.g., cryptarithmetic column constraints

15. CONSTRAINT PROPAGATION: INFERENCE IN CSPS
In CSPs there is a choice:
an algorithm can search (choose a new variable assignment from several possibilities)
or do a specific type of inference INFERENCE called constraint propagation: using the constraints to reduce the number of legal values for a variable, which in turn can reduce the legal values
The key idea is local consistency. If we treat each variable as a node in a graph

16. different types of local consistency Node consistency:
A single variable (corresponding to a node in the CSP network) is node-consistent if all the values in the variable’s domain satisfy the variable’s unary constraints
Arc consistency:
A variable in a CSP is arc-consistent if every value in its domain satisfies the variable’s binary constraints.
Path consistency:
tightens the binary constraints by using implicit constraints that are inferred by looking at triples of variables.
K-consistency:
a consistent value can always be assigned to any kth variable

17. K-Consistency Increasing degrees of consistency
1-Consistency (Node Consistency): Each single node’s domain has a value which meets that node’s unary constraints
2-Consistency (Arc Consistency): For each pair of nodes, any consistent assignment to one can be extended to the other
K-Consistency: For each k nodes, any consistent assignment to k-1 can be extended to the kth node.
Higher k more expensive to compute
(You need to know the k=2 case: arc consistency)

18. Example Constraint Network
A  B A
{1,2,3,4} B
{1,2,4}
A  D
B  C
B  D D
{1,2,3,4} C
{1,3,4}
E  B
E  A
C  D
E  D
E  C E
{1,2,3,4}
Artificial Intelligence, Lecture 4.1, Page 19

19. Solving CSPs
Backtracking

20. we quickly notice something terrible
Solving CSPs
Search:Depth Limited
We could apply a standard depth-limited search
CSP with n variables with domain size d
Branching factor at top = nd
At next level: (n − 1)d
In the end n!d n leaves. But only d n possible complete assignments
we quickly notice something terrible

21. Variable assignments are commutative: (WA = red ⇒
Solving CSPs
Backtracking
Variable assignments are commutative: (WA = red ⇒
NT = green) ⇐⇒ (NT = green ⇒ WA = red )
Only need to consider assignments to a single variable at each node
Backtracks when variable has no legal value
Can solve n-queens for n ≈ 25
commutativity. A problem is commutative if the order of application of any given set of actions has no effect on the outcome
The term backtracking search is used for a depth-first search that chooses values for one variable at a time and backtracks when a variable has no legal values left to assign

22. Solving CSPs Backtracking
function BacktrackingSearch(csp)
return Backtrack({},csp)//returnssolutionorfailure
function Backtrack(assignment,csp)
if assignmentiscomplete return assignment u_var=SelectUnassignedVariable(csp)
for eachvalue in OrderDomainValues(u_var,assignment,csp) do
if isConsistent(value,assignment) add{u_var=value}toassignment inferences=Inference(csp,u_var,value) if inferences!=failure
addinferencestoassignment
result=Backtrack(assignment,csp)
if result!=failure then return result
remove{u_var=value}andinferencesfromassignment
return failure

23. Backtracking
Example (from Marc Erich Latoshik)

24. Backtracking
Example

25. Backtracking
Example

26. Backtracking
Example

28. e Really important how to choose the next variable:
Backtracking
Heuristics and Considerations
e Really important how to choose the next variable:
-Minimum Remaining Values (MRV) heuristic (fewest legal values)
-Degree Heuristics (involved in most constraints)
-Least constraining value (prefers flexibility for the future)
e Check for constraint consistency; i.e. Inference with AC-3 (arc consistency)

29. Improving backtracking efficiency
General-purpose methods can give huge gains in speed:
Which variable should be assigned next?
In what order should its values be tried?
Can we detect inevitable failure early?

30. Most constrained variable
choose the variable with the fewest legal values
a.k.a. minimum remaining values (MRV) heuristic

31. Most constraining variable
Tie-breaker among most constrained variables
Most constraining variable:
choose the variable with the most constraints on remaining variables

32. Least constraining value
Given a variable, choose the least constraining value:
Least constraining value (prefers flexibility for the future)
Combining these heuristics makes 1000 queens feasible

33. Forward Checking
The idea of searching in a tree of variable assignments is very powerful. However generic backtracking is not a very good algorithm.
• (Note that although BT is much faster than simple enumeration algorithms for solving CSPs take time that can grow exponentially with the size of the problem.)

34. Forward Checking
• Forward Checking is based on the idea of looking ahead in the tree to see if we have already made assigning a value to one of the unassigned variable impossible.
• It is based on the idea of pruning the domains of the unassigned variables.

35. Inference in CSPs Forward Checking
function ForwardChecking(csp)//returnsanewdomainfor eachvar
for eachvariableX in csp do
for eachunassignedvariableYconnectedtoX
for eachvalued in Domain(Y)
if disinconsistentwithValue(X) Domain(Y)={Domain(Y)-d} do
One of the simplest forms of inference is called forward checking. Whenever a vari-
able X is assigned, the forward-checking process establishes arc consistency for it: for each
unassigned variable Y that is connected to X by a constraint, delete from Y ’s domain any
value that is inconsistent with the value chosen for X. Because forward checking only does
arc consistency inferences, there is no reason to do forward checking if we have already done
arc consistency as a preprocessing step
return csp//whithmodifieddomains
forward checking. simplest forms of inference. Whenever a variable X is assigned, the forward-checking process establishes arc consistency for it: for each unassigned variable Y that is connected to X by a constraint, delete from Y ’s domain any value that is inconsistent with the value chosen for X. Because forward checking only does arc consistency inferences, there is no reason to do forward checking if we have already done arc consistency as a preprocessing step

36. Inference in CSP
Example WA NT Q
NSW V SA T
RGB R
R R GB
B B
RGB G G
RB R
RGB B B
RGB RGB
Domains AfterWA AfterQ AfterV
Idea:
Keep track of remaining legal values for unassigned variables
Terminate search when any variable has no legal values

37. Inference in CSP WA NT Q NSW V SA T RGB R R R GB B B RGB G G RB R
Example WA NT Q
NSW V SA T
RGB R
R R GB
B B
RGB G G
RB R
RGB B B
RGB RGB
Domains AfterWA AfterQ AfterV

38. Inference in CSP WA NT Q NSW V SA T RGB R R R GB B B RGB G G RB R
Example WA NT Q
NSW V SA T
RGB R
R R GB
B B
RGB G G
RB R
RGB B B
RGB RGB
Domains AfterWA AfterQ AfterV

39. Inference in CSP WA NT Q NSW V SA T RGB R R R GB B B RGB G G RB R
Example WA NT Q
NSW V SA T
RGB R
R R GB
B B
RGB G G
RB R
RGB B B
RGB RGB
Domains AfterWA AfterQ AfterV

40. Forward checking Idea:
Keep track of remaining legal values for unassigned variables
Terminate search when any variable has no legal values

41. Forward checking Idea:
Keep track of remaining legal values for unassigned variables
Terminate search when any variable has no legal values

42. Forward checking Idea:
Keep track of remaining legal values for unassigned variables
Terminate search when any variable has no legal values

43. Forward checking Idea:
Keep track of remaining legal values for unassigned variables
Terminate search when any variable has no legal values

44. Constraint propagation
Forward checking propagates information from assigned to unassigned variables, but doesn't provide early detection for all failures:
Constraint propagation repeatedly enforces constraints locally

45. Local search use a complete state formulation Initial assignment
Local Search in CSPs
Heuristics
Local search use a complete state formulation
Initial assignment
Change one variable at a time using heuristics

46. Local search for CSPs
Hill-climbing typically work with "complete" states, i.e., all variables assigned
To apply to CSPs:
allow states with unsatisfied constraints
operators reassign variable values
Value selection by min-conflicts heuristic:
choose value that violates the fewest constraints
i.e., hill-climb with h(n) = total number of violated constraints

47. Example: 4-Queens
Evaluation: h(n) = number of attacks
Given random initial state, can solve n-queens in almost constant time for arbitrary n with high probability (e.g., n = 10,000,000)

48. Local Search in CSPs Min-Conflicts
function MinConflicts(csp,max_steps)
//csp,max_stepsisnumofstepsbeforegivingup current=aninitialassignment for csp for i=1tomax_steps do
if currentisasolution for csp
return current
var =arandomlychosenconflictedvariable in
csp
value=thevaluev forvar thatminimizes Conflicts
set var =value in current
return failure

49. Local Search in CSPs
Example

50. Local Search in CSPs
Example

51. Local Search in CSPs
Example

53. SUMMARY
• Constraint satisfaction problems (CSPs) represent a state with a set of variable/value
pairs and represent the conditions for a solution by a set of constraints on the variables.
Many important real-world problems can be described as CSPs.
• A number of inference techniques use the constraints to infer which variable/value pairs
are consistent and which are not. These include node, arc, path, and k-consistency.
• Backtracking search, a form of depth-first search, is commonly used for solving CSPs.
Inference can be interwoven with search.
• The minimum-remaining-values and degree heuristics are domain-independent methods for deciding which variable to choose next in a backtracking search. The least constraining-value heuristic helps in deciding which value to try first for a given
variable. Backtracking occurs when no legal assignment can be found for a variable.
• Local search using the min-conflicts heuristic has also been applied to constraint satisfaction problems with great success.
• The complexity of solving a CSP is strongly related to the structure of its constraint
graph.