1. Dr. Jouhaina Chaouachi Siala
Backtracking
Dr. Jouhaina Chaouachi Siala

2. Backtracking
Backtracking is used to solve problems in which a sequence of objects is selected from a specified set so that the sequence satisfies some criterion. 
Backtracking is used to solve a class of problems known as Constraint Satisfaction Problems (CSP).
All possible solutions defines the State space of the problem and can be represented as a tree called the State Space Tree.
Backtracking is a modified depth-first search of a tree.
Often the goal is to find any feasible solution rather than an optimal solution. 

3. Depth-First Search of a Tree
In backtracking we  traverse the state-space tree in a depth first order.
Consider the state-space tree shown below:   
Depth-first traversal:
Follow a path as deep as possible until a leaf is reached.
At a leaf, back up until
reach a node with unvisited child.
Repeat again 1 2 3 7 8 4 5 6 11 12 16 13 14 15 9 10
Nodes are numbered in the order of
a depth-first traversal of the tree.

4. Backtracking: n-Queens Problem -I-
Problem: Place n queens on an n x n chessboard so that no two
queens threaten each other.
No two queens threaten each other: Not in the same row.
Assign each queen to a different row. Each queen can be placed in one of the four columns x 4 x 4 x 4 = 256 possible solutions. 1 2 3 4
1,1
1,2
1,3
1,4
2,1
3,1
4,1
4,4
Q in row 1 is
in column 1
Checked first.
Checked next

5. Backtracking: n-Queens Problem -II-
Make the search more efficient :
Not in the same column.
Not in the same diagonal.
1,1
1,2
2,1 x
2,2
3,2
4,1
4,4
2,3
2,4
3,1
4,2
4,3

6. Backtracking Revisited
Backtracking: After determining that a node leads to a “dead end” go back to the node’s parent and proceed with search on next child.
Nonpromising nodes: Cannot lead to a solution
Promising Nodes: May lead to a solution
1,1
1,2
promising
2,1
2,2
2,1
2,2
2,3
2,4 x x x x x
3,2
3,1 x x x x x x x x x x
Dead end
nonpromising
4,1
4,4
4,1
4,2
4,3 x x x x x x

7. General Algorithm for Backtracking
Backtracking algorithm is similar to a depth first search algorithm. Only difference is that in backtracking the children of a node are visited only if the node is promising and no solution at the node.
Use a function promising called the promising function (different in each application of backtracking) to determine if a node is promising.
void checknode(node v) {
node u;
if (promising(v))
if (there is a solution at v)
write the solution;
else
for (each child u of v)
checknode(u); }
void df_tree_search(node v) {
node u;
visit v;
for (each child u of v)
df_tree_search(u); }
General recursive Algorithm
for depth first search.
General Algorithm for the
Backtracking Approach

8. Efficient General Algorithm for Backtracking
Previously introduced general algorithm for backtracking makes a call to checknode before knowing if the node is promising.
A more efficient solution could be obtained by visiting only promising nodes.
void checknode(node v) {
node u;
if (promising(v))
if (there is a solution at v)
write the solution;
else
for (each child u of v)
checknode(u); }
void expand(node v) {
node u;
for (each child u of v)
if (promising(u))
if (there is a solution at u)
write the solution;
else
expand(u); }
General Algorithm for the
Backtracking Approach
Efficient General Algorithm
for the Backtracking Approach

9. The Backtracking Algorithm for the n-Queens Pb
void queens (index i) {
index j;
if (promising(i))
if (i = = n)
cout << col[1] through col[n];
else
for (j = 1; j <= n; j++) {
col[i + 1] = j;
queens(i + 1); }
col[i]: column where the queen in
the ith row is placed.
if solution found
See if queen in (i + 1)st row can be
Positioned in each of the n columns
Check if any queen threatens
Queen in the ith row.
bool promising(index i) {
index k;
bool switch;
k =1;
switch = true;
while (k < i && switch) {
if (col[i] = = col[k] || (col[i] - col[k] = =abs(i - k))
switch = false;
k++; }
return switch;
If queen in the kth row is in the
same column as queen in the
ith row, then:
col(i) = col(k)
If queen in the kth row threatens
queen in the ith row along one
of its diagonals, then:
col(i) - col(k) = i - k or
col(i) - col(k) = k - i

10. Test Run for the n-Queens Problem Algorithm
j) <3,1> is promising
k) <4,1> is nonpromising
<4,2> is nonpromising
<4,3> is promising
Solution is found.
a) <1,1> is promising
b) <2,1> is nonpromising
<2,2> is nonpromising
<2,3> is promising
c) <3,1> is nonpromising
<3,2> is nonpromising
<3,3> is nonpromising
<3,4> is nonpromising
d) backtrack to <1,1>
<2,4> is promising
e) <3,1> is nonpromising
<3,2> is promising
f) <4,1> is nonpromising
<4,2> is nonpromising
<4,3> is nonpromising
<4,4> is nonpromising
g) backtrack to <2,4>
h) backtrack to root
<1,2> is promising
i) <2,1> is nonpromising
<2,2> is nonpromising
<2,3> is nonpromising
<2,4> is promising
Root
1,1
1,2
2,1
2,2
2,1
2,2
2,3
2,4 x x x x x
3,2
3,1 x x x x x x x x x x
4,1
4,4
4,1
4,2
4,3 x x x x x x

11. Efficiency of the Backtracking Algorithm
Consider the three following algorithms:
Algorithm 1: Check all nodes in state space tree.
Algorithm 2: Eliminate nodes where two queens are in same row or in same column. Generate n! candidate solutions. Still need to check about diagonals.
Algorithm 3: Use the Backtracking approach.
number of nodes
checked by
Algoritm 1
number of candidate
Solutions checked by Algorithm 2
number of nodes
checked by
Backtracking
number of nodes
Found Promising
By Backtracking n 4 8 12 14
341
19,173,961
9.73 x 1012
1.20 x 1016 24
40,320
4.79 x 108
8.72 x 1010 61
15,721
1.1 x 107
3.78 x 108 17
2057
8.56 x 105
2.74 x 107

12. Graph Coloring: The m-Coloring Problem
Find all ways to color an undirected graph using at most m colors, so that no two adjacent vertices are of the same color.
An important application of the graph coloring problem is map coloring.
Any map can be represented by a planar graph.
A graph is called planar if it can be drawn in a plane such that no two edges cross each other. v1 v2
v color 1
v2 color 2
v3 color 3
v4 color 2
Solution for the
3-coloring Pb: v1 v3 v2 v4 v2 v3 v1 v4 v4 v3 v5 v5
No solution for the 2-coloring Pb.
Map
Planar graph

13. State Space Tree for the m-Coloring Problem
Each path from the root to a leaf is a candidate solution.
A candidate solution is a solution if any two adjacent vertices are of different colors.
Start 1 3 2 x v1 v2 v3 v4
State Space Tree
State Space Tree produced
by Backtracking

14. Backtracking Algorithm for the m-Coloring Pb
vcol[i]: is the color (integer between 1
and m) assigned to the ith vertex
void m_coloring (index i) {
int color;
if (promising(i))
if (i = = n)
cout << vcol[1] through vcol[n];
else
for (color = 1; color <= m; color++) {
vcol[i + 1] = color;
m_coloring(i + 1); }
bool promising(index i) {
index j;
bool switch;
j =1;
switch = true;
while (j < i && switch) {
if (W[i][j] && vcol[i] = = vcol[j])
switch = false;
j++;
return switch;
if solution found
Try every color for next vertex
Check if an adjacent vertex
is already this color
Graph represented by an adjacency matrix . W[i][j] is true if there is an edge between the two vertices i and j otherwise it is false.

15. Backtracking Approach to the 0-1Knapsack Problem
State space tree is constructed based on whether to include or exclude item i at level i.  
This is an optimization problem. We can not know if a node contains an optimal solution until the search is over. We will always expand a promising node to look at its children.
What constitutes a non-promising node so that we can trim the state space tree.
A node is non promising if:
Total weight at the node is at least the maximum weight
Potential profit for an expansion from that node is not better than the best profit seen so far.
One way to find a bound on this potential is to assume that the entire remaining weight capacity is filled with the most expensive (per weight) items.

16. Backtracking Approach to the 0-1Knapsack Problem
Totweight = weight +  wj
Bound = (profit +  pj ) + (W - totweight) x pk/wk
Bound  maxprofit
Bound = (16 – 7) x 5 = 115

17. State Space Tree for 0-1Knapsack Pb. Using Backtracking
i pi wi pi/wi
$ $20
2 $ $6
3 $ $5
4 $ $2
Example: 4 items and maximum weight W = 16
(0,0) $0
$115
Include Item 1
Item 1
$40 2
(1,1)
(1,2)
P1/w1 = 20
$40 2
$115 $0
$82
Item 2
$30 5 x
(2,1)
(2,2)
Exclude Item 1
P2/w2 = 6
$70 7
$115
$40 2
$98
Item 3
$50 10
Total profits so far
(3,1)
(3,2)
(3,3)
(3,4)
P3/w3 = 5
$120 17 $0
$70 7
$80
$90 12
$98
$40 2
$50
Total weight so far
Item 4
$10 5 x x
(4,1)
(4,2)
(4,3)
(4,4)
Bound on total profits
P4/w4 = 2
$80 12
$70 7
$100 17 $0
$90 12 x x x x

18. Test Run of the 0-1Knapsack Backtracking Algorithm
1) maxprft = 0;
2) Visit node (0,0)
a) prft = $0; wght = 0;
b) bond = $115
3) Visit node (1,1)
a) prft = $40; wght = 2;
b) maxprft = $40;
c) bond = $115
4) Visit node (2,1)
a) prft = $70; wght = 7;
b) maxprft = $70;
5) Visit node (3,1)
a) prft = $120; wght =17;
b) non prom.
c) bond not computed
6) Backtrack to node(2,1)
7) Visit node (3,2)
a) prft = $70; wght = 7;
b) bond = $80;
8) Visit node (4,1)
a) prft = $80; wght = 12;
b) maxprft = $80;.
c) bond = $80;
d) non prom.
9) Backtrack to node(3,2)
10) Visit node (4,2)
a) prft = $70; wght =7;
b) bond = $70
c) non prom.
11) Backtrack to node(1,1)
12) Visit node (2,2)
a) prft = $40; wght = 2;
b) bond = $98
13) Visit node (3,3)
a) prft = $90; wght = 12;
b) maxprft = $90;
c) bond = $98;
14) Visit node (4,3)
a) prft = $100; wght = 17;
b) non prom.
15) Backtrack to node(3,3)
16) Visit node (4,4)
a) prft = $90; wght =12;
b) bond = $90
c) non prom.
17) Backtrack to node(2,2)
18) Visit node (3,4)
a) prft = $40; wght = 2;
b) bond = $50;
19) Backtrack to root(0,0)
20) Visit node (1,2)
a) prft = $0; wght = 0;
b) bond = $82;
c) non prom.
21) Backtrack to root(0,0)
No more childs !
Done

19. Branch and Bound (B&B)
Branch and Bound is very similar to Backtracking.
Differences with the Backtracking approach are:
Not limited to a depth-first traversal of the state space tree (any traversal order is OK).
It is used only for optimization problems.
Visit the best nodes first: Best-first search with B&B pruning.
Best-first search is a modification of breadth-first search.
As in the case of backtracking algorithms, B&B Algorithms are usually exponential-time (or worse) in the worst case.

20. Breadth-First Search of a Tree
In B&B we  traverse the state-space tree in a modified version of a breadth first search.
Consider the state-space tree shown below:   
Breadth-first traversal: 1 2 5 7 3 6 13 14 4 10 12 11 15 16 8 9
Visit the root first.
Visit all nodes at level 1, followed by all nodes at level 2, …
At each level visit nodes from left to right
Level 1
Level 2
Level 3
Nodes are numbered in the order of
a breadth-first traversal of the tree.

21. General Algorithm for Breadth-first Search
enqueue(Q,v):
insert v at the end of the queue Q.
dequeue(Q,v):
Remove v from front of queue Q.
void bf_tree_search(tree T) {
queue_of_node Q;
node u,v;
initialize(Q);
v = root of T;
visit v;
enqueue(Q,v);
while (!empty(Q)) {
dequeue(Q,v);
for (each child u of v) {
visit u; }
Q: 1
while 1st pass
while 2nd pass
Q: null; v =1
Q: 3,4; v = 2
After for loop:
Q: 2,3,4
After for loop:
Q: 3,4,5,6,7 1 2 5 7 3 6 13 14 4 10 12 11 15 16 8 9

22. Breadth-First Search for 0-1 Knapsack Pb. Using B&B
Example: 4 items and maximum weight W = 16
(0,0) $0
$115
Item 1
$40 2
(1,1)
(1,2)
P1/w1 = 20
$40 2
$115 $0
$82
Item 2
$30 5
(2,1)
(2,2)
(2,3)
(2,4)
P2/w2 = 6
$70 7
$115
$40 2
$98
$30 5
$82 $0
$60
Item 3
$50 10
(3,1)
(3,2)
(3,3)
(3,4)
(3,5)
(3,6)
P3/w3 = 5
$120 17 $0
$70 7
$80
$90 12
$98
$40 2
$50
$80 15
$82
$30 5
$40
Item 4
$10 5
(4,1)
(4,2)
(4,3)
(4,4)
P4/w4 = 2
$80 12
$70 7
$100 17 $0
$90 12

23. General Algorithm for Breadth-First Search
void bf_bb(State_Space_Tree T
number& best) {
queue_of_node Q;
node u,v;
initialize(Q);
v = root of T;
enqueue(Q,v);
best = value(v);
while (!empty(Q)) {
dequeue(Q,v);
for (each child u of v) {
if (value(u) is better than best)
best = value(u);
if (bound(u) is better than best)
enqueue(Q,u); }

24. while(j ≤ n && totwght + w[j] ≤ W){ totwght = totwght + w[j];
void knapsack2(int n, const int p[],
const int w[], int W,
int& maxprft) {
queue_of_node Q; node u,v;
initialize(Q);
v.level = 0; v.prft = 0; v.wgth = 0;
maxprft = 0;
enqueue(Q,v);
while (!empty(Q)) {
dequeue(Q,v);
u.level = v.level + 1;
u.wght = v.wght + w[u.level];
u.prft = v.prft + p[u.level]
if (u.wght <= W && u.prft>maxprft)
maxprft = u.prft;
if (bound(u) > maxprft)
enqueue(Q,u);
u.wght = v.wght; u.prft = v.prft; }
float bound(node u) {
index j,k
int totwght;
float result;
if (u.wght >= W)
return 0;
else {
result = u.prft;
j = u.level + 1;
totwght = u.wght;
while(j ≤ n && totwght + w[j] ≤ W){
totwght = totwght + w[j];
result = result + p[j];
j+ +; }
k = j;
if (k <= n)
result = result + (W - totwght) x
p[k]/w[k];
return result; }

25. General Algorithm for Best-First Search
void bestf_bb(State_Space_Tree T
number& best) {
priority_queue_of_node PQ;
node u,v;
initialize(PQ);
v = root of T;
insert(PQ,v);
best = value(v);
while (!empty(PQ)) {
remove(PQ,v);
if (bound(v) is better than best)
for (each child u of v) {
if (value(u) is better than best)
best = value(u);
if (bound(u) is better than best)
insert(PQ,u); }

26. while(j ≤ n && totwght + w[j] ≤ W){ totwght = totwght + w[j];
void knapsack3(int n, const int p[],
const int w[], int W,
int& maxprft) {
prty_queue_of_node PQ; node u,v;
initialize(PQ);
v.level = 0; v.prft = 0; v.wgth = 0;
maxprft = 0; v.bond = bond(v);
insert(PQ,v);
while (!empty(PQ)) {
remove(PQ,v);
if (v.bound > maxprft) {
u.level = v.level + 1;
u.wght = v.wght + w[u.level];
u.prft = v.prft + p[u.level]
if (u.wght <= W && u.prft > maxprft)
maxprft = u.prft;
u.bond = bond(u);
if (u.bond > maxprft) insert(PQ,u);
u.wght = v.wght; u.prft = v.prft;
}} }
float bond(node u) {
index j,k
int totwght;
float result;
if (u.wght >= W)
return 0;
else {
result = u.prft;
j = u.level + 1;
totwght = u.wght;
while(j ≤ n && totwght + w[j] ≤ W){
totwght = totwght + w[j];
result = result + p[j];
j+ +; }
k = j;
if (k <= n)
result = result + (W - totwght) x
p[k]/w[k];
return result; }

27. Best First State Space Tree for 0-1Knapsack Pb. Using B&B
Example: 4 items and maximum weight W = 16
(0,0) $0
$115
Item 1
$40 2
(1,1)
(1,2)
P1/w1 = 20
$40 2
$115 $0
$82
Item 2
$30 5
(2,1)
(2,2)
P2/w2 = 6
$70 7
$115
$40 2
$98
Item 3
$50 10
(3,1)
(3,2)
(3,3)
(3,4)
P3/w3 = 5
$120 17 $0
$70 7
$80
$90 12
$98
$40 2
$50
Item 4
$10 5
(4,1)
(4,2)
P4/w4 = 2
$100 17 $0
$90 12

28. Test Run of the 0-1Knapsack Best-First Algorithm
1) Visit node (0,0)
a) prft = $0; wght = 0;
b) bond = $115
c) maxprft = 0;
2) Visit node (1,1)
a) prft = $40; wght = 2;
b) maxprft = $40;
c) bond = $115
3) Visit node (1,2)
b) bond = $82
4) Find prom. not expanded
higher bond node: (1,1)
a) visit child of (1,1)
5) Visit node (2,1)
a) prft = $70; wght = 7;
b) maxprft = $70;
6) Visit node (2,2)
b) bond = $98
7) Find prom. not expanded
higher bond node: (2,1)
a) visit child of (2,1)
8) Visit node (3,1)
a) prft = $120; wght =17;
b) non prom.
c) bond = $0
9) Visit node (3,2)
a) prft = $70; wght = 7;
b) bond = $80;
10) Find prom. not expanded
higher bond node: (2,2)
a) visit child of (2,2)
11) Visit node (3,3)
a) prft = $90; wght = 12;
b) maxprft = $90;
c) (1,2) , (3,2) become
non prom.
d) bond = $98;
12) Visit node (3,4)
a) prft = $40; wght =2;
b) bond = $50
c) non prom.
13) Find prom. not expanded
higher bond node: (3,3)
a) visit child of (3,3)
14) Visit node (4,1)
a) prft = $100; wght = 17;
b) non prom.
c) bond = $0;
15) Visit node (4,2)
a) prft = $90; wght = 12;
b) bond = $90;
c) non prom.
No more promising
unexpanded nodes !
Done

29. Best First State Space Tree for 0-1Knapsack Pb. Using B&B
Example: 4 items and maximum weight W = 16
(0,0) $0
$120
Item 1
$60 5
(1,1)
(1,2)
P1/w1 = 20
$60 5
$120 $0
$82
Item 2
$30 5
(2,1)
(2,2)
P2/w2 = 6
$90 10
$120
$60 5
$112
Item 3
$50 10
(3,1)
(3,2)
(3,3)
(3,4)
P3/w3 = 5
$140 20 $0
$90 10
$100
$110 15
$112
$60 5
$70
Item 4
$10 5
(4,1)
(4,2)
P4/w4 = 2
$120 20 $0
$110 15