1. Ch 13 – Backtracking + Branch-and-Bound
There are two principal approaches to tackling NP-hard problems or other “intractable” problems:
Use a strategy that guarantees solving the problem exactly but doesn’t guarantee to find a solution in polynomial time
Use an approximation algorithm that can find an approximate (sub-optimal) solution in polynomial time

2. Exact solutions The exact solution approach includes:
exhaustive search (brute force)
useful only for small instances
backtracking
eliminates some cases from consideration
branch-and-bound
further cuts down on the search
fast solutions for most instances
worst case is still exponential

3. Backtracking : Colouring Example
Region (Variable)
Assignment
Alternatives NT Q T WA
Regions/variables: will be assigned values in the order shown (top to bottom); value order is red, green, blue;
Assignment shows the values currently assigned to the variables;
Alternatives are for backtracking;
Result: exactly like (uninformed) depth-first search; in general very inefficient; NT
NSW SA V SA V
NSW Q T WA

4. Backtracking Construct the state space tree: nodes: partial solutions
edges: choices in completing solutions
Explore the state space tree using depth-first search
“Prune” non-promising nodes
DFS stops exploring subtree rooted at nodes leading to no solutions and...
“backtracks” to its parent node

5. Example: n-Queen problem
Place n queens on an n by n chess board so that no two of them are on the same row, column, or diagonal

6. State-space of the four-queens problem

7. Example: m-coloring problem
Given a graph and an integer m, color its vertices using no more than m colors so that no two adjacent vertices are the same color.
Run on the following graph with m=3. c d a e b
Careful to draw the state-space by coloring vertices in alphabetical order.
This way you get some actual “backtracking” – most small examples of graph coloring
yield a solution so fast that it is hard to appreciate what is going on.

8. Example: Ham Circuit problem
Given a graph, find one of its Hamiltonian cycles.
Run on the following graph. c d a e b f

9. Example: Subset-Sum problem
Find a subset of a given set S = {s1, s2, …, sn} whose sum is equal to a given integer d.
Run on S = {3, 5, 6, 7} and d =15.

10. Branch and Bound An enhancement of backtracking
Applicable to optimization problems
Uses a lower/upper bound for the value of the objective function for each node (partial solution) so as to:
guide the search through state-space
rule out certain branches as “unpromising”
Lower bound for minimization problems
Upper bound for maximization problems

11. Example: The job-employee assignment problem
Select 1 element in each row of the cost matrix C :
no 2 selected elements are in the same col;
and the sum is minimized
For example:
Job 1 Job 2 Job 3 Job 4
Person a
Person b
Person c
Person d
Lower bound: Any solution to this problem will have
total cost of at least: = 10.

12. Assignment problem: lower bounds
Job 1 Job 2 Job 3 Job 4
Person a
Person b
Person c
Person d
Lower bound: same formula, but remove the selected col.

13. Complete state-space

14. Knapsack problem Run BB algorithm on the following data:
W = 10 (max weight)
Elements (weight, benefit):
(4,40), (7,42), (5,25), (3,12)
Upper bound is calculated as
ub = v + (W-w)(vi+1/wi+1)
v = value of already selected items
w = weight of already selected items

15. Knapsack problem
(4,40, 10),
(7,42, 6),
(5,25, 5),
(3,12, 4)
W = 10

16. Traveling salesman problem
Given a finite set C={c1,...,cm} of cities, a distance function d(ci, cj) of nonnegative values, find the length of the minimum distance tour which includes every city exactly once.
Lower bound: average of the best 2 edges for each city
( )/2
Heuristic: Choose an order between cities to reduce space by half.

17. Traveling salesman example

18. Summary
Backtracking consists of doing a DFS of the state space tree, checking whether each node is promising and if the node is nonpromising backtracking to the its parent.
Branch and Bound employs lower bounds for minimization problems, and upper bounds for maximization problems.
It prunes branches that will lead to inferior solutions (than the ones already found).