1. Divide and Conquer Branch and Bound Dynamic Programming
Artificial Intelligence: COSC4362-Fall 2013
Homework #3
Sarah Al-Bassam - ID#

2. Divide and Conquer
works by recursively breaking down a problem into two or more sub-problems of the same type.
The solutions to the sub-problems are then combined to give a solution to the original problem.

3. Divide and Conquer Examples: Sorting Multiplying large numbers
Syntactic analysis 

4. Divide and Conquer Example: sort the array A [p .. r]
Divide: the array A will be divided and partitioned into two sub arrays ( A[p .. q-1] and A[q+1 .. r] )
elements in A[p .. q-1] ≤ A[q]
elements in A[q+1 .. r] >A[q].
Conquer: the two sub arrays will be sorted by recursive calls to quicksort.
Combine: the two sub arrays will already be sorted.

5. Divide and Conquer
QUICKSORT ( A, p, r ) if p < r q = PARTITION ( A, p, r ) QUICKSORT (A, p, q-1 ) QUICKSORT( A, q+1, r) PARTITION ( A, p, r ) x = A [ r ] i = p – 1 for j = p to r – 1 if A [ j ] <= x i = i + 1 exchange A [ i ] with A [ j ] exchange A [ i + 1 ] with A [ r ] return i +1

6. QUICKSORT A [ 2, 8, 7, 1, 3, 5, 6, 4] p = 0, r = 7, x = 4, A[ r ] = 4j
A[ j ]
A[ i ]
2, 8, 7, 1, 3, 5, 6, 4
-1, 0 2 1 8 7
0,1 3
1,8
2,8,1
2, 1, 7, 8, 3, 5, 6, 4
1,2 4
3,7
1,7,3
2, 1, 3, 8, 7, 5, 6, 4 5 6
Table 1: Part of tracing the QUICKSORT algorithm on the array A.

7. QUICKSORT
A [ 2, 8, 7, 1, 3, 5, 6, 4 ] p = 0 r = 7 x = 4 A[ r ] = 4 At the end, exchange A[3] (8) with A[7] (4) and return i + 1 (3). A [ 2, 1, 3, 4, 7, 5, 6, 8]

8. Branch and Bound
It is a general algorithm for finding optimal solutions of various optimization problems.
A branch-and-bound algorithm consists of a systematic enumeration of all candidate solutions.
By using upper and lower estimated bounds of the quantity being optimized, large subsets of fruitless candidates are discarded.

9. Branch and Bound 𝒙 ∈𝑺 (set of candidate solutions).
Goal → find the maximum value of 𝒇(𝒙)
𝒙 ∈𝑺 (set of candidate solutions).
Branching:
Split 𝑺 into 𝑺 𝟏, 𝑺 𝟐, … , where 𝑺 𝟏 ∪ 𝑺 𝟐 ∪ … = 𝑺
MAX 𝒇(𝒙) over 𝑺 is 𝐦𝐚𝐱 { 𝒗 𝟏, 𝒗 𝟐, … } .
Bounding:
Compute the upper and lower bounds for MAX 𝒇(𝒙) over 𝑺 .
if 𝑨tree node Lower bound < 𝑩 tree node upper bound then discard 𝑨 from the search.

10. Branch and Bound The recursion stops when:
the current candidate set 𝑺 is reduced to a single element.
or the upper bound for set 𝑺 matches the lower bound.
Either way, any element of 𝑺 will be a maximum of the function within 𝑺.

11. 0 – 1 Knapsack Problem (KP)
Given: 𝒏 items to pack in some knapsack of capacity 𝒄. each item 𝒋 has a profit 𝒑 𝒊 and weight 𝒘 𝒊
Goal: maximize the profit sum without having the weight sum exceed 𝒄.
Maximize 𝒛= 𝒋=𝟏 𝒏 𝒑 𝒋 𝒙 𝒋
Subject to 𝒋=𝟏 𝒏 𝒘 𝒋 𝒙 𝒋 ≤ c
𝒙 ∈ 𝟎 , 𝟏 𝒋=𝟏, …, n

12. 0 – 1 Knapsack Problem (KP)
EXPBRANCH:
A recursive procedure.
At each recursion will insert/remove one item to/from the knapsack.
At each recursion we have:
𝑬: Exception list
𝒖 :upper bound
𝒔 and 𝒕 where 𝒔 <𝒃 ≤𝒕
𝒙 𝒋 𝒋 ∈ 𝒔+𝟏, 𝒕−𝟏 , initially 𝒔, 𝒕 = 𝒃 −𝟏, 𝒃
𝑷:𝑝𝑟𝑜𝑓𝑖𝑡= 𝒋=𝟏 𝒏 𝒑 𝒋 𝒙 𝒋
𝑾:𝑤𝑖𝑔ℎ𝑡= 𝒋=𝟏 𝒏 𝒘 𝒋 𝒙 𝒋 ≤ c
If 𝑾 ≤ 𝒄 𝑡ℎ𝑒𝑛 𝑖𝑛𝑠𝑒𝑟𝑡 𝑠𝑜𝑚𝑒 𝑖𝑡𝑒𝑚 𝒋 ≥ 𝒕
If 𝑾 >𝒄 𝑡ℎ𝑒𝑛 𝑟𝑒𝑚𝑜𝑣𝑒 𝑠𝑜𝑚𝑒 𝑖𝑡𝑒𝑚 𝒋 ≤ 𝒔
the algorithm is based on solving an "expanding core" which initially only contains the break item, but which expanded each time the branch and bound algorithm reaches the border of the core

13. 0 – 1 Knapsack Problem (KP)
EXPBRANCH:
We backtrack when upper bound doesn’t exceed 𝒛, upper bound is determined by:

14. 0 – 1 Knapsack Problem (KP)

15. 0 – 1 Knapsack Problem (KP)
𝑛=6, 𝑐=20, 𝑏=4, 𝑎𝑛𝑑 𝜎, 𝜏 = 0, 7
Figure 1 illustrates the branching tree:
Traversed top-down in left right direction.
Each box corresponds to one procedure call.
Each down going arrow determines one iteration within the main loop of the procedure
Each up going arrow illustrates the return from a sub iteration.
Table 2: j, 𝑝 𝑖 , and 𝑤 𝑖 values for 0 – 1 KP example.

16. 0 – 1 Knapsack Problem (KP)
The final solution is the vector x = ( )
The corresponding objective is z = 25
Figure 1: Branching tree for 0 – 1 KP example.

17. Dynamic Programming applied when the sub problems overlap.
solves each sub problem just once and then saves its answer in a data structure.
avoids the work of re-computing the answer every time it solves each sub-problem.

18. Dynamic Programming To develop a dynamic programming:
Characterize the structure of the optimal solution.
Recursively define the value of the optimal solution.
Compute the value of the optimal solution.
Construct an optimal solution from the computed information.

19. Rod-Cutting Problem
Given: the price 𝒑 𝒊 where i is the length of the rod in inches = 1, 2, 3, ..,
Problem: Serling Enterprises buys long steel rods and cuts them into shorter rods, which it then sells; the management of Serling Enterprises wants to know the maximum revenue 𝒓 𝒏 obtainable by cutting up the rod and selling the pieces.
Length i 1 2 3 4 5 6 7 8 9 10
Price 𝒑 𝒊 17 20 24 30
Table 3: i and 𝑝 𝑖 values for Rod-Cutting problem example.

20. Rod-Cutting Problem CUT-ROD ( p, n )
if n == 0
return 0
q = -1
for i = 1 to n
q = max ( q, p[ i ] + CUT-ROD ( p, n-i ))
return q
The running time of CUT-ROD is exponential ( 2 𝑛 ). .
CUT-ROD calls itself recursively over and over again with the same parameter values.

21. Rod-Cutting Problem
Figure 2: The recursion tree showing the recursive calls resulting from ROD-CUT(p, n) for n = 4.

22. Table 4: Tracing the ROD-CUT algorithm on presented example.
Rod-Cutting Problem n i
p [ i ]
CUT-ROD ( p, n-i )
p [ i ] + CUT-ROD ( p, n-i )
max 4 1 2 3 5 8 9
(p, 4 - 1) = (p, 3) = 8
(p, 4 - 2) = (p, 2) = 5
(p, 4 - 3) = (p, 1) = 1
(p, 4 - 4) = (p, 0) = 0
1 + 8 = 9
5 + 5 = 10
8 + 1 = 9
9 + 0 = 9 10
(p, 3 - 1) = (p, 2) = 5
(p, 3 - 2) = (p, 1) = 1
(p, 3 - 3) = (p, 0) = 0
1 + 5 = 6
5 + 1 = 6
8 + 0 = 8
(p, 2 - 1) = (p, 1) = 1
(p, 2 - 2) = (p, 0) = 0
1 + 1 = 2
5 + 0 = 5
(p, 1 - 1) = (p, 0) = 0
1 + 0 = 1
Table 4: Tracing the ROD-CUT algorithm on presented example.

23. Rod-Cutting Problem
MEMOIZED-CUT-ROD ( p, n ) let r [ 0 .. n ] be a new array for i = 0 to n r [ i ] = -1 return MEMOIZED-CUT-ROD-AUX ( p, n, r ) MEMOIZED-CUT-ROD-AUX ( p, n, r ) if r [ n ] >= 0 return r [ n ] if n = = 0 q = 0 else q = -1 for i = 1 to n q = max (q, p [ i ] + MEMOIZED-CUT-ROD-AUX ( p, n - i, r )) r [ n ] = q return q

24. Figure 3: A reduced version of the tree of Figure 2
Rod-Cutting Problem
each sub problem will be solved only once and its solution will be saved in a data structure to refer to it when needed. The exponential time solution will be transformed into a polynomial time solution.
Figure 3: A reduced version of the tree of Figure 2

25. References
[1] D. Pisinger, “Algorithms for Knapsack Problems,” Ph.D. thesis, University of Copenhagen, Copenhagen, Denmark, Feb [2] T. Cormen, C. Leiserson, R. Rivest, C. Stein, Introduction to Algorithms, 3rd ed., Massachusetts: Massachusetts Institute of Technology, [3] The Wikipedia website [Online]. Available: [Accessed: December. 1, 2012].