1. Topic: Divide and Conquer
General idea:
Divide a problem into subprograms of the same kind; solve subprograms using the same approach, and combine partial solution (if necessary).
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

2. Topic: Divide and Conquer
List List 1 List 2
n elements n/2 elements n/2 elements
min, max min1, max1 min2, max2
min = MIN ( min1, min2 )
max = MAX ( max1, max2)
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

3. Topic: Divide and Conquer
The Divide-and-Conquer algorithm:
procedure Rmaxmin (i, j, fmax, fmin); // i, j are index #, fmax, begin // fmin are output parameters
case:
i = j: fmax ← fmin ← A (i);
i = j –1: if A (i) < A (j) then fmax ← A (j);
fmin ← A (i);
else fmax ← A (i);
fmin ← A (j);
else: mid ← ;
call Rmaxmin (i, mid, gmax, gmin);
call Rmaxmin (mid+1, j, hmax, hmin);
fmax ← MAX (gmax, hmax);
fmin ← MIN (gmin, hmin);
end
end;
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

4. Topic: Divide and Conquer
Example: find max and min in the array:
22, 13, -5, -8, 15, 60, 17, 31, 47 ( n = 9 )
Index:
Array:
Rmaxmin(1, 9, 60, -8)
1, 5, 22, , 9, 60, 17
1, 3, 22, , 5, 15, ,7, 60, , 9, 47, 31
1,2, 22, , 3,-5, -5
(1)
(6)
(7)
(2)
(5)
(8)
(3)
(4)
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

5. Topic: Divide and Conquer
Find the maximum and minimum
The problem: Given a list of unordered n elements, find max and min
The straightforward algorithm:
max ← min ← A (1);
for i ← 2 to n do
if A (i) > max, max ← A (i);
if A (i) < min, min ← A (i);
Key comparisons: 2(n – 1)
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

6. Topic: Divide and Conquer
Analysis: For algorithm containing recursive calls, we can use recurrence relation to find its complexity
T(n) - # of comparisons needed for Rmaxmin
Recurrence relation:
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

7. Topic: Divide and Conquer
Assume n = 2k for some integer k
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

8. Topic: Divide and Conquer
Theorem:
Claim:
They don’t have to be the same constant.
We make them the same here for simplicity.
It will not affect the overall result.
where are a, b, c constants
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

9. Topic: Divide and Conquer
Proof: Assume n = ck
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

10. Topic: Divide and Conquer
If a < c, is a constant
T(n) = bn · a constant
 T(n) = O(n)
if a = c,
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

11. Topic: Divide and Conquer
If a > c,
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

12. Topic: Divide and Conquer
2. Matrix multiplication
The problem:
Multiply two matrices A and B, each of size
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

13. Topic: Divide and Conquer
The traditional way:
use three for-loop
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

14. Topic: Divide and Conquer
The Divide-and-Conquer way:
transform the problem of multiplying A and B, each of size [n×n] into 8 subproblems, each of size
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

15. Topic: Divide and Conquer
which an2 is for addition
so, it is no improvement compared with the traditional way
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

16. Topic: Divide and Conquer
Example:
use Divide-and-Conquer way to solve it as following:
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

17. Topic: Divide and Conquer
Strassen’s matrix multiplication:
· Discover a way to compute the Cij’s using 7 multiplications and 18 additions or subtractions
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

18. Topic: Divide and Conquer
Algorithm :
procedure Strassen (n, A, B, C) // n is size, A,B the input matrices, C output matrix
begin
if n = 2,
else
(cont.)
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

19. Topic: Divide and Conquer
else
Partition A into 4 submatrices: ;
Partition B into 4 submatrices: ;
call Strassen ( ;
call Strassen (
call Strassen ( ;
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

20. Topic: Divide and Conquer
call Strassen ( ;
end;
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

21. Topic: Divide and Conquer
Analysis:
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

22. Topic: Divide and Conquer
Assume n = 2k for some integer k
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

23. Topic: Divide and Conquer
3. Integer multiplication
The problem:
Multiply two large integers (n digits)
The traditional way:
Use two loops, it takes O(n2) operations
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

24. Topic: Divide and Conquer
The Divide-and-Conquer way:
Suppose x and y are large integers, divide x into two parts (a and b), and divide y into c and d. x: y: a b c d
So, transform the problem of multiplying two n-digit integers
into four subproblems of multiplying two -digit integers.
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

25. Topic: Divide and Conquer
Worst-Case is:
However, it is same as the traditional way.
Instead, we write:
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

26. Topic: Divide and Conquer
The algorithm by Divide-and-Conquer:
function multiplication (x,y)
begin
n = MAX ( # of digits in x, # of digits in y);
if (x = 0) or (y = 0), return 0;
else if (n = 1), return x*y in the usual way;
else
m = ;
a = x divide 10m;
b = x rem 10m;
c = y divide 10m;
d = y rem 10m;
p1= MULTIPLICATION (a, c);
p2= MULTIPLICATION (b, d);
p3 = MULTIPLICATION (a+b, c+d);
return ;
end;
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

27. Topic: Divide and Conquer
4. Merge Sort
The problem: 
Given a list of n numbers, sort them in non-decreasing order
idea: Split each part into 2 equal parts and sort each part using Mergesort, then merge the two sorted sub-lists into one sorted list.
The algorithm:
procedure MergeSort (low, high)
begin if
then
call MergeSort (low, mid);
call MergeSort (mid+1, high);
call Merge (low, mid, high);
end;
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

28. Topic: Divide and Conquer
procedure Merge (low, mid, high)
begin
while (i  mid and j  high)
if A(i) < A(j), then U(k) A(i);
i++;
else U(k) A(j);
j++;
k++;
if (i > mid)
move A(j) through A(high) to U(k) through U(high);
else
move A(i) through A(mid) to U(k) through U(high);
for p  low to high
A(p)  U(p);
end;
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

29. Topic: Divide and Conquer
Analysis:
Worst-Case for MergeSort is:
Best-Case & Average-Case? Merge sort pays no attention to the original order of the list, it will keep divide the list into half until sub-lists of length 1, then start merging.
Therefore Best-Case & Average-Case are the same as the Worst-Case O(nLogn)
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

30. Topic: Divide and Conquer
5. Quick Sort
The problem:
Same as Merge Sort
Compared with Merge Sort:
Quick Sort also use Divide-and-Conquer strategy
Quick Sort eliminates merging operations
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

31. Topic: Divide and Conquer
How it works?
Use partition
Eg: Sort array 65, 70, 75, 80, 85, 60, 55, 50, 45 45 85
Pivot item 65
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

32. Topic: Divide and Conquer
The algorithm:
procedure QuickSort (p, q)
begin
if p < q,
then call Partition (p, q, pivotposition);
call QuickSort (p, pivotposition –1);
call QuickSort (pivotposition+1, q);
end;
procedure Partition (low, high, pivotposition)
v  A(low);
j  low;
for i  (low + 1) to high
if A(i) < v,
j++;
A(i)  A(j);
pivotposition  j;
A(low)  A(pivotposition);
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

33. Topic: Divide and Conquer
Analysis:
Worst-Case:
Call Partition O(n) times, each time takes O(n) steps. So O(n2), and it is worse than Merge Sort in the Worst-Case.
Best-Case:
Split the list evenly each time.
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

34. Topic: Divide and Conquer
Average-Case:
Assume pivot is selected randomly, so the probability of pivot being the kth element is equal, k.
C(n) = # of key comparison for n items
average it over all k, (CA(n) is average performance)
multiply both side by n
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

35. Topic: Divide and Conquer
replace n by n-1
subtract (2) from (1)
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

36. Topic: Divide and Conquer
divide n(n + 1) for both sides
 Best-Case = Average-Case
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

37. Topic: Divide and Conquer
6. Selection (Project #2)
The problem:
Given a list of n elements find the kth smallest one
Note: special cases: when k = 1, min
when k = n, max
The solving strategy:
Algorithm #1
Use sorting strategy like MergeSort, sort the whole list
and then return the kth element of the sorted list
Both Best-Case and Worst-Case take O(n Log n)
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

38. Topic: Divide and Conquer
Algorithm #2
idea: use “partition” (from Quick Sort) procedure repeatedly until we find the kth smallest element.
For each iteration:
If pivot position i = k  Done!
If pivot position i < k  to select (k-i)th element in the 2nd sub-list.
If pivot position i > k  to select kth element in the 1st sub-list.
Best-Case is O(n) (when pivot is the kth smallest one)
Worst-Case will call partition O(n) times,
each time takes O(n), so Worst-Case is O(n2).
i = pivot position
pivot
1st sub-list
2nd sub-list
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

39. Topic: Divide and Conquer
procedure Algorththm#2 (A, n, k) // A is array, n is # of array, k is key
begin
m  1;
j  n;
loop
call Partition (m, j, pivotposition);
case:
k = pivotposition:
return A(k);
k < pivotposition:
j  pivotposition – 1;
else:
m  pivotposition + 1;
k  k - pivotposition;
end loop;
end;
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

40. Topic: Divide and Conquer
Algorithm #3
idea: Same as Algorithm#2
except it always find a special element of the list
called MM (Median of Medians),
and use MM as the pivot.
In class KLA activity to explain MM.
We can show that both Best-Case and Worst-Case take O(n)
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

41. Topic: Divide and Conquer
By choosing pivot more carefully, we can obtain a selection algorithm with Worst-Case complexity O(n)
How: Make sure pivot v is chosen s.t. at least some fraction of the elements will be smaller than v and at least some other fraction of elements will be greater than v.
Example: 35 (n) elements divide into 7 (n/r) subsets of size 5 (r)
Elements  mm
Non-
decreasing
Order
(all columns)
mm (median of medians)
The middle row is in
non-decreasing order
Elements  mm
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

42. Topic: Divide and Conquer
By choosing pivot more carefully, we can obtain a selection algorithm with Worst-Case complexity O(n)
Use mm (median of medians) rule to choose pivot
procedure Select2 (A, n, k)
begin
if n  r, then sort A and return the kth element;
divide A into subset of size r each, ignore excess
elements, and let be the set of
medians of the subsets;
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

43. Topic: Divide and Conquer
Select ;
use “Partition” to partition A using v as the pivot;
// Assume v is at pivotposition
case:
k = pivotposition: return (v);
k < pivotposition: let S be the set of elements A (1,…, pivotposition –1),
return Select2 (S, pivotposition –1, k);
else: let R be the set of element
A (pivotposition+1,…, n),
return Select2 (R, n- pivotposition, k-pivotposition);
  end case;
end;
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

44. Topic: Divide and Conquer
Analysis:
How many Mi’s  (or ) mm?
At least
How many elements  (or ) mm?
How many elements > mm (or < mm)?
At most
Assume r = 5
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

45. Topic: Divide and Conquer
Therefore, procedure Select2 the Worst-Case complexity is:
where c is chosen sufficiently large so that
T(n)  cn for n < 24.
Proof:
Use induction to show T(n)  20  cn (n  24)
IB (induction base):
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer

46. Topic: Divide and Conquer
IH (induction hypothesis):
Suppose
IS (induction solution):
When n = m;
 T(n) = O(n) complexity of Select2 of n elements
Young
CS 331 D&A of Algo.
Topic: Divide and Conquer