1. Divide and Conquer Strategy

3. General Concept of Divide & Conquer
Given a function to compute on n inputs, the divide- and-conquer strategy consists of:
splitting the inputs into k distinct subsets, 1kn, yielding k subproblems.
solving these subproblems
combining the subsolutions into solution of the whole.
if the subproblems are relatively large, then divide & conquer is applied again.
if the subproblems are small, the are solved without splitting.

4. Divide & Conquer

5. The Divide and Conquer Algorithm
Divide_Conquer(problem P) {
if Small(P) return S(P);
else {
divide P into smaller instances P1, P2, …, Pk, k1;
Apply Divide_Conquer to each of these subproblems;
return Combine(Divide_Conque(P1), Divide_Conque(P2),…, Divide_Conque(Pk)); }

6. Divde_Conquer recurrence relation
The computing time of Divide_Conquer is
T(n) is the time for Divide_Conquer on any input size n.
g(n) is the time to compute the answer directly (for small inputs)
f(n) is the time for dividing P and combining the solutions.
n small
otherwise

7. Three Steps of The Divide and Conquer Approach
The most well known algorithm design strategy:
Divide the problem into two or more smaller subproblems.
Conquer the subproblems by solving them recursively.
Combine the solutions to the subproblems into the solutions for the original problem.

8. A Typical Divide and Conquer Case
a problem of size n
subproblem 1
of size n/2
subproblem 2
of size n/2
a solution to
subproblem 1
a solution to
subproblem 2
a solution to
the original problem

9. An Example: Calculating a0 + a1 + … + an-1
ALGORITHM RecursiveSum(A[0..n-1])
//Input: An array A[0..n-1] of orderable elements
//Output: the summation of the array elements
if n > 1
return ( RecursiveSum(A[0.. n/2 – 1]) + RecursiveSum(A[n/2 .. n– 1]) )
Efficiency: (for n = 2k)
A(n) = 2A(n/2) + 1, n >1
A(1) = 0;

10. Code
int sumArray(int anArray[],int start,int end){ if(start==end) return anArray[start]; if(start<end){ int mid=(start+end)/2; int lsum=sumArray(anArray,start,mid-1); int rsum=sumArray(anArray,mid+1,end); return lsum+rsum+anArray[mid]; } return 0;

11. Bubble Sort
BubbleSort(array A, int n) for i  0 to n-1 for j  0 to n-2-i if A[j] > A[j+1] swap(A[j],A[j+1])
Animate

12. How many swaps does bubble sort make?
O(log n)
O(n)
O(n log n)
O(n2)

13. In Practice Bubble Sort in practice:
“The bubble sort has little to recommend it, except a catchy name” – D. Knuth
“the generic bad algorithm” – The Jargon File
Poor interaction with CPUs – causes cache misses
Number of swaps is O(n2)

14. Selection Sort
SelectionSort(array A, int n) for i ← 0 to n-2 do min ← i for j ← (i + 1) to n-1 do if A[j] < A[min] min ← j swap A[i] and A[min]
Animate

15. How many swaps does selection sort make?
O(log n)
O(n)
O(n log n)
O(n2)

16. In Practice Selection Sort in practice:
On average, takes about 40% the time of Bubble Sort
Requires O(n) swaps
Very fast on small lists (< ~20) – better than the O(n log n) sorts
Good as a base case

17. Insertion Sort
insertionSort(array A, int n) for i = 1 to n-1 do value = A[i] j = i-1 while j >= 0 and A[j] > value do A[j + 1] = A[j] j = j-1 A[j+1] = value
Animate

18. How many swaps does insertion sort make?
O(log n)
O(n)
O(n log n)
O(n2)

19. In Practice Insertion Sort in practice:
On average, takes about 20% the time of Bubble Sort
Requires O(n2) swaps
Finishes a sorted list in O(n) time
Very fast on small lists (< ~20) – better than the O(n log n) sorts
Good as a base case

20. Sorting Characteristics
In-place sort: A sort that does not require an extra array to hold “scratch work”
Stable sort: A sort that preserves the relative order of elements with the same value
Content sensitive: A sort that will tailor its execution to the data – runs faster for certain types of data

21. Comparison Runtime In place Stable Content Sensitive Bubble O(n2) Yes
Selection No
Insertion

22. Merge Sort

23. What is the runtime of bubble sort that sorts only the first 20 elements of a list of n elements?
(1)
(log n)
(n)
(n log n)
(n2)

24. MergeSort
MergeSort(array A, int i, int j) (1) if (j – i < 20) (2) InsertionSort(A[i…j], j-i+1) (3) else (4) int mid  (i+j)/2 (5) MergeSort(A, i, mid) (6) MergeSort(A, mid+1, j) (7) Merge(A,i,j)

25. Merge 1 1 3 2 4 3 6 4 2 5 5 6 7 7 8 8 1 2 3 4 5 6 7 8

26. Merge Function O(1) O(n) O(n) O(n) O(n)
Merge(array A, int i, int j, int mid)
(1) array tmp;
(2) p1  i, p2  mid+1, k  0
(3) while (p1  mid && p2  j)
(4) if (A[p1] < A[p2])
(5) tmp[k++] = A[p1++]
(6) else
(7) tmp[k++] = A[p2++]
(8) while (p1  mid)
(9) tmp[k++] = A[p1++]
(10) while (p2  j)
tmp[j++] = A[p2++]
copy tmp to A
O(1)
O(n)
O(n)
O(n)
O(n)

27. MergeSort (1) T(n/2) T(n/2) cn T(n) = 2T(n/2) + cn
Let T(n) be the runtime on an array of size n.
MergeSort
MergeSort(array A, int i, int j) (1) n  j – i (2) if (n < 20) (3) InsertionSort(A[i…j], j-i+1) (4) else (5) int mid  (i+j)/2 (6) MergeSort(A, i, mid) (7) MergeSort(A, mid+1, j) (8) Merge(A,i,j)
(1)
T(n/2)
T(n/2) cn
T(n) = 2T(n/2) + cn

28. Bounding T(n)
T(n) = 2T(n/2) + f(n) where f(n) = O(n)  2T(n/2) + cn = 2(2T(n/4) + c(n/2)) + cn = 4T(n/4) + 2cn = 4(2T(n/8) + c(n/4)) + 2cn = 8T(n/8) + 3cn = … = 2iT(n/2i ) + icn

29. Bounding T(n)
T(n) = 2iT(n/2i) + icn, T(1) = c’ If i = log2 n then: T(n) = 2i T(n/2i) + icn = 2log n T(n/2log n) + cnlog2n = nT(1) + cnlog2n = nc’ + cnlog2n = O(n log n)

30. Merge Sort Characteristics
Stable: Yes – no reason to swap elements unless the first is larger
In-place: No – Merge requires an extra array
Can you write an O(n) in-place merge?
Content Sensitive: No – will behave in exactly the same way, regardless of execution

31. Comparison Runtime In place Stable Content Sensitive Bubble O(n2) Yes
Selection No
Insertion
Merge
O(n log n)

32. QuickSort

33. QuickSort QuickSort(array A, int i, int j) if i < j
q  Partition(A, i, j)
QuickSort(A, i, q-1)
QuickSort(A, q+1, j)

34. Partition Algorithm: Example3 3 8 8 1 8 4 1 2 7 7 4 5 8 6 7 2 8 5 6 7 6
Return this index

35. Partition O(n) Partition(array A, int i, int j) int pivot  A[j]
for r  i to j-1
if A[r] ≤ pivot
p  p+1
swap(A[p], A[r])
swap(A[p+1], p[j])
return p+1
O(n)

36. QuickSort QuickSort(array A, int i, int j) if i < j
q  Partition(A, i, j)
QuickSort(A, i, q-1)
QuickSort(A, q+1, j)
T(n) = 2T(???) + cn

37. Runtime
Suppose the list is already sorted: T(n) = T(0) + T(n-1) + cn = 2T(0) + T(n-2) + cn + c(n-1) = 3T(0) + T(n-3) + cn + c(n-1) + c(n-2) = ... = iT(0) + T(n-i) + cn + … + c(n-i+1) = nT(0) + T(0) + cn = O(n2)

38. QuickSort QuickSort(array A, int i, int j) if i < j
q  Partition(A, i, j)
QuickSort(A, i, q-1)
QuickSort(A, q+1, j)
What is the recurrence if we manage to get the median as the pivot (in O(1) time)?

39. What is the recurrence for QuickSort if we could pick the median as the pivot in O(n) time?
T(n) = 3T(n/2) + O(n)
T(n) = T(n/3) + O(n)
T(n) = 2T(n/4) + O(n)
T(n) = T(n1/2) + O(n)
T(n) = 2T(n/2) + O(n)

40. Runtime
Suppose we can pick the “middle” element each time? T(n) = 2T(n/2) + cn = O(n log n)  Like merge sort

41. Picking a Median Can we pick a median for the pivot?
Sure – sort and pick the “middle”: O(n log n)
Now what is the recurrence?

42. What is the recurrence for QuickSort if we pick the median by sorting?
T(n) = T(n)/2 + O(n)
T(n) = 2T(n/2) + O(n)
T(n) = 3T(n/2) + O(n)
T(n) = T(n/2) + O(nlog n)
T(n) = 2T(n/2) + O(nlog n)
T(n) = 3T(n/2) + O(nlog n)

43. Picking a Median Can we pick a median for the pivot?
Sure – sort and pick the “middle”: O(n log n)
Problem: If T(n) = 2T(n/2) + cn log n then T(n) = (n log n)
We need to pick the median in O(n) time if we want the O(n log n) quick sort time
This is possible – but difficult
We will tackle this next week

44. Picking a Pivot Suppose the pivot were a random value from the list
We can show that the average runtime for QuickSort is then O(n log n)
So we do exactly that:
Pick a random element
Swap the value to the end
Proceed with Partition

45. Partition Partition(array A, int i, int j) swap(A[j], A[random(i,j)])
int pivot  A[j]
int p  i-1
for r  i to j-1
if A[r] ≤ pivot
p  p+1
swap(A[p], A[r])
swap(A[p+1], p[j])
return p+1

46. Quick Sort Characteristics
Stable: No – partition shuffles everything around In-place: Yes Content Sensitive: No (depends on pivot choice)

47. Comparison Runtime In place Stable Content Sensitive Bubble O(n2) Yes
Selection No
Insertion
Merge
O(n log n)
QuickSort
O(n log n)*
* Average case in standard implementation – worst case of O(n2)