1. Application: Efficiency of Algorithms II
Lecture 42
Section 9.5
Mon, Apr 4, 2005

2. Algorithm Analysis
We will analyze the merge sort algorithm.

3. The Merge Sort Algorithm
The Merge Sort Algorithm sorts a list of numbers by repeatedly merging longer and longer sublists.
The initial sublists are of size 1.
The final sublist is the entire list.

4. Example Consider the following list. 88 16 2 43 57 67 79 34 5 71 62 6949 90 29 65

5. Example Each sublist of length 1 is sorted (trivially). 88 16 2 43 5767 79 34 5 71 62 69 49 90 29 65

6. Example Merge adjacent lists of length 1 into lists of length 2. 88 1643 57 67 79 34 5 71 62 69 49 90 29 65

7. Example
Merge adjacent lists of length 1 into sorted lists of length 2. 16 88 2 43 57 67 79 34 5 71 62 69 49 90 29 65

8. Example
Merge adjacent lists of length 1 into sorted lists of length 2. 16 88 2 43 57 67 79 34 5 71 62 69 49 90 29 65

9. Example
Merge adjacent lists of length 1 into sorted lists of length 2. 16 88 2 43 57 67 79 34 5 71 62 69 49 90 29 65

10. Example
Merge adjacent lists of length 1 into sorted lists of length 2. 16 88 2 43 57 67 34 79 5 71 62 69 49 90 29 65

11. Example
Merge adjacent lists of length 1 into sorted lists of length 2. 16 88 2 43 57 67 34 79 5 71 62 69 49 90 29 65

12. Example
Merge adjacent lists of length 1 into sorted lists of length 2. 16 88 2 43 57 67 34 79 5 71 62 69 49 90 29 65

13. Example
Merge adjacent lists of length 1 into sorted lists of length 2. 16 88 2 43 57 67 34 79 5 71 62 69 49 90 29 65

14. Example
Merge adjacent lists of length 1 into sorted lists of length 2. 16 88 2 43 57 67 34 79 5 71 62 69 49 90 29 65

15. Example
Then merge adjacent lists of length 2 into sorted lists of length 4. 16 88 2 43 57 67 34 79 5 71 62 69 49 90 29 65

16. Example
Then merge adjacent lists of length 2 into sorted lists of length 4. 2 16 43 88 57 67 34 79 5 71 62 69 49 90 29 65

17. Example
Then merge adjacent lists of length 2 into sorted lists of length 4. 2 16 43 88 34 57 67 79 5 71 62 69 49 90 29 65

18. Example
Then merge adjacent lists of length 2 into sorted lists of length 4. 2 16 43 88 34 57 67 79 5 62 69 71 49 90 29 65

19. Example
Then merge adjacent lists of length 2 into sorted lists of length 4. 2 16 43 88 34 57 67 79 5 62 69 71 29 49 65 90

20. Example
Then merge adjacent lists of length 4 into sorted lists of length 8. 2 16 43 88 34 57 67 79 5 62 69 71 29 49 65 90

21. Example
Then merge adjacent lists of length 4 into sorted lists of length 8. 2 16 34 43 57 67 79 88 5 62 69 71 29 49 65 90

22. Example
Then merge adjacent lists of length 4 into sorted lists of length 8. 2 16 34 43 57 67 79 88 5 29 49 62 65 69 71 90

23. Example
Finally, merge adjacent lists of length 8 into a sorted list of length 16. 2 16 34 43 57 67 79 88 5 29 49 62 65 69 71 90

24. Example
Finally, merge adjacent lists of length 8 into a sorted list of length 16. 2 5 16 29 34 43 49 47 62 65 67 69 71 79 88 90

25. Analyzing the Merge Sort
We will use two functions:
MergeSort().
Merge().
To analyze the algorithm, we will count the number of comparisons required to sort a list of length n.

26. The MergeSort() Function
The MergeSort() function is recursive.
void MergeSort(int a[], int low, int high) {
if (low < high)
int mid = (low + high)/2;
MergeSort(a, low, mid);
MergeSort(a, mid + 1, high);
Merge(a, low, mid, high); }
return;

27. The MergeSort() Function
The initial call would be to a non-recursive “starter” function.
where MergeSort() is
MergeSort(a, size);
void MergeSort(int a[], int size) {
MergeSort(a, 0, size – 1);
return; }

28. The Merge() Function
The real action takes place in the Merge() function.
This function makes a single pass down each of the two lists to be merged, comparing elements of the first list to elements of the second list.
The smaller element is copied into the new list.

29. The Merge() Function void Merge(int a[], int low, int mid, int high) {
int b[high – low + 1];
int i = low;
int j = mid + 1;
int k = 0;
while (i <= mid && j <= high)
if (a[i] < a[j])
b[k++] = a[i++];
else
b[k++] = a[j++]; } :

30. The Merge() Function : while (i <= mid) b[k++] = a[i++];
while (j <= high)
b[k++] = a[j++];
j = low;
for (i = 0; i < high; i++)
a[j++] = b[i];
return; }

31. Example 2 16 43 88 34 57 67 79

32. Example 2 16 43 88 34 57 67 79 2

33. Example 2 16 43 88 34 57 67 79 2

34. Example 2 16 43 88 34 57 67 79 2 16

35. Example 2 16 43 88 34 57 67 79 2 16

36. Example 2 16 43 88 34 57 67 79 2 16 34

37. Example 2 16 43 88 34 57 67 79 2 16 34

38. Example 2 16 43 88 34 57 67 79 2 16 34 43

39. Example 2 16 43 88 34 57 67 79 2 16 34 43

40. Example 2 16 43 88 34 57 67 79 2 16 34 43 57

41. Example 2 16 43 88 34 57 67 79 2 16 34 43 57

42. Example 2 16 43 88 34 57 67 79 2 16 34 43 57 67

43. Example 2 16 43 88 34 57 67 79 2 16 34 43 57 67

44. Example 2 16 43 88 34 57 67 79 2 16 34 43 57 67 79

45. Example 2 16 43 88 34 57 67 79 2 16 34 43 57 67 79

46. Example 2 16 43 88 34 57 67 79 2 16 34 43 57 67 79 88

47. Analysis of Merge() What is the growth rate of Merge()?
Inspect the loops.
The length of each loop is proportional to the length of the lists.
Therefore, the run-time of Merge() is (n), where n is the combined length of the two lists.
In fact, # comparisons  n – 1.

48. Analysis of MergeSort()
Now we can analyze MergeSort().
Let cn be the number of comparisons needed by MergeSort() for a list of length n.
Then
cn  cfloor(n/2) + cceiling(n/2) + (n – 1).
Furthermore, c1 = 0.

49. Analysis of MergeSort()
Assume the worst case:
cn = cfloor(n/2) + cceiling(n/2) + (n – 1).
Show by induction that
cn  n log2 n – n + 1.
The base case is trivial.
So suppose it is true for all positive integers n < k for some k  2.
Show that it is true when n = k.

50. Analysis of MergeSort()
Suppose k is even.
Then
ck = 2ck/2 + (k – 1)
 2(k/2 log2 k/2 – k/2 + 1) + (k – 1)
= (k log2 k/2 – k + 2) + (k – 1)
= k(log2 k – 1) + 1
= k log2 k – k + 1.

51. Analysis of MergeSort()
Suppose k is odd.
Then we need the following lemma.
Lemma: if k > 1, then
(k – 1)log(k – 1) + (k + 1)log(k + 1) > 2k log k.
Proof:
Calculus.

52. Analysis of MergeSort()
Then
ck = c(k – 1)/2 + c(k + 1)/2 + (k – 1)
 [(k – 1)/2 log2 (k – 1)/2 – (k – 1)/2 + 1]
+ [(k + 1)/2 log2 (k + 1)/2 – (k + 1)/2 + 1]
+ k – 1
= [(k – 1)/2 log2 (k – 1)/2]
+ [(k + 1)/2 log2 (k + 1)/2] + 1

53. Analysis of MergeSort()
= (1/2)[(k – 1)(log2 (k – 1) – 1)]
+ (1/2)[(k + 1)(log2 (k + 1) – 1)] + 1
= (1/2)[(k – 1)log2 (k – 1)
+ (k + 1)log2 (k + 1)] – k + 1
> k log2 k – k + 1 (by the lemma).

54. Analysis of MergeSort()
Thus, cn is (n log2 n).
Now we will show that cn is O(n log2 n).
It will follow that cn is (n log2 n).

55. Analysis of MergeSort()
Show by induction that
cn  2n log2 n.
The base case is trivial.
Suppose that it is true for all positive integers n < k for some k  2.
Show that it is true when n = k.

56. Analysis of MergeSort()
Suppose k is even.
Then
ck = 2ck/2 + (k – 1)
 4(k/2) log2 (k/2) + k – 1
= 2k (log2 k – 1) + k – 1
= 2k log2 k – k – 1
< 2k log2 k.

57. Analysis of MergeSort()
Suppose k is odd.
Then
ck = c(k – 1)/2 + c(k + 1)/2 + (k – 1)
 [2(k – 1)/2 log2 (k – 1)/2]
+ [2(k + 1)/2 log2 (k + 1)/2]
+ (k – 1)
= (k – 1)log2 (k – 1)/2
+ (k + 1)log2 (k + 1)/2 + (k – 1)

58. Analysis of MergeSort()
= (k – 1)log2 (k – 1)/2
+ (k + 1)log2 (k + 1)/2 + (k – 1)
= (k – 1)log2 (k – 1) – (k – 1)
+ (k + 1)log2 (k + 1) – (k + 1)
+ (k – 1)
= (k – 1)log2 (k – 1) + (k + 1)log2 (k + 1) – k – 1
< 2k log2 k – k – 1
< 2k log2 k.

59. Analysis of MergeSort()
Therefore, the worst case of the merge sort is (n log2 n).

60. Analysis of MergeSort()
Suppose that MergeSort() sorts a list of 100 numbers in 1 s.
How long will it take to sort a list of 1 billion numbers?
(109 log 109)/(102 log 102)
= 107(9 log 10)/(2 log 10)
= 4.5 x 107 s
= 45 sec.