Review Quick Sort Quick Sort Algorithm Time Complexity Examples Best case Average case Worst case Examples

Quick Sort Example

Merge Sort Merge Sort Merge Sort Algorithm Time Complexity Examples Best case Average case Worst case Examples

Introduction Merging is the process of combining two or more sorted array into a third sorted array Apply divide-and-conquer to sorting problem Divide the array into approximately n/2 sub-arrays of size two and sort the elements in each sub array Merging each sub-array with the adjacent sub-array will get another sorted sub-array Repeat this process until there is only one array remaining of size n Since at any time the two arrays being merged are both sub-arrays of A, lower and upper bounds are required to indicate the sub- arrays being merged

Let A be an array of n number of elements to be sorted A[1], A[2] ...... A[n] Step 1: Divide the array A into approximately n/2 sorted sub-array, i.e., the elements in the (A [1], A [2]), (A [3], A [4]), (A [k], A [k + 1]), (A [n – 1], A [n]) sub-arrays are in sorted order Step 2: Merge each pair of pairs to obtain the following list of sorted sub-array The elements in the sub-array are also in the sorted order (A [1], A [2], A [3], A [4)),...... (A [k – 1], A [k], A [k + 1], A [k + 2]), ...... (A [n – 3], A [n – 2], A [n – 1], A [n]). Step 3: Repeat the step 2 recursively until there is only one sorted array of size n

Example To illustrate the merge sort algorithm consider the following array with 7 elements [42], [33], [23], [74], [44], [67], [49]

cont…

Merging The key to Merge Sort is merging two sorted lists into one, such that if you have two lists X (x1x2…xm) and Y(y1y2…yn) the resulting list is Z(z1z2…zm+n) Example: L1 = { 3 8 9 } L2 = { 1 5 7 } merge(L1, L2) = { 1 3 5 7 8 9 }

Merging (cont.) X: 3 10 23 54 Y: 1 5 25 75 Result:

Merging (cont.) X: 3 10 23 54 Y: 5 25 75 Result: 1

Merging (cont.) X: 10 23 54 Y: 5 25 75 Result: 1 3

Merging (cont.) X: 10 23 54 Y: 25 75 Result: 1 3 5

Merging (cont.) X: 23 54 Y: 25 75 Result: 1 3 5 10

Merging (cont.) X: 54 Y: 25 75 Result: 1 3 5 10 23

Merging (cont.) X: 54 Y: 75 Result: 1 3 5 10 23 25

Merging (cont.) X: Y: 75 Result: 1 3 5 10 23 25 54

Merging (cont.) X: Y: Result: 1 3 5 10 23 25 54 75

Divide And Conquer Merging a two lists of one element each is the same as sorting them. Merge sort divides up an unsorted list until the above condition is met and then sorts the divided parts back together in pairs. Specifically this can be done by recursively dividing the unsorted list in half, merge sorting the right side then the left side and then merging the right and left back together.

Merge Sort Algorithm Given a list L with a length k: If k == 1  the list is sorted Else: Merge Sort the left side (1 thru k/2) Merge Sort the right side (k/2+1 thru k) Merge the right side with the left side

Merge Sort Example 99 6 86 15 58 35 4

Merge Sort Example 99 6 86 15 58 35 4 99 6 86 15 58 35 86 4

Merge Sort Example 99 6 86 15 58 35 4 99 6 86 15 58 35 86 4 99 6 86 15 58 35 86 4

Merge Sort Example 99 6 86 15 58 35 4 99 6 86 15 58 35 86 4 99 6 86 15 58 35 86 4 99 6 86 15 58 35 86 4

Merge Sort Example 99 6 86 15 58 35 4 99 6 86 15 58 35 86 4 99 6 86 15 58 35 86 4 99 6 86 15 58 35 86 4 4

Merge Sort Example 99 6 86 15 58 35 86 4 Merge 4

Merge Sort Example 6 99 15 86 58 35 4 86 99 6 86 15 58 35 86 4 Merge

Merge Sort Example 6 15 86 99 4 35 58 86 6 99 15 86 58 35 4 86 Merge

Merge Sort Example 4 6 15 35 58 86 99 6 15 86 99 4 35 58 86 Merge

Merge Sort Example 4 6 15 35 58 86 99

Implementing Merge Sort There are two basic ways to implement merge sort: In Place: Merging is done with only the input array Pro: Requires only the space needed to hold the array Con: Takes longer to merge because if the next element is in the right side then all of the elements must be moved down. Double Storage: Merging is done with a temporary array of the same size as the input array. Pro: Faster than In Place since the temp array holds the resulting array until both left and right sides are merged into the temp array, then the temp array is appended over the input array. Con: The memory requirement is doubled.

T(N) = 2T(N/2) + N = O(N logN) Merge Sort Analysis The Double Memory Merge Sort runs O (N log N) for all cases, because of its Divide and Conquer approach. T(N) = 2T(N/2) + N = O(N logN)

Cont… There are other variants of Merge Sorts including k-way merge sorting, but the common variant is the Double Memory Merge Sort. Though the running time is O(N logN) and runs much faster than insertion sort and bubble sort, merge sort’s large memory demands makes it not very practical for main memory sorting.

Merging two sorted arrays To merge two sorted arrays into a third (sorted) array, repeatedly compare the two least elements and copy the smaller of the two 7 9 14 18 5 10 12 20 5 7 9 14 10 18 12 20

Merging parts of a single array The procedure really is no different if the two “arrays” are really portions of a single array first part second part 7 9 14 18 5 10 12 20 5 7 9 14 10 18 12 20

Sorting an array If we start with an array in which the left half is sorted and the right half is sorted, we can merge them into a single sorted array We need to copy our temporary array back up 7 9 14 18 5 10 12 20 7 9 14 18 5 10 12 20 This is a sorting technique called mergesort

Recursion How did the left and right halves of our array get sorted? Obviously, by mergesorting them To mergesort an array: Mergesort the left half Mergesort the right half Merge the two sorted halves Each recursive call is with a smaller portion of the array The base case: Mergesort an array of size 1

The actual code from Lafore, p. 287 private void recMergeSort(long[] workspace, int lowerBound, int upperBound) { if (lowerBound == upperBound) return; int mid = (lowerBound + upperBound) / 2; recMergeSort(workspace, lowerBound, mid); recMergeSort(workspace, mid + 1, upperBound); merge(workspace, lowerBound, mid + 1, upperBound); } from Lafore, p. 287

The merge method The merge method is too ugly to show It requires: The index of the first element in the left subarray The index of the last element in the right subarray The index of the first element in the right subarray That is, the dividing line between the two subarrays The index of the next value in the left subarray The index of the next value in the right subarray The index of the destination in the workspace But conceptually, it isn’t difficult!

Analysis of the merge operation The basic operation is: compare two elements, move one of them to the workspace array This takes constant time We do this comparison n times Hence, the whole thing takes O(n) time Now we move the n elements back to the original array Each move takes constant time Hence moving all n elements takes O(n) time Total time: O(n) + O(n) = O(n)

But wait...there’s more So far, we’ve found that it takes O(n) time to merge two sorted halves of an array How did these subarrays get sorted? At the next level down, we had four subarrays, and we merged them pairwise Since merging arrays is linear time, when we cut the array size in half, we cut the time in half But there are two arrays of size n/2 Hence, all merges combined at the next lower level take the same amount of time as a single merge at the upper level

Analysis II So far we have seen that it takes O(n) time to merge two subarrays of size n/2 O(n) time to merge four subarrays of size n/4 into two subarrays of size n/2 O(n) time to merge eight subarrays of size n/8 into four subarrays of size n/4 Etc. How many levels deep do we have to proceed? How many times can we divide an array of size n into two halves? log2n, of course

Analysis III So if our recursion goes log n levels deep... ...and we do O(n) work at each level... ...our total time is: log n * O(n)... ...or in other words, O(n log n) For large arrays, this is much better than Bubblesort, Selection sort, or Insertion sort, all of which are O(n2) Mergesort does, however, require a “workspace” array as large as our original array

MergeSort (Example) - 1

MergeSort (Example) - 2

MergeSort (Example) - 3

MergeSort (Example) - 4

MergeSort (Example) - 5

MergeSort (Example) - 6

MergeSort (Example) - 7

MergeSort (Example) - 8

MergeSort (Example) - 9

MergeSort (Example) - 10

MergeSort (Example) - 11

MergeSort (Example) - 12

MergeSort (Example) - 13

MergeSort (Example) - 14

MergeSort (Example) - 15

MergeSort (Example) - 16

MergeSort (Example) - 17

MergeSort (Example) - 18

MergeSort (Example) - 19

MergeSort (Example) - 20

MergeSort (Example) - 21

MergeSort (Example) - 22

14 23 45 98 6 33 42 67

14 23 45 98 6 33 42 67 Merge

14 23 45 98 6 33 42 67 6 Merge

14 23 45 98 6 33 42 67 6 14 Merge

14 23 45 98 6 33 42 67 6 14 23 Merge

14 23 45 98 6 33 42 67 6 14 23 33 Merge

14 23 45 98 6 33 42 67 6 14 23 33 42 Merge

14 23 45 98 6 33 42 67 6 14 23 33 42 45 Merge

14 23 45 98 6 33 42 67 6 14 23 33 42 45 67 Merge

14 23 45 98 6 33 42 67 6 14 23 33 42 45 67 98 Merge

Merge Sort Example

Summary Merge Sort Merge Sort Algorithm Time Complexity Examples Best case Average case Worst case Examples