1. Data Structures Advanced Sorts Part 1: Mergesort
Phil Tayco
Slide version 1.2
Mar. 19, 2018

2. Advanced Sorts Practical recursion
Now that we have done some exercises to learn recursion, we can look at some practical ways to apply it
Two things to keep in mind when reviewing these sort algorithms:
The base case not only represents when the recursive loop process stops, but also where a significant operation or action can take place
The inductive case that makes the recursive calls is another way of saying to perform the same total action on a smaller set of of information

3. Advanced Sorts Merge Consider 2 arrays that are already sorted
The algorithm to merge them into one sorted array requires comparing the first elements between the arrays and putting the smaller value into the new array
Whichever array element gets selected, the index of it is increased and the process is repeated

4. Advanced Sorts
Example start 1 3 5 7 2 4 6 8 9

5. Advanced Sorts
First merge compares elements at first index of both arrays 1 3 5 7 2 4 6 8 9

6. Advanced Sorts
2nd array has the smaller value so that number is moved into the new array at the first spot. The 2nd array pointer is also increased by 1 1 3 5 7 2 4 6 8 9

7. Advanced Sorts
On the next iteration, the 1st array element is the smaller value between the two and is subsequently moved 3 5 7 2 4 6 8 9 1

8. Advanced Sorts
After a few more iterations, the new array contains more of the elements from the 2 arrays while maintaining the sort 7 8 9 1 2 3 4 5 6

9. Advanced Sorts Merge Eventually, one of the arrays will be empty
At this point, because of the logic of the algorithm, the other array will contain the remaining elements to place into the new array
These remaining elements will already be sorted and will be greater than the last element placed into the new array
This means we simply add those elements in linearly to complete the merge

10. Advanced Sorts
The 7 has just been added into the array, completing the merge of elements from the 1st array 8 9 1 2 3 4 5 6 7

11. Advanced Sorts
All that remains is to linearly add the remaining elements in 2nd array in the remaining spots of the new array 9 1 2 3 4 5 6 7 8

12. Advanced Sorts
When the remaining array is empty, the merge is complete 1 2 3 4 5 6 7 8 9

13. Advanced Sorts Analysis
Using comparisons, the amount of time to fill in the new array is simply O(n)
Notice for disk space, the 2 arrays sorted that are merged require another array allocated that is equal to the size of the arrays combined
Another way of looking at this is to say that the merge process requires doubling the amount of space already allocated for the 2 arrays

14. Advanced Sorts int[] arrayOne = {1, 3, 5, 7};
int[] arrayTwo = {0, 2, 4, 6, 8, 9};
int[] mergedArray = new int[10];
int oneIndex = 0;
int twoIndex = 0;
int mergedIndex = 0;
int oneSize = arrayOne.length;
int twoSize = arrayTwo.length;

15. Advanced Sorts while (oneIndex < oneSize && twoIndex < twoSize)
if (arrayOne[oneIndex] < arrayTwo[twoIndex])
mergedArray[mergedIndex++] = arrayOne[oneIndex++];
else
mergedArray[mergedIndex++] = arrayTwo[twoIndex++];
while (oneIndex < oneSize)
while (twoIndex < twoSize)

16. Advanced Sorts Code Analysis
This example uses fixed values and variables without a function call to demonstrate the merge algorithm
The 3 while loops make up the merge algorithm using these accessible variables. The comparisons in total will logically work out to O(n)
In a general setting as part of a function, we’ll need to manage these variables (pass them in and out effectively) to get the algorithm to work
The key is to recognize that the 2 arrays are already sorted

17. Advanced Sorts Mergesort
Given the merge algorithm discussion, if you have 2 arrays that are part of an overall array of numbers to sort, you can merge them if the 2 arrays are separated and already sorted
Another way of looking at this is to take the overall array in question and split it in half
We then need to sort the left and right sides and then merge them together
Thinking in recursive terms, if the algorithm is to perform Mergesort, then the inductive case is Mergesort the left side, Mergesort the right side and then merge the sides together

18. Advanced Sorts Mergesort
What’s the base case? Remember that this represents when the recursion process stops
In this algorithm, you can’t Mergesort an array that has only 1 element (it’s already “sorted”)
The overall recursive algorithm is then:
If the array has only one element, return
Otherwise:
Mergesort (left side)
Mergesort (right side)
Merge(left and right side of array together)
This is best viewed with an example array that is a power of 2 (can you see why?)

19. Advanced Sorts
Start with Mergesort(full array) 7 1 4 5 8 2 3 6

20. Advanced Sorts Base case not reached. Mergesort (array[0..3]) 7 1 4 58 2 3 6 7 1 4 5

21. Advanced Sorts Base case not reached again. Mergesort (array[0..1]) 74 5 8 2 3 6 7 1 4 5 7 1

22. Advanced Sorts Base case not reached again. Mergesort (array[0]) 7 1 45 8 2 3 6 7 1 4 5 7 1 7

23. Advanced Sorts
Base case reached, return to Mergesort(array[0..1]) and now Mergsort its right side (array[1]) 7 1 4 5 8 2 3 6 7 1 4 5 7 1 1

24. Advanced Sorts
Base case reached again, return to Mergesort(array[0..1]) 7 1 4 5 8 2 3 6 7 1 4 5 7 1

25. Advanced Sorts
For Mergesort([0..1]), the Mergesort calls for its left and right side are complete. Now Merge [0] and [1] 7 1 4 5 8 2 3 6 7 1 4 5 1 7

26. Advanced Sorts
For Mergesort([0..3]), the Mergesort([0..1]) is done. Now call Mergesort for its right side ([2..3]) 7 1 4 5 8 2 3 6 1 7 4 5 4 5

27. Advanced Sorts
Similar to Mergesort([0..1]), Mergesort([2..3]) will hit 2 base cases and merge (notice there is no net change) 7 1 4 5 8 2 3 6 1 7 4 5 4 5

28. Advanced Sorts
We return to Mergesort([0..3]). Its left and right are now mergesorted so we perform a merge 7 1 4 5 8 2 3 6 1 7 4 5

29. Advanced Sorts
This results in a merge that completes the sorting of [0..3] 7 1 4 5 8 2 3 6 1 4 5 7

30. Advanced Sorts
We return to the original Mergesort([0..7]). The left side is done and the process repeats on the right side! 1 4 5 7 8 2 3 6

31. Advanced Sorts
The recursive process is the same resulting sorting [4..7] followed by [4..5] and [6..7] 1 4 5 7 8 2 3 6 2 8 3 6

32. Advanced Sorts
[4..5] and [6..7] merge and return to [0..7] leaving only one step left to perform at the [0..7] level… 1 4 5 7 2 3 6 8

33. Advanced Sorts
The last step in Mergesort[0..7] is a merge! The array is now sorted!
Now, let’s take a look at the code… 1 2 3 4 5 6 7 8

34. Advanced Sorts public class AdvancedSortingSupplement {
public static int[] numbers;
public final static int MAX_NUMBERS = 15;
public static Random gen;
public static void mergeSort()
int[] temp = new int[numbers.length];
recursiveMergeSort(temp, 0, numbers.length - 1); }

35. Advanced Sorts private static void recursiveMergeSort(int[] temp,
int low, int hi) {
if (low == hi)
return;
int mid = (low + hi) / 2;
recursiveMergeSort(temp, low, mid);
recursiveMergeSort(temp, mid+1, hi);
merge(temp, low, mid+1, hi); }

36. Advanced Sorts private static void merge(int[] temp, int low,
int mid, int high) {
int j = 0;
int loBound = low;
int midBound = mid - 1;
int n = high - loBound + 1;
while (low <= midBound && mid <= high)
if (numbers[low] < numbers[mid])
temp[j++] = numbers[low++];
else
temp[j++] = numbers[mid++];

37. Advanced Sorts while (low <= midBound) temp[j++] = numbers[low++];
while (mid <= high)
temp[j++] = numbers[mid++];
for (int c = 0; c < n; c++)
numbers[loBound+c] = temp[c]; }

38. Advanced Sorts Mergesort analysis
This code contains a global array called “numbers” representing the actual array storing the necessary data
When mergeSort is called, it sorts numbers using the merge sort algorithm
The mergeSort method itself creates a temporary array the size of the number of elements numbers has
The recursiveMergeSort is then called passing in the temp array and numbers’ lower and upper bounds

39. Advanced Sorts Mergesort analysis
The recursiveMergeSort function uses the temporary array as a parameter while applying the merge sort recursive algorithm
When the merge occurs, the same merge algorithm previously discussed merging the appropriate sub arrays within the temp array
When the appropriate merge of temp array is applied, those contents are copied back into numbers – this is where the extra memory space for the merge sort is required and utilized
When all merges are done, the temp array will no longer be used and the memory is freed

40. Advanced Sorts Example of code walkthrough with 4 elements numbers 4 13 2
main()
mergeSort();

41. recursiveMergeSort(0..3);
Advanced Sorts
main calls mergeSort to start the process. Temp space is created
numbers 4 1 3 2
mergeSort()
recursiveMergeSort(0..3);
main()
mergeSort();
temp

42. Advanced Sorts The first recursive call occurs with mergeSort(0..1)
numbers 4 1 3 2
recursiveMergeSort(0..3)
recursiveMergeSort(0..1);
recursiveMergeSort(2..3);
mergeSort()
recursiveMergeSort(0..3);
main()
mergeSort();
temp

43. Advanced Sorts (0..1) will make recursive calls as well
recursiveMergeSort(0..1)
recursiveMergeSort(0);
recursiveMergeSort(1);
numbers 4 1 3 2
recursiveMergeSort(0..3)
recursiveMergeSort(0..1);
recursiveMergeSort(2..3);
mergeSort()
recursiveMergeSort(0..3);
main()
mergeSort();
temp

44. Advanced Sorts The call to merge will merge [0] and [1] within temp
recursiveMergeSort(0..1)
merge(0, 1, 1);
numbers 4 1 3 2
recursiveMergeSort(0..3)
recursiveMergeSort(0..1);
recursiveMergeSort(2..3);
mergeSort()
recursiveMergeSort(0..3);
main()
mergeSort();
temp 1 4

45. Advanced Sorts Last step of merge copies temp data to theArray
recursiveMergeSort(0..1)
merge(0, 1, 1);
numbers 1 4 3 2
recursiveMergeSort(0..3)
recursiveMergeSort(0..1);
recursiveMergeSort(2..3);
mergeSort()
recursiveMergeSort(0..3);
main()
mergeSort();
temp 1 4

46. Advanced Sorts
After merge, (0..1) is done. We return to (0..3) where (2..3) is now called
numbers 1 4 3 2
recursiveMergeSort(0..3)
recursiveMergeSort(0..1);
recursiveMergeSort(2..3);
mergeSort()
recursiveMergeSort(0..3);
main()
mergeSort();
temp 1 4

47. Advanced Sorts (2..3) recursively calls (2) and (3)
which are base cases
Notice temp data from (0..1) is
still present…
recursiveMergeSort(2..3)
recursiveMergeSort(2);
recursiveMergeSort(3);
numbers 1 4 3 2
recursiveMergeSort(0..3)
recursiveMergeSort(0..1);
recursiveMergeSort(2..3);
mergeSort()
recursiveMergeSort(0..3);
main()
mergeSort();
temp 1 4

48. Advanced Sorts 3rd step of (2..3) calls merge and
will merge 3 and 2 in temp
Notice the 2 and 3 replace the
1 and 4 from the previous
merge
recursiveMergeSort(2..3)
merge(2, 3, 3);
numbers 1 4 3 2
recursiveMergeSort(0..3)
recursiveMergeSort(0..1);
recursiveMergeSort(2..3);
mergeSort()
recursiveMergeSort(0..3);
main()
mergeSort();
temp 2 3

49. Advanced Sorts
Temp’s [2..3] data is copied back to theArray. (2..3) is done. Back in (0..3), now we merge
numbers 1 4 2 3
recursiveMergeSort(0..3)
merge(0, 2, 3);
mergeSort()
recursiveMergeSort(0..3);
main()
mergeSort();
temp 2 3

50. Advanced Sorts
Merge of 0..3 in temp occurs by merging theArray[0..1] and theArray[2..3]
numbers 1 4 2 3
recursiveMergeSort(0..3)
merge(0, 2, 3);
mergeSort()
recursiveMergeSort(0..3);
main()
mergeSort();
temp 1 2 3 4

51. recursiveMergeSort(0..3);
Advanced Sorts
Data from temp copied back to theArray and completes the merge of [0..3]
numbers 1 2 3 4
mergeSort()
recursiveMergeSort(0..3);
main()
mergeSort();
temp 1 2 3 4

52. Advanced Sorts
mergeSort function is now complete removing the need for temp and theArray is now sorted
numbers 1 2 3 4
main()
mergeSort();

53. Advanced Sorts Efficiency
The approach is using the divide and conquer method where we repeatedly split the array in half like we did with binary search
Binary search performs at O(log n) implying that there is an O(log n) performance occurring too
There are 2 parts where comparisons occur:
Recursive Merge Sort checking for base case
The merge function

54. Advanced Sorts Efficiency
In the base case check, there is only one comparison. This will always occur and is a direct function of the size of the array
N = 4: 7 comparisons
N = 8: 15 comparisons
N = 16: 31 comparisons
As N increases, number of comparisons increase linearly (2n – 1 to be exact)
Number of comparisons just for the base case check is O(n)

55. Advanced Sorts Efficiency
In the merge, there are a series of loops occurring:
Checking for sub array elements into temp (3 per iteration)
Checking for copying remaining uncopied elements into temp (1 for completed array and at least 1 for incomplete array copying into temp)
Copying temp elements back to theArray – this is a linear O(n) operation
Running the numbers (worst case):
N = 4: 31 comparisons
N = 8: 83 comparisons
N = 16: 203 comparisons

56. Advanced Sorts Efficiency This pattern is a little tougher to see
If we combine the base case check numbers with the merge numbers
N = 4: = 38
N = 8: = 98
N = 16: = 234
Is this an improvement over the simple sorts that perform at O(n2) in the worst case? For those numbers:
N = 4: 16
N = 8: 64
N = 16: 256

57. Advanced Sorts Efficiency
At first, mergesort comparisons are higher than the simple sorts
However, notice the numbers start getting closer together as N doubles in size
Remember that Big-O is a measure of the performance as the size of N increases
The numbers show mergesort eventually starts to outperform the simple sorts
N = 32: 538 vs 1024
N = 64: 1210 vs 4096
As N increases, O(n2) clearly loses to mergesort

58. Advanced Sorts Efficiency
Mergesort, though, is not O(n) as its numbers are growing more than just linearly. If we tried to apply a linear formula:
N = 4: 38 comparisons = 9n + 2
N = 8: 98 comparisons = 12n + 4
N = 16: 234 comparisons = 14n + 10
N = 32: 538 comparisons = 16n + 26
So the performance does not go below linear and our efficiency is somewhere in between O(n) and O(n2)

59. Advanced Sorts Efficiency
Whatever the increase from O(n) is, it will not be additive as O(n) + C is still considered O(n)
The increase factor would have to be multiplied with n but be considered less than n as well (if it was n * n, you would have n2)
What order is in between a fixed number (constant) and a factor less than a multiple of a variable (linear)?
This implies a logarithmic factor! O(n log n)
Thus, we have a 5th category of performance between linear and exponential

60. Advanced Sorts Efficiency
The exact formula is a mathematician’s fun exercise to find. Does O(n log n) make sense?
Another way of looking at n log n is that some set amount of operations is performing at log n, n times (or vice versa)
The number of comparisons in the base case checks perform linearly, but there is something more at work here
If the base case is reached, there are no more recursive function calls. The question is then, how many times does the recursion take place?
Take a look at how the divide and conquer method is seen like a “tree” structure

61. Advanced Sorts Here’s the overall view of mergesort with 8
There are 4 “levels” of the mergesort process
Each level contains a complete set of N elements divided according to their level 7 1 4 5 8 2 3 6 7 1 4 5 8 2 3 6 7 1 4 5 8 2 3 6 7 1 4 5 8 2 3 6

62. Advanced Sorts Base cases are reached in red
The base case level is the lowest one making the recursive cases the remaining top levels 7 1 4 5 8 2 3 6 7 1 4 5 8 2 3 6 7 1 4 5 8 2 3 6 7 1 4 5 8 2 3 6

63. Advanced Sorts Recursive cases are in blue
3 levels of recursive cases for N = 8 showing a logarithmic relationship of log2 N - 1 7 1 4 5 8 2 3 6 7 1 4 5 8 2 3 6 7 1 4 5 8 2 3 6 7 1 4 5 8 2 3 6

64. Advanced Sorts
N Log N
The log n relationship of recursive levels in the process is key
With each recursive case, 2 function recursive calls are made, but more important is that a merge function call is made
Looking at the merge code, there are 3 loops acting sequentially. They are not nested so the order of these operations is linear (around O(3n) to be more accurate)
The n with each merge is more significant with noting that at each recursive level, the total amount of elements affected on that row is the original n

65. Advanced Sorts Put it all together
While the amount of total n per recursive level varies, it varies linearly. That is, with each set of merge calls at every level, it is roughly around a total of some multiplicative factor times n
This makes each recursive level an O(xn) set of comparison operations when totaling the merge function calls where x is some value
As a category, this makes each recursive level O(n) because of the comparisons in the merge function calls
Each level occurs O(log n) times. Put it together and as a category, mergesort is O(n log n)

66. Advanced Sorts Summary
Mergesort compared to the simple sorts in the worst cases is a significant improvement
The only downside to mergesort that the simple sorts have is the use of temp space (Simple sorts operate in the space of n while mergesort requires a second n of space)
It would be nice if we could keep the n log n efficiency without the use of extra space. Guess where we will head next…

67. Advanced Sorts Summary: Comparisons Notes Bubble O(n2)
Swaps on average greater than Selection
Selection
Swaps are O(n)
Insertion
O(n2) (worst case)
Range is O(n) to (n2) Excellent for partially sorted lists
Merge
O(n log n)
Requires 2x memory space because of temp array