1. Winter 2006CISC121 - Prof. McLeod1 Stuff Assn 5 is available and due Friday, 7pm. Winding down! Possible Remaining topics: –Shell and Radix sorts (not on exam) –Storage of numbers in Java – the source and effects of roundoff error (possibly on exam?) We still need to pick a pre-exam tutorial time. (Exam is April 27).

2. Winter 2006CISC121 - Prof. McLeod2 Last Time Two more unrolling examples. Mergesort algorithm and code.

3. Winter 2006CISC121 - Prof. McLeod3 Today Complexity of Mergesort. Quicksort.

4. Winter 2006CISC121 - Prof. McLeod4 Mergesort – “ aMergeSort ” Code Code for sorting arrays: public static void aMergeSort (int[] A) { aMergeSort(A, 0, A.length-1); } // end aMergeSort public static void aMergeSort (int[] A, int first, int last) { if (first < last) { int mid = (first + last) / 2; aMergeSort(A, first, mid); aMergeSort(A, mid + 1, last); aMerge(A, first, last); } // end if } // end aMergeSort recursive

5. Winter 2006CISC121 - Prof. McLeod5 Mergesort – “ aMerge ” Code private static void aMerge (int[] A, int first, int last) { int mid = (first + last) / 2; int i1 = 0, i2 = first, i3 = mid + 1; int[] temp = new int[last - first + 1]; while (i2 <= mid && i3 <= last) { if (A[i2] < A[i3]) { temp[i1] = A[i2]; i2++; } else { temp[i1] = A[i3]; i3++; } i1++; } // end while

6. Winter 2006CISC121 - Prof. McLeod6 Mergesort – “ aMerge ” Code - Cont. while (i2 <= mid) { temp[i1] = A[i2]; i2++; i1++; } // end while while (i3 <= last) { temp[i1] = A[i3]; i3++; i1++; } // end while i1 = 0; i2 = first; while (i2 <= last) { A[i2] = temp[i1]; i1++; i2++; } // end while } // end aMerge

7. Winter 2006CISC121 - Prof. McLeod7 Complexity of Mergesort Consider the aMergeSort code shown above: Suppose that the entire method takes t(n) time, where n is A.length. We want to know the big O notation for t(n) (yes, “we” really do!). There are no loops in aMergeSort, just some constant time operations, the two recursive calls and the call to aMerge.

8. Winter 2006CISC121 - Prof. McLeod8 Complexity of Mergesort - Cont. What is the time function for aMerge ? There is some O(1) stuff and four loops that are O(n): So,

9. Winter 2006CISC121 - Prof. McLeod9 Complexity of Mergesort - Cont. So far, we have not made any mention of the state of the data. Does it make any difference if the data is in reverse order (worst case), random order (average case) or in order already (best case)? Express t(n) in a recursive expression:

10. Winter 2006CISC121 - Prof. McLeod10 Complexity of Mergesort - Cont. Assume that n is a power of 2: (It is easy enough to show that the proof still holds when n is not a power of two - but I’m not going to do that here).

11. Winter 2006CISC121 - Prof. McLeod11 Complexity of Mergesort - Cont. Substitute n/2 for n, to get t(n/2):

12. Winter 2006CISC121 - Prof. McLeod12 Complexity of Mergesort - Cont. Do the next unrolling, which will be n/2 2 : So, after i unrolling’s:

13. Winter 2006CISC121 - Prof. McLeod13 Complexity of Mergesort - Cont. This recursion stops when the anchor case, n  1 is encountered. This will occur when: Substituting this back in the equation on the previous slide:

14. Winter 2006CISC121 - Prof. McLeod14 Complexity of Mergesort - Cont. At the anchor case: Now the equation can be simplified to yield the big O notation, which indicates that t(n) is O(nlog(n)).

15. Winter 2006CISC121 - Prof. McLeod15 Complexity of Mergesort - Cont. This also can be seen just by considering the “tree” constructed by the recursive calls: n n/2 n/4 n/8 And so on…

16. Winter 2006CISC121 - Prof. McLeod16 Complexity of Mergesort - Cont. The dividing stops when the size of the subarray is one, or when n/2 i =1. As before this gives i=log(n), which is the “height” of the “tree”. At every line, n operations must take place to merge the elements, so the “width” of the tree is n. The total number of operations would be n times log(n).

17. Winter 2006CISC121 - Prof. McLeod17 Quicksort Quicksort was invented by C.A.R. Hoare (1961). Used the same reasoning of dividing a large array into smaller arrays that can be then sorted efficiently, as does the Shell sort algorithm. Recursion allows the algorithm to be more easily coded. The algorithm starts by dividing the array into two subarrays based on a key value called the “pivot” or “bound” value. All values less than the pivot go into the lower subarray, and all others go into the upper subarray. Then each of these subarrays is subjected to the same process using a recursive call. This continues until the subarrays are of size 1.

18. Winter 2006CISC121 - Prof. McLeod18 Quicksort – Cont. Pseudocode: Quicksort(array): –If array.length > 1: Choose pivot value. For all elements in array: –If element < pivot then it goes in array1, otherwise it goes in array2. Quicksort(array1). Quicksort(array2).

19. Winter 2006CISC121 - Prof. McLeod19 Quicksort – Cont. How to choose the pivot value? The idea is to get the subarrays to have about the same size. The simplest approach is to choose the first element of the array as the pivot. Better yet, if the array is already sorted to some degree, it will be safer to choose a value from the middle of the array, as is done in the code shown on the next few slides.

20. Winter 2006CISC121 - Prof. McLeod20 Code for “preparation” method: public static void quickSort(int[] A) { int max = 0; for (int i = 1; i < A.length; i++) if (A[i] > A[max]) max = i; swap(A, A.length-1, max); quickSort(A, 0, A.length-2); } // end quickSort(array) Quicksort – Cont.

21. Winter 2006CISC121 - Prof. McLeod21 This method locates the maximum element in A and moves it to the last position in the array. Prevents “lower” (in the next method) from running off the end of the array, in the remote chance that the pivot is chosen as the largest value in the array. (It also calls the typical “ swap ” method.) Quicksort – Cont.

22. Winter 2006CISC121 - Prof. McLeod22 public static void quickSort (int[] A, int first, int last) { int lower = first + 1; int upper = last; swap(A, first, (first+last)/2); int pivot = A[first]; while (lower <= upper) { while (A[lower] < pivot) lower++; while (A[upper] > pivot) upper--; if (lower < upper) swap(A, lower++, upper--); else lower++; } swap(A, upper, first); if (first < upper - 1) quickSort(A, first, upper-1); if (upper + 1 < last) quickSort(A, upper+1, last); } // end quickSort(subarrays)

23. Winter 2006CISC121 - Prof. McLeod23 Example - sorting array: 8 5 4 7 6 1 6 3 8 12 10 After “preparation”: 8 5 4 7 6 1 6 3 8 10 12 After selection of pivot value and initial assignment of lower and upper: 6 5 4 7 8 1 6 3 8 10 12 Upper and lower move in until a swap has to be made: 6 5 4 7 8 1 6 3 8 10 12 Make the swap and move to next swap to be made: 6 5 4 3 8 1 6 7 8 10 12 Max value pivot lowerupper pivot lowerupper pivot lowerupper

24. Winter 2006CISC121 - Prof. McLeod24 Make the swap, and then upper and lower pass each other: 6 5 4 3 6 1 8 7 8 10 12 Put pivot value back in, and divide into two subarrays: 1 5 4 3 6 6 8 7 8 10 12 (Note how old pivot and max value are already in correct positions and do not need to be included in next recursive call). pivot lowerupper

25. Winter 2006CISC121 - Prof. McLeod25 Two recursive calls with subarrays: 4 5 1 3 667 8 8 10 12 After upper and lower have moved through arrays and pivots have been replaced: 13 4 5 667 8 8 10 12 Last set of recursive calls with subarrays: 1 3 4 5 66 7 8 8 10 12 Array is sorted: 1 3 4 5 6 6 7 8 8 10 12 pivot

26. Winter 2006CISC121 - Prof. McLeod26 Quicksort – Cont. Note that new arrays are not created, only the bounds of the subarrays in the array A are changed. The recursive calls will only contain different bounding values. And the subarrays get smaller for each call. The anchor case is when the size of the subarray is one.

27. Winter 2006CISC121 - Prof. McLeod27 Quicksort - Cont. The worst case is when a near-median value is not chosen – the pivot value is always a maximum or a minimum value. Now the algorithm is O(n 2 ). However, if the pivot values are always near the median value of the arrays, the algorithm is O(nlog(n)) – which is the best case. (See the derivation of this complexity for merge sort). The average case also turns out to be O(nlog(n)).

28. Winter 2006CISC121 - Prof. McLeod28 Choice of Pivot Value Several techniques can be used: –Choose first value –Choose middle value –Calculate “pseudo-median”. This is how quicksort is implemented in Arrays.sort(). –Why not calculate the actual median? The technique you use will depend how random the data set is.

29. Winter 2006CISC121 - Prof. McLeod29 Quicksort - Cont. So, the choice of the pivot value can be critical. Knowledge of the nature of the data to be sorted might suggest another algorithm to use in choosing the pivot value. Experiment has shown that Quicksort is faster than any other efficient sort. It has also been suggested that Quicksort can be made even faster by choosing to use a simple sort like insertion sort for subarray sizes less than about 30.

30. Winter 2006CISC121 - Prof. McLeod30 Comparison of Quicksort and Mergesort Mergesort is slower than Quicksort primarily because it does not do any preliminary shuffling of the data when it divides it. It also requires more memory because of the creation of temporary arrays (or lists) to complete the merge operation.

31. Winter 2006CISC121 - Prof. McLeod31 Comparison of Quicksort and Mergesort However, merge sort is very useful when sorting data that does not have all be in memory at once (“in place”). For example, if the data must be saved on disk because it will not all fit in memory, then mergesort will make the minimum number of file reads and writes. If you have to sort lists, it is very easy to code mergesort for them.

32. Winter 2006CISC121 - Prof. McLeod32 Comparison of Sorts Sorting 50,000 random int ’s (average case) on a PIII using Ready To Program. Selection sort is not influenced by the state of the data, where bubble and insertion are. SortTime (msec)# Comparisons# Swaps Insertion11000~623,000,00049,999 Selection18000~1,250,000,00050,000 Bubble40000~623,000,000 simple O(n 2 ) O(n)

33. Winter 2006CISC121 - Prof. McLeod33 Comparison of Sorts – Cont. SortTime (msec)# Comparisons# Swaps Shell50~1,300,000~1,784,000 Quicksort40360,000218,000 Merge (array)90718,100784,464 Merge (lists)1810718,100784,464 Radix (queues)960N/A1,000,000 Arrays.sort40?? Collections.sort (Vectors) 250?? Java Efficient O(n 1.25 ) O(nlog(n)) O(n)