1. Sorting and Runtime Complexity CS255

2. Sorting Different ways to sort: –Bubble –Exchange –Insertion –Merge –Quick –more…

3. Bubble Sort Compare all pairs of adjacent elements from front to back –Swap a pair if out of order This completes 1 bubble pass –Bubbles the biggest element to the end N bubble passes are needed

4. Bubble Sort – bubble pass 31842765 31482765 31428765 38142765 38142765 31427865 31427685 31427658

5. Exchange Sort Compare 1 st element to each other element –Swap a pair if out of order This completes one pass –Places the smallest element in the 1 st position Repeat for the 2 nd, 3 rd, 4 th, etc elements

6. Exchange Sort – single pass 28341765 28341765 18342765 38241765 38241765 18342765 18342765 18342765

7. Insertion Sort Build a new list in sorted order –The previous sorts sorted “in place” Start with an empty new list Pull each element from the original list, one at a time and insert into the new list in the correct position –Compare the element to each in the new list from front to back until we reach the correct position

8. Insertion Sort 38241765 3 8241765 241765 41765 1765 765 65 5 38 238 2348 12348 123478 1234678 12345678

9. Comparison of running times How fast is each of the 3 sorts? We would like our analysis to be independent of the type of machine –So we can’t just use milliseconds –We need something else to count For sorting we will count the number of comparisons

10. Running times So how many comparisons does each sort make? –Does it depend on the number of elements in the Vector? –Does it depend on the specific values in the Vector?

11. Bubble’s running time for (int i=0; i<v.size(); i++) { for (int j=0; j<v.size()-1; j++) { Integer a = (Integer)v.get(j); Integer b = (Integer)v.get(j+1); if (a.compareTo(b) > 0) { v.set(j, b); v.set(j+1, a); }

12. Is this any different? for (int i=0; i<v.size(); i++) { for (int j=0; j<v.size()–i-1; j++) { Integer a = (Integer)v.get(j); Integer b = (Integer)v.get(j+1); if (a.compareTo(b) > 0) { v.set(j, b); v.set(j+1, a); }

13. Insertion Sort running time Vector sortedV = new Vector(); for (int i=0; i<v.size(); i++) { Integer a = (Integer)v.get(i); int j=0; while ((j<v.size()) && (((Integer)v.get(j)).compareTo(a) < 0) { j++; } sortedV.add(j, a); }

14. Merge Sort Break the list into 2 equal pieces Sort each piece Merge the results together

15. 38541762 38541762 34581267 12345678

16. Breaking Apart Vector left = new Vector(); for (int i=0; i<v.size()/2; i++) { left.add(v.get(i)); } Vector right = new Vector(); for (int i=v.size()/2; i<v.size(); i++) { right.add(v.get(i)); }

17. Breaking Apart – a better way Vector left = new Vector(); left.addAll(v.subList(0, v.size()/2)); Vector right = new Vector(); right.addAll(v.subList(v.size()/2, v.size());

18. Merging Only need to compare the first items from each list Put the smaller of the two in the new merged list Repeat until one of the lists is empty –Transfer all the remaining items from the other list into the merged list Write the code in class

19. Sorting Vector sortedLeft = mergeSort(left); Vector sortedRight = mergeSort(right); This is an example of “recursion” – a function defined in terms of itself

20. 38541762 38541762 38175462 38541672

21. Merge sort recursion public Vector mergeSort(Vector v) { Vector sortedVector = null; if (v.size() == 1) { // Base case sortedVector = new Vector(v); } else { // Recursive case // 1. Break apart into left and right // 2. Recurse sortedLeft = mergeSort(left); sortedRight = mergeSort(right); // 3. Merge sortedLeft and sortedRight } return sortedVector; }

22. Quicksort Break the list into 2 pieces based on a pivot –The pivot is usually the first item in the list –All items smaller than the pivot go in the left and all items larger go in the right Sort each piece (recursion again) Combine the results together

23. 38542761 21385476 12345678 12345678

24. Running times In trying to find the running times of mergesort and quicksort we need to determine where the “work” takes place –“Work” is the number of comparisons Where does the work occur for merge? Where does the work occur for quick?

25. MergeSort Running Time The work occurs when the merge occurs How much work is done when merging 2 vectors of length L? 3458 1267

26. 38541762 38541762 38175462 38541672

27. MergeSort Running Time So we have N amount of work at each level If we know how many levels a full merge sort has we can compute the running time as: –O ( N * number of levels ) So how many levels are in a full merge sort?

28. MergeSort Running Time Does this big-O running time change depending on the input or is it always the same (best, worst, and average times are all the same)?

29. QuickSort Running Time Lets try and take the same approach as we did with MergeSort…. The work is done on the split How much work is done to split a vector of length L? 38541762

30. 38542761 21385476 123 45 5476 21 8 76 4576

31. QuickSort Running Time So we have a problem just counting the work per level and the number of levels. First, the amount of work per level varies –Even in this single example Second, the number of levels can vary So, does the amount of work change based on the input?

32. QuickSort Running Time What is the best case? –What is the best case’s running time? What is the worst case? –What is the worst case’s running time?

33. Computational Complexity So, we have seen 5 different sorts –What is the best running time of any sort possible (not just the 5 we have seen)? To answer this question we need to prove that any sort would have to do at least some minimal amount of work (lower bound) –To do so we will use decision trees (ch 7.8)

34. // Sorts an array of 3 items void sortthree(int s[]) { a=s[1]; b=s[2]; c=s[3]; if (a < b) { if (b < c) { S = a,b,c; } else { if (a < c) { S = a,c,b; } else { S = c,a,b; }} } else if (b < c) { if (a < c) { S = b,a,c; } else { S = b,c,a; }} else { S = c,b,a; } a < b b < c a,b,ca < c a,c,bc,a,b a < cc,b,a b,a,cb,c,a

35. Decision Trees A decision tree can be created for every comparison-based sorting algorithm –The following is a decision tree for a 3 element Exchange sort Note that “c < b” means that the Exchange sort compares the array item whose current value is c with the one whose current value is b – not that it compares s[3] to s[2].

36. b < a c < a c < b b < ac < a c,b,ab,c,a a < bc < b b,a,c c,a,b a,c,ba,b,c

37. Decision Trees So what does this tell us… –Note that there are 6 leaves in each of the examples given (each N=3) In general there will be N! leaves in a decision tree corresponding to the N! permutations of the array –The number of comparisons (“work”) is equal to the depth of the tree (from root to leaf) Worst case behavior is the path from the root to the deepest leaf

38. Decision Trees Thus, to get a lower bound on the worst case behavior we need to find the shortest tree possible that can still hold N! leaves –No comparison-based sort could do better A tree of depth d can hold 2 d leaves –So, what is the minimal d where 2 d >= N! Solving for d we get d >= log 2 (N!) –The minimal depth must be at least log 2 (N!)

39. Decision Trees According to Lemma 7.4 (p. 291): log 2 (N!) >= n log 2 (n) – 1.45n Putting that together with the previous result d must be at least as great as (n log 2 (n) – 1.45n) Applying Big-O d must be at least O(n log 2 (n)) No comparison-based sorting algorithm can have a running time better than O(n log 2 (n))

40. Radix Sort Note that the previous theory applies only to comparison-based sorts –That is sorts based on comparison of keys –We know nothing about the keys except that they are ordered However, if we have additional information about the keys we can do even better

41. 239234879878123458416217 239234217879878123458416 123123 123 217217239239234234416416458458879879878878 217234239416458878879

42. Radix Sort Running Time Because there are no comparisons we need to find something else to count as “work” –Moving an element into its correct sub-pile is a good thing to count (think of it as a 10-way comparison) At the top level we need to move N items into their sub-piles At the next level we need to move N items into their sub-piles Etc.

43. Radix Sort Running Time And there are as many levels as digits So the complete run time is: O(N D) Where N is the number of elements And D is the number of digits of each element Note that D is usually small relative to N implying that usually this reduces to O(N) But if D is large (about the size of N) then this can also turn into O(N 2 ) or higher

44. Radix Sort Running Time A radix sort can also be used for sorting Strings since we have buckets of length 26 rather than 10