1. 1 CMPSCI 187 Computer Science 187 Introduction to Introduction to Programming with Data Structures Introduction to Introduction to Programming with Data Structures Lecture 22 Sorting I Lecture 22 Sorting I AnnouncementsAnnouncements 1.Two OWL assignments are up - both are due 12/13/2006 at 5 pm a.A sorting OWL b.A final survey 1.Two OWL assignments are up - both are due 12/13/2006 at 5 pm a.A sorting OWL b.A final survey

2. 2 CMPSCI 187 Sorting and Searching l Sorting and searching are probably the two most important high level operations in computer science. l Every time you H look up a phone number H look up a word in a dictionary..........you are searching! l Most searching algorithms depend on the fact that the data -your dictionary- is sorted l Sorting shows up when H you decide to alphabetize your novels H print mailing labels by zip code H print your union local membership by seniority

3. 3 CMPSCI 187 Sorting: An Introduction l Sorting a very rich area - many different ways to sort a set of keys. l Some algorithms are better than others. l A lot is known theoretically about sorting. l For example, if a sorting algorithm is based on key comparisons to perform sorting, it cannot have a complexity better than O(n log n), where n is the number of elements to be sorted. l There are O(n) sorting algorithms. l Some algorithms work better in primary memory, others work better when using disks.

4. 4 CMPSCI 187 A Taxonomy of Sorting Algorithms Set of all sorting algorithms Comparison sorts Address Calculation Sorts Proxmap Sort Radix Sort Transposition Sorts Insertion Sorts Priority Q Sorts Divide & Conquer Diminishing Increment Bubble Sort Insertion Sort Tree Sort Selection Sort Heap Sort Merge Sort Quick Sort Shell Sort Will do Done

5. 5 CMPSCI 187 Using Java API Sorting Methods Java API provides a class Arrays with several overloaded sort methods for different array types Class Collections provides similar sorting methods l Sorting methods for arrays of primitive types: H Based on the Quicksort algorithm Method of sorting for arrays of objects (and List ): H Based on Mergesort l In practice you would tend to use these

6. 6 CMPSCI 187 Framework of Discussion l Searching and sorting a list (array) of characters, strings, or integers, commonly called ‘keys’. l SEARCH: we seek the position (or a position, or first position) of a given ‘key’. l SORTING: we seek to place the list of ‘keys’ in alphabetical order. l Ingredients: Comparison based sorting and swapping

7. 7 CMPSCI 187 Conventions of Presentation Write algorithms for arrays of Comparable objects l For convenience, examples show integers  These would be wrapped as Integer ; or  You can implement separately for int arrays F Or you can use generics l Generally use n for the length of the array F Elements 0 through n-1

8. 8 CMPSCI 187 Introduction to Sorting l Imagine we have an array of integers to sort: int[] arr = {3,14,2,-10,1}; l We can proceed through the array swapping values that are out of order until the array is ordered. l Stop when all elements are ‘in final location’ that is, in order.

9. 9 CMPSCI 187 BubbleSort in Java public void bubbleSort(int[] arr) { int sorted = 0 while (sorted < arr.length) { for (int k=1;k < arr.length-sorted; k++) if (arr[k-1] > arr[k]) swap(arr, k-1, k); sorted++; } Is the final list in ascending or descending order?

10. 10 CMPSCI 187 Complexity of BubbleSort l What is the time O(.) complexity of BubbleSort? F What should we count? l Can you see a way to make bubble sort faster? F Can we change the big-O complexity of BubbleSort?

11. 11 CMPSCI 187 Complexity of BubbleSort sorted = 0 sorted = 1 sorted = 2. sorted=arr.length-1 If n = arr.length, the number of comparisons is is n-1 + n-2 +... + 2 + 1 = n x(n-1)/2 This is O(n 2 )... these are still unsorted....... n x n grid..

12. 12 CMPSCI 187 Sorting Two Key-Comparison Algorithms: Selection Sort find “best” value, exchange it with current position Insertion Sort sorts values seen so far by selecting the next value and inserting it in proper order Put a list of data in sorted order (ascending or descending) perhaps as a prelude to searching.

13. 13 CMPSCI 187 Selection Sort l A relatively easy to understand algorithm l Sorts an array in passes F Each pass selects the next smallest element F At the end of the pass, places it where it belongs l Efficiency is O(n 2 ), hence called a quadratic sort l Performs: F O(n 2 ) comparisons F O(n) exchanges (swaps)

14. 14 CMPSCI 187 35 65 30 60 20 Selection Sort l Basic Algorithm F start at beginning of data F find largest value in data table to right F exchange with current position F increment start F repeat

15. 15 CMPSCI 187 Selection Sort Example 3565306020 scan 0-4, smallest 20 swap 35 and 20 2065306035scan 1-4, smallest 30 swap 65 and 30 2030656035scan 2-4, smallest 35 swap 65 and 35 2030356065scan 3-4, smallest 60 swap 60 and 60 2030356065done

16. 16 CMPSCI 187 Find the ‘Best’ l First we need a method to find the “best” value: F ‘best’ ~ ‘smallest’ int minIndex(int[] vals, int start) { //returns the index of the smallest value in //the vals array starting from ‘start’ //pre: 0 <= start < vals.length //post: ix contains index of minimum int min = vals[start]; int ix = start; for (int cur=start+1;cur < vals.length;i++) { if (vals[cur] < min) { min = vals[cur]; ix = cur;} } return ix; } Easy to modify to maxIndex. Use compareTo to make general.

17. 17 CMPSCI 187 Selection Sort l Using minIndex, Selection Sort is straightforward: public void selectionSort (int[] arr) { int first = 0; int min; while (first < arr.length) { min = minIndex(arr,first); swap(arr, min, first); first=first+1; }

18. 18 CMPSCI 187 first = 0 first = 1 first = 2. first=arr.length-1 If n = arr.length, the number of comparisons is n-1 + n-2 +... + 2 + 1 = n x (n-1)/2 This is O(n 2 ) Complexity of Selection Sort these are still unsorted... n x n grid

19. 19 CMPSCI 187 Java Framework l We’ll assume an Array class which contains the following methods: public static Comparable[] initialize(int size) ……returns an array of size 'size' initialized to random three letter strings over the alphabet 'a'-'k’ or to some other useful set of strings. public static void printArray(String msg,Comparable[] data) …… prints the message string on one line followed by the data array on succeeding lines l Also use the Words class and supporting code

20. 20 CMPSCI 187 Insertion Sort l When you are dealt a hand of cards you use insertion sort to order the cards. l For each item in the list, insertion sort “inserts” it in the “right” place (i.e. in order) l Do this one using ‘comparable’ interface and ‘compareTo’ method

21. 21 CMPSCI 187 Aside: Comparable in Java l Recall the comparable interface: interface Comparable { // if k1 and k2 are Comparable keys, then // k1.compareTo(k2) returns // 0 if k1==k2 // +1 if k1>k2 // -1 if k1<k2 int compareTo(Comparable key); String toString(); //converts this object to a printable string }

22. 22 CMPSCI 187 Insertion Sort l Given a list of six strings: l Think of the list as being divided into two parts: a sorted list and an unsorted list. l In each pass, the first element of the unsorted list is added to the sorted list by inserting it at the appropriate place. rt up it as im is sortedunsorted

23. 23 CMPSCI 187 Insertion Sort sortedunsorted rt up it as im is sortedunsorted rt up it as im is sortedunsorted it rt up as im is sortedunsorted as it rt up im is sortedunsorted as im it rt up is as im is it rt up Original List 1 2 3 4 5: Sorted List

24. 24 CMPSCI 187 Insertion Sort as it rt up im is holder = ‘im’ 1 strider 1. Get the next element from unsorted array (current) 2. Slide everything up the array until a spot is made for ‘current’ element 3. Insert current element at correct location current as xx it rt up is holder = ‘im’ 2 stridercurrent

25. 25 CMPSCI 187 Insertion Sort 1. Get the next element from unsorted array (current) 2. Slide everything up the array until a spot is made for ‘current’ element 3. Insert current element at correct location as xx it rt up is holder = ‘im’ 3 stridercurrent im

26. 26 CMPSCI 187 The insertionSort Method //Sort an array using the insertionSort algorithm public Comparable[] insertionSort() { int current; Comparable holder; int strider; for (current = 1; current < data.length; current++) { holder = data[current]; for (strider=current-1; strider>=0 && (data[strider].compareTo(holder)>0);) { data[strider+1] = data[strider]; strider--;} data[strider+1] = holder; } return data; } as it rt up im is current holder = ‘im’ 1 2 3 1 2 3 1. Get the next element 2. Slide everything up the array until a spot is made for ‘current’ element 3. Insert current element at correct location Data[strider]>holder strider

27. 27 CMPSCI 187 Complexity of Insertion Sort this is an n x n grid these are all sorted current = 1 current = 2 current = 3. current=arr.length-1 If n = arr.length, the number of comparisons is 1 + 2 +... + n-2 + n-1 = n x (n-1) / 2 This is O(n 2 )

28. 28 CMPSCI 187 Comparison of Quadratic Sorts l None good for large arrays!

29. 29 CMPSCI 187 A Choice (of sorts) l We’ve now seen several sorting algorithms (there are more to come) l How do we decide which to use? F All of these O(n 2 ) sorts use linear space. F That’s good on PDAs and other memory limited environments l We’ll see some faster algorithms next.

30. 30 CMPSCI 187 Faster Sorting l Think about sorting exams….. F I’ll give you half the list to sort, I’ll sort the other half and we’ll merge the results… l The first faster sort we’ll look at is MergeSort l MergeSort is a divide and conquer algorithm

31. 31 CMPSCI 187 Divide and Conquer l Divide and Conquer -We’ve seen it before l Can create a very efficient sort: Merge Sort. l Three distinct steps: F Divide: If the input size is too large to deal with in a straightforward manner, divide the data into two or more disjoint subsets. F Recur: Use divide and conquer to (recursively) solve the subproblems associated with the data subsets. F Conquer: Take the solutions to the subproblems and “merge” these solutions into a solution for the original problem.

32. 32 CMPSCI 187 Merge Sort l Algorithm: F Divide: If S has at least two elements (nothing needs to be done if S has zero or one elements), remove all the elements from S and put them into two ‘lists’, S1 and S2, each containing about half of the elements of S. (i.e. S1 contains the first n/2 elements and S2 contains the remaining ~n/2 elements. F Recur: Recursively sort the lists S1 and S2. F Conquer: Put the elements back into S by merging the sorted lists S1 and S2 into a unique sorted sequence. l Complexity of O(n log n)

33. 33 CMPSCI 187 MergeSort Example 852463451731965031 852463451731965031 852463451731965031 852463451731965031 96 divide down Interesting mergesort animation: http://www.cse.iitk.ac.in/users/dsrkg/cs210/applets/sortingII/mergeSort/mergeSort.html

34. 34 CMPSCI 187 MergeSort Example 172431 4550638596 244563851731 5096 248545631731963150 852463451731965031 96 merge up

35. 35 CMPSCI 187 The Context import java.io.*; import java.util.*; public class SearchTest { public static void main(String[ ] args) { Comparable[ ] data; int last, target, index; int arraySize=10; data = Array.initialize(arraySize); Array.printArray("The data array:",data); //testing the mergeSort method MergeSort sorter = new MergeSort(data); System.out.println("Sorting using merge sort......"); Comparable[] sdata=sorter.mergeSort(); Array.printArray("The sorted array:",sdata); }

36. 36 CMPSCI 187 The Class MergeSort public class MergeSort { Comparable[] data; public MergeSort(Comparable[] newData) {data = newData;} public Comparable[] mergeSort() { return mergeSort2(0, data.length); } private Comparable[] mergeSort2(int first, int n) …… private Comparable[] merge(int first, int n1, int n2) ……

37. 37 CMPSCI 187 Java Implementation of MergeSort private Comparable[] mergeSort2(int first, int n) { //n is the number of elements in the array being sorted int n1; // Size of the first half of the array int n2; // Size of the second half of the array if (n > 1) { // Compute sizes of the two halves n1 = n / 2; n2 = n - n1; //recursively sort the two halves of the (sub)array mergeSort2(first, n1); // Sort data[first] through data[first+n1-1] mergeSort2(first + n1, n2); // Sort data[first+n1] to the end // Merge the two sorted halves. merge(first, n1, n2); } return data; } n1 n2 first 8524634517319650 31

38. 38 CMPSCI 187 A Merge Algorithm Definition of the algorithm for merging two sorted sequences into a unique sorted sequence: Comparable[] merge(int first, int n1, int n2) Input: Two halves of an array (on whose elements a total order relation is defined) sorted in nondecreasing order: first through first+n1-1 first+n1 through n2 Ouput: The original data array containing the union of the elements from the two halves sorted in nondecreasing order.

39. 39 CMPSCI 187 Sorted array returned from mergeSort2 Merging Two Lists n1 n2 first 24 6345173150 318596 first+n1 Two sorted subarrays 24 6345 85 173150 3196 Temporary array of size n1+n2 first+n1 first 17 24 31 45 506385 Copy remaining in list to temporary 96

40. 40 CMPSCI 187 Merge Algorithm (Two Sequences) Merging two sequences: 1. Access the first item from both sequences 2. While neither sequence is finished a) Compare the current items of both b) Copy smaller current item to the output c) Access next item from that input sequence 3. Copy any remaining from first sequence to output 4. Copy any remaining from second to output

41. 41 CMPSCI 187 Java Implementation of Merge (with comments) private Comparable[] merge(int first, int n1, int n2) { Comparable [ ] temp = new Comparable[n1+n2]; // Allocate the temporary array int copied = 0; // Number of elements copied from data to temp int copied1 = 0; // Number copied from the first half of data int copied2 = 0; // Number copied from the second half of data int i; // Array index to copy from temp back into data // Merge elements, copying from two halves of data to the temporary array. while ((copied1 < n1) && (copied2 < n2)) { if (data[first + copied1].compareTo(data[first + n1 + copied2])<0) temp[copied++] = data[first + (copied1++)]; else temp[copied++] = data[first + n1 + (copied2++)]; } // Copy any remaining entries in the left and right subarrays. while (copied1 < n1) temp[copied++] = data[first + (copied1++)]; while (copied2 < n2) temp[copied++] = data[first + n1 + (copied2++)]; // Copy from temp back to the data array. for (i = 0; i < n1+n2; i++) data[first + i] = temp[i]; return data; }

42. 42 CMPSCI 187 Java Implementation of Merge (without comments) private Comparable[] merge(int first, int n1, int n2) { Comparable [ ] temp = new Comparable[n1+n2]; int copied = 0; int copied1 = 0; int copied2 = 0; int i; // Merge elements, copying from two halves of data to the temporary array. while ((copied1 < n1) && (copied2 < n2)) { if (data[first + copied1].compareTo(data[first + n1 + copied2])<0) temp[copied++] = data[first + (copied1++)]; else temp[copied++] = data[first + n1 + (copied2++)]; } // Copy any remaining entries in the left and right subarrays. while (copied1 < n1) temp[copied++] = data[first + (copied1++)]; while (copied2 < n2) temp[copied++] = data[first + n1 + (copied2++)]; // Copy from temp back to the data array. for (i = 0; i < n1+n2; i++) data[first + i] = temp[i]; return data; }

43. 43 CMPSCI 187 Analysis of Merge l Two input sequences, total length n elements F Must move each element to the output F Merge time is O(n) l Must store both input and output sequences F An array cannot be merged in place F Additional space needed: O(n)

44. 44 CMPSCI 187 Complexity of MergeSort (until each partition is <=1) this is bounded by log 2 n n At each level, the number of comparisons needed to do all the merging is n-1. This is O(n log n)

45. 45 CMPSCI 187 Merge Sort Analysis l Splitting/copying n elements to subarrays: O(n) l Merging back into original array: O(n) l Recursive calls: 2, each of size n/2 F Their total non-recursive work: O(n) l Next level: 4 calls, each of size n/4 F Non-recursive work again O(n) l Size sequence: n, n/2, n/4,..., 1 F Number of levels = log n F Total work: O(n log n)