1. AITI Lecture 22 Sorting Adapted from MIT Course 1.00 Spring 2003 Lecture 32 (Teachers: Please do not erase the above note)

2. Sorting Order Sorting implies that the items to be sorted are ordered. We use the same approach that we employed for binary search trees. That is, we will sort Objects, and whenever we invoke a sort, –we either supply a Comparator to order the elements to be sorted or –the sort routine can assume that the objects to be sorted belong to a class like Integer that implements Comparable and possesses a native ordering.

3. The Sort Interface There are times when we want to sort only part of a collection. We define an interface for sorts with this in mind. A Sort must supply two methods, one to sort an entire array of Objects and one to sort a portion of an array specified by start and end elements (a true closed interval, not half open): public interface Sort { public void sort(Object[] d,int start,int end); public void sort(Object[] d); }

4. Using the Sort Interface This interface makes sorting algorithms become classes. We must create an instance of the class before we can use it to perform a sort. Each sort class will have two constructors. The default constructor will sort elements that have a native order. The other constructor will take a Comparator as the only argument. Here is an example code fragment that creates an instance of InsertionSort to sort an array of Integers : Integer [] iArray = new Integer[ 100 ];... // initialize iArray Sort iSort = new InsertionSort(); iSort.sort( iArray ); Our examples will be simpler than this, though.

5. Insertion Sort Insertion sorting is a formalized version of the way most of us sort cards. In an insertion sort, items are selected one at a time from an unsorted list and inserted into a sorted list. It is a good example of an elementary sort. It runs slowly, which is ok on small datasets but unacceptable on large ones. (See the exercise) To save memory and unnecessary copying we sort the array in place. We do this by using the low end of the array to grow the sorted list against the unsorted upper end of the array.

6. Insertion Sort Diagram 74363 Initially SortedUnsorted 47363 After 1 insertion SortedUnsorted 47363 After 3 insertions SortedUnsorted...

7. Run the InsertionSort Simulation Download InsertionSort.jar, the simulation that you will use to examine the algorithm. Type in a series of numbers each followed by a return. These are the numbers to sort. Click start and then stepInto to single step through the code. reset restarts the current sort. new allows you to specify a new sort.

8. InsertionSort Questions Use the simulator to explore the following questions and let us know when you think you know the answers: –How many elements have to be moved in the inner for loop if you run InsertionSort on an already sorted list? Does it run in O(1), O(n), or O(n 2 ) time? –What order of elements will produce the worst performance? Does this case run in O(1), O(n), or O(n 2 ) time? Why? –Does the average case have more in common with the best case or the worst case?

9. Insertion Sort Code public class InsertionSortMain { private static class Item implements Comparable { public double key; public String value; public Item(double k, String v) { key= k; value= v; } public int compareTo(Object o) { Item other= (Item) o; if (key < other.key ) return -1; else if (key > other.key) return 1; else return 0; } }

10. Insertion Sort Code p.2 public static void main(String[] args) { Item a= new Item(77.0, "Al"); Item b= new Item(9.0, "Bob"); Item c= new Item(22.0, "Cindy"); Item d= new Item(5.0, "Debra"); Item[] names= {a, b, c, d}; insertionSort(names); for (int i=0; i < names.length; i++) System.out.println(names[i].key + names[i].value); } public static void insertionSort(Comparable[] data) { for (int pos= 1; pos < data.length; pos++) { Comparable v= data[pos]; int i= 0; for (i= pos; i>0 && data[i-1].compareTo(v)>0; i--) data[i]= data[i-1]; data[i]= v; } } }

11. Quicksort overview Most efficient general purpose sort, O(n lg n) Basic strategy –Split array (or vector) of data to be sorted into 2 subarrays so that: Everything in first subarray is smaller than a known value Everything in second subarray is larger than that value –Technique is called ‘partitioning’ Known value is called the ‘pivot element’ –Once we’ve partitioned, pivot element will be located in its final position –Then we continue splitting the subarrays into smaller subarrays, until the resulting pieces have only one element (recursion!)

12. Quicksort algorithm 1.Choose an element as pivot. We use right element 2.Start indexes at left and (right-1) elements 3.Move left index until we find an element> pivot 4.Move right index until we find an element < pivot 5.If indexes haven’t crossed, swap values and repeat steps 3 and 4 6.If indexes have crossed, swap pivot and left index values 7.Call quicksort on the subarrays to the left and right of the pivot value

13. Example Quicksort(a, 0, 6) Original45805585507065 i i j jpivot 1 st swap45505585807065 i i j j 2 nd swap45505565807085 quicksort(a,0,2)quicksort(a,4,6) final position

14. Partitioning Partitioning is the key step in quicksort. In our version of quicksort, the pivot is chosen to be the last element of the (sub)array to be sorted. We scan the (sub)array from the left end using index low looking for an element >= the pivot. When we find one we scan from the right end using index high looking for an element <= pivot. If low < high, we swap them and start scanning for another pair of swappable elements. If low >= high, we are done and we swap low with the pivot, which now stands between the two partitions.

15. Recursive Partitioning 84579 6 632 23469687 5 243 5 89667 243 5 97668 current pivot pivot swap 5 partition swap old pivot 5

16. Quicksort Simulation Double click the QuickSort.jar file. It works the same way as the InsertionSort simulation. Use the simulator slowly. If you click too quickly it may lose events and show incorrect results. Sorry. What is the worst case behavior for quicksort? What happens when you quicksort an already- sorted array?

17. A Better Quicksort The choice of pivot is crucial to quicksort's performance. The ideal pivot is the median of the subarray, that is, the middle member of the sorted array. But we can't find the median without sorting first. It turns out that the median of the first, middle and last element of each subarray is a good substitute for the median. It guarantees that each part of the partition will have at least two elements, provided that the array has at least four, but its performance is usually much better. And there are no natural cases that will produce worst case behavior. Try running MedianQuickSort from Median QuickSort.jar. Other improvements: –Convert from recursive to iterative, for efficiency, and always process shortest subarray first (limit stack size, pops, pushes) –When subarray is small enough (5-25 elements) use insertion sort

18. Quicksort main(), swap public class QuicksortMain { public static void main(String[] args) { String input= JOptionPane.showInputDialog("Enter no of elements to sort:"); int size= Integer.parseInt(input); int[] sortdata= new int[size]; sortdata[0]= sortdata[size-1]= 500; for (int i=1; i < size-1; i++) sortdata[i]= (int) (1000*Math.random()); quicksort(sortdata, 0, size-1); System.out.println("Done"); for (int i=0; i < size; i++) System.out.println(sortdata[i]); System.exit(0); } public static void swap(int[] a, int i, int j) { int temp= a[i]; // Swaps a[i] and a[j] a[i]= a[j]; a[j]= temp; }

19. Quicksort, partition public static int partition(int[] a, int left, int right) { int i= left -1; int j= right; int pvalue= a[right]; // Partition element for ( ; ; ) { while ( a[++i] < pvalue) ; // Move index to right while ( a[--j] > pvalue) ; // Move index to left if (i >= j) break; // Indexes cross swap(a, i, j); // Exchange elements } swap(a, i, right); // Exchange pivot, right return i; } public static void quicksort(int[] a, int left, int right) { if (right <= left) return; int part= partition(a, left, right); quicksort(a, left, part-1); quicksort(a, part+1, right); } }

20. Counting sort O(n) sort when keys are small integers Procedure: –Count the number of records with a given key –Calculate the number of records with keys that are less by adding up their counts –This tells you where the final position of your keys will be –Move keys to final positions and you’re done You never compare any value pairs! –This is special case of radix sorting

21. Counting sort example Input Count: 1 st 2 nd Output i a 000ib 1310012 2521123 3232333 4340345 514 size range Move values to output vector or array

22. Steps in the CountingSort Algorithm 1.Copy the numbers to be sorted to a temporary array. 2.Initialize an array indexed by key values (a histogram of keys) to 0. 3.Iterate over the array to be sorted counting the frequency of each key. 4.Now calculate the cumulative histogram for each key value, k. The first element, k=0, is the same as the frequency of key k. The second, k=1, is the sum of the frequency for k=0 and k=1. The third is the sum of the second plus the frequency for k=2. And so on.

23. Steps in the CountingSort Algorithm, 2 5.The first element of the cumulative histogram contains the number of elements of the original array with values <= 0. the second, those <= 1. They lay out blocks of values in the sorted array. 6.Starting with the last element in the original array and working back to the first, look up its key in the cumulative histogram to find its destination in the sorted array. It will be the histogram value – 1. 7.Decrement the entry in the cumulative histogram so the next key is not stored on top of the first.

24. CountingSort 43 range = 8 7645382 frequencies 001221111 012347658 cumulative histogram 001358769 012347658 There are no keys 0, 1 Key 2 occurs 1 time and should occupy Position 0. Key 3 occurs 2 times and should occupy positions 1 and 2. Key 4 occurs 2 times and should occupy positions 3 and 4. Etc. sorted output 233447658 012347658

25. CountingSort Questions Double click the CountingSort.jar file. Note that since CountingSort requires the specification of a range (or two passes over the data), it can’t be implemented using our Sort interface. Experiment with the simulation. Type the numbers to be sorted first. Then enter the range in the range field and press the start button. Use stepinto to trace the code. Does counting sort have a best case or worst case? Does it sort in O(1), O(n), or O(n log n) time? Why?

26. Counting Sort main() public class CountingSort { public static void main(String[] args) { int size= 10; // Size-number of items to sort int range= 100; // Range= number of keys (0-99) int[] a= new int[size+1]; // Input data-key, unsorted String[] s= new String[size+1]; // Input data-value int[] b= new int[size+1]; // Output data-key,sorted String[] t= new String[size+1]; // Output data-value,srt String input= ""; for (int i=0; i < size; i++) { input= JOptionPane.showInputDialog("Enter key:"); a[i]= Integer.parseInt(input); s[i]= JOptionPane.showInputDialog("Enter value:");} countingSort(a, s, b, t, range); for (int i=1; i <= size; i++) System.out.println(b[i] + " " + t[i]); System.exit(0); }

27. Counting sort code public static void countingSort(int[] a, String[] s, int[] b, String[] t, int range) { int size= a.length -1; int[] count= new int[range]; // Count of records w/key for (int i=1; i <= size; i++) // Count keys in bucket count[a[i]]++; for (int j=1; j < range; j++) // Count keys before this count[j] += count[j-1]; // 1st position of this key for (int i=size; i >= 1; i--) { // Move keys to end spot t[count[a[i]]]= s[i]; // Move value b[count[a[i]]--]= a[i]; // Move key }

28. Applications of sorting Business applications are obvious: –Reports sorted by ID, department, etc. Many technical applications use sorting: –Satellite payload –Optimal maintenance policy (e.g., pavement) –Processor scheduling –Generating graphs (networks)

29. Satellite payload ExperimentWeightValueVal/Wgt A242.0 B461.5 C252.5 D351.7 Assume satellite can carry max weight= 7

30. Satellite, p.2 024681012 Weight 20 15 10 5 0 Value x x x x C A D B Max=7 Sort in Value/Weight ratio. Maximum derivative gives optimum value. Gives solution for all maximum weights M. (‘Greedy algorithm’) Last item in solution may be fractional. Often this is acceptable. If not, we use decision trees (‘branch and bound’) as in PS9

31. Processor scheduling You have a set of n jobs to run on a processor (CPU) or machine Each job i has a deadline d i and profit p i There is one processor or machine Each job takes 1 unit of time (simplification) You earn the profit if and only if the job is completed by its deadline –“Profit” can be the priority of the computer job in a real time system that discards tasks that cannot be completed by their deadline

32. Processor scheduling example Number of jobs n=5 Job (i)ProfitDeadlineProfit/Time A1002100 B19119 C27227 D25125 E15315 Solution is {C, A, E} for profit of 142

33. Processor scheduling algorithm Sort jobs by profit/time ratio (slope or derivative) Place each job at latest time that meets its deadline –Nothing is gained by scheduling it earlier, and scheduling it earlier could prevent another more profitable job from being done 0123 Time A (100) C (27) D, B infeasible E (15)