1. 1 CSCD 326 Data Structures I Algorithm Analysis

2. 2 Algorithms and Program Design Is it enough if an algorithm implementation "just works" ? A working algorithm that is poorly designed could execute slowly and thus make one part of a software system sluggish and unresponsive to users. Thus, a central part of computer science is using tools to compare the execution speed of algorithms. This is usually done before an algorithm is implemented in a programming language.

3. 3 Algorithm Analysis Mathematical tools and techniques that allow comparison of different solutions to the same problem. These techniques analyze algorithms independently of specific implementations, computers, or data. These techniques do not provide fine details about the speed of a solution but allow determination of whether one solution is much (order of magnitude) slower or faster than another.

4. 4 The Growth Rate Function The simplest way to analyze an algorithm is to count the operations it performs. This is usually not practical since the number of operations depends on the number of data elements to be handled. So - derive a function of n (the data set size) that describes the number of operations as a function of n. This function f(n) is called the growth rate function of the algorithm.

5. 5 Finding f(n) for(i = 0 ; i < n ; i++) for(j = 0 ; j < n ; j++) { System.out.println("i: "+i); System.out.println("j: "+j); } Involves analysisof the number of loop iterations The outer loop executes: one setup statement -- i = 0 once. one comparison on each of n iterations one auto-increment n times one final comparison to end the loop one for statement (the inner for) n times what loop does. so f(n) = 2n + n*inner + 2

6. 6 Finding f(n) for(i = 0 ; i < n ; i++) for(j = 0 ; j < n ; j++) { System.out.println("i: "+i); System.out.println("j: "+j); } The inner loop executes: one setup statement -- j = 0 once. one comparison on each of n iterations one auto-increment n times one final comparison to end the loop two output statements n times. so f(n) = 4n + 2

7. 7 Finding f(n) for(i = 0 ; i < n ; i++) for(j = 0 ; j < n ; j++) { System.out.println("i: "+i); System.out.println("j: "+j); } But the inner loop is done once on each iteration of the outer loop so the final f(n) is: f(n) = 2n +n*(4n + 2) + 2 f(n) = 4n 2 + 4n + 2

8. 8 Big - O Notation Full definition of Big-O notation: Let f and g be functions on the non-negative integers. Then f(n) is of order at most g(n) : denoted f(n) = O(g(n)) This means that there exists a positive constant C such that: |f(n)| <= C(g(n)) for all sufficiently large positive integers n.

9. 9 Big - O Notation What it really means: for f(n) = 4n 2 + 4n + 2 - g(n) is the dominant term of the polynomial - n 2 in this case and thus, 4n 2 + 4n + 2 = 1 and C = 10. Note that the definition of Big-O notation only holds true for sufficiently large values of n. So an algorithm whose growth rate function is: f(n) = 4n 2 + 4n + 2 is said to have time complexity O(n 2 ). Note that Big-O notation provides only a rough categorization of algorithm speed.

10. 10 Comparison of Algorithms with Different Time Complexities Execution times assume: 1  sec execution time per instruction Exactly f(n) instructions will be carried out. n = 10 4 = 10000 log 2 n Execution Time f(n) n n log 2 n n2n2 n3n3 2n2n 1.32877 * 10 -5 sec 0.1 sec 0.13 sec 100 sec — 10 8 µsec 10 6 sec — 11 days 13:46:40 >10 2992 years

11. 11 Graph of Growth Rate Functions

12. 12 Algorithm Time Complexities Complexity Name Big-O O(log n) O(n) O(n log n) O(n 2 ) O(n 3 ) O(2 n ) Logarithmic Linear Log-Linear Quadratic Cubic Exponential O(1) Constant Example Swap Binary Search Linear Search The “Good” Sort Selection Sort Matrix Multiplication Towers of Hanoi

13. 13 Algorithms and Analysis In the following section we will re-examine some basic searching and sorting algorithms and analyze them.

14. 14 Search and Sort Testing Program public class SortSearchTest { public static void main(String[] args) { Integer[] values = new Integer[10]; values[0] = new Integer(15); values[1] = new Integer(9); values[2] = new Integer(13); values[3] = new Integer(20); values[4] = new Integer(5); values[5] = new Integer(0); values[6] = new Integer(7); values[7] = new Integer(10); values[8] = new Integer(3); values[9] = new Integer(2); printem(values); System.out.println("2 found at: "+ linearSearch(values, new Integer(2))); bubbleSort(values, values.length); printem(values); } public static void printem(Comparable[] anArray){ for(int i = 0; i < anArray.length; i++) { System.out.println("array["+i+"]: "+anArray[i]); }

15. 15 Linear (Sequential) Search public static int linearSearch(Comparable[ ] anArray, Comparable target) { int index = 0; while(index < anArray.length) { if (target.compareTo(anArray[index]) == 0 ) return index; index++; } return -1; } The algorithm basically steps through the array until it finds its target element (using target.compareTo()) or it steps through the entire array - returns the index of target or -1 if target is not found

16. 16 Analysis of Linear Search Worst Case: target is not found f(n) = 3n +1 +2 so the algorithm is O(n) Average case: harder to analyze target will be found, on average, after n/2 loop iterations so f(n) = 3(n/2) + 3 and the algorithm is still O(n) Best case: target is the first item in the array only one loop iteration so algorithm is O(1)

17. 17 Binary Search public static int binarySearch(Comparable[ ] anArray, Comparable target) { int index, cmp; int left = 0, right = anArray.length - 1; while( left <= right ) { index = (left + right) / 2; cmp = target.compareTo(anArray[index]; if( cmp < 0) right = index - 1; if( cmp > 0) left = index + 1; else return index; } return -1; // target not found }

18. 18 Binary Search (2) Binary Search depends on two things: 1) Random access memory (an array) is used so we can go directly to the center of the array. 2) The array must be sorted.

19. 19 Binary Search Trace 02 3 5 7 91013 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] Pass 1 -left = 0 right = 9 index = 4 target = 3 0 2 3 5 791013 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] Pass 2 -left = 0 right = 3 index = 1 02 3 579 10 13 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] Pass 3 -left = 2 right = 3 index = 2 return 2 (successful search)

20. 20 Binary Search Trace 2 0 2 35 7 9 1013 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] Pass 1 -left = 0 right = 9 index = 4 target = 1 0235 7 9 10 13 1520 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] Pass 2 -left = 0 right = 3 index = 1 0 2 3 5 7 9 1013 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] Pass 3 -left = 0 right = 0 index = 0 Pass 4 -left = 1 right = 0 return -1 -- unsuccessful search

21. 21 Binary Search Analysis Worst case - target is not found while( left <= right ) { index = (left + right) / 2; On each loop iteration the array is (ideally) divided in half. To determine f(n) consider number of items left in the array on each iteration: 321684201 Pass 1Pass 2Pass 3Pass 4Pass 5Pass 6Pass 7

22. 22 Binary Search Analysis 321684201 Pass 1Pass 2Pass 3Pass 4Pass 5Pass 6Pass 7 n Number of compares 32 16 8 4 2 1 7 6 5 4 3 2 log 2 n 5 4 3 2 1...

23. 23 Binary Search Analysis The relation between n and number of comparisons for n = 2 k is: f(n) = log 2 n + 2 Or for non-integer powers of 2: f(n) = log 2 n + 2 And the time complexity is: O(log 2 n)

24. 24 Bubblesort public static void bubbleSort(Comparable[] theArray, int n) { // --------------------------------------------------- // Sorts the items in an array into ascending order. // Precondition: theArray is an array of n items. // Postcondition: theArray is sorted into ascending order. //NOTE: n specifies size and is only needed if you aren’t sorting whole array // --------------------------------------------------- for (int pass = 1; pass < n; ++pass) { // Invariant: theArray[n+1-pass..n-1] is sorted // and > theArray[0..n-pass] for (int index = 0; index < n-pass; ++index) { // Invariant: theArray[0..index-1] <= theArray[index] int nextIndex = index + 1; if (theArray[index].compareTo(theArray[nextIndex])>0){ // exchange items Comparable temp = theArray[index]; theArray[index] = theArray[nextIndex]; theArray[nextIndex] = temp; } // end if } // end for // Assertion: theArray[0..n-pass-1] < theArray[n-pass] } // end for } // end bubbleSort

25. 25 Bubblesort Trace 15 9 13 20 5 0 7 10 3 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] Pass = 1 After index = 0 9 15 13 20 5 0 7 10 3 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] After index = 1 9 13 15 20 5 0 7 10 32 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] After index = 2 9 13 15 20 50 7 10 3 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] After index = 3 9 13 15 5 20 07 10 3 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] After index = 4 9 13 15 5 0 20 71032 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

26. 26 Bubblesort Trace (2) 9 13 15 5 0 20 7 10 3 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] Pass = 1 After index = 5 9 13 15 5 0 7 20 10 3 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] After index = 6 9 13 15 5 0 7 10 20 32 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] After index = 7 9 13 15 5 07 10 3 20 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] After index = 8 9 13 15 5 0 710 3 2 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

27. 27 Bubblesort Trace (3) After Pass = 1 9 13 15 5 0 7 10 3 2 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] After Pass = 2 9 13 5 0 7 10 3 2 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] After Pass = 3 9 5 0 7 10 3 2 13 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] After Pass = 4 5 0 7 9 3 2 10 13 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] After Pass = 5 0 5 7 3 2 9 10 13 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] After Pass = 6 0 5 3 2 7 9 10 13 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

28. 28 Bubblesort Trace (4) After Pass = 7 After Pass = 8 After Pass = 9 0 3 2 5 7 9 10 13 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] 0 2 3 5 7 910 13 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] 0 2 3 5 7 91013 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

29. 29 Bubblesort Analysis Basic loop structure: for ( int pass = 1 ; pass < n ; ++pass )... for ( int index = 0 ; index < n-pass ; ++index )... The outer loop iterates n - 1 times. Inner loop iterations depends on which iteration of the outer loop. So to derive the growth rate function f(n) we must look at each outer loop iteration.

30. 30 Bubblesort Analysis (2) Make a table of inner loop iterations for each outer loop iteration: Value of pass Number of inner loop iterations 1 2 3 4... n -1 n -2 n -3 n -4... 1

31. 31 BubbleSort : Analysis (3) Now sum the column containing the number of inner loop iterations: (n -1) + (n -2) + (n -3) +... + 1 = n (n -1) / 2 And thus the growth rate function f(n) is: f(n) = n 2 / 2 - n / 2 and the dominant term is n 2 So BubbleSort has time complexity O (n 2 ) Are there any special cases (best, worst, average)?

32. 32 Improved Bubblesort public static void bubbleSort(Comparable[] theArray, int n) { // Sorts the items in an array into ascending order. // Precondition: theArray is an array of n items. // Postcondition: theArray is sorted into ascending order. boolean sorted = false; // false when swaps occur for (int pass = 1;!sorted; ++pass) { // Invariant: theArray[n+1-pass..n-1] is sorted and > theArray[0..n-pass] sorted = true; // assume sorted for (int index = 0; index < n-pass; ++index) { // Invariant: theArray[0..index-1] <= theArray[index] int nextIndex = index + 1; //avoid multiple addition operations if (theArray[index].compareTo(theArray[nextIndex]) > 0) { Comparable temp = theArray[index]; theArray[index] = theArray[nextIndex]; theArray[nextIndex] = temp; sorted = false; // signal exchange } // end if } // end for // Assertion: theArray[0..n-pass-1] < theArray[n-pass] } // end for } // end bubbleSort

33. 33 Improved Bubblesort (2) The boolean variable (sorted) will stay true if no swaps are made while iterating the inner loop. If this ever happens the sort is complete and there is no need to iterate further. Consider the following array: Here the 20 will move to the bottom of the array on the first pass in the outer loop and after the second pass the outer loop exits. 20 0 2 3 5 7910 13 15 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

34. 34 Improved Bubblesort Analysis Does this change the O(n 2 ) time complexity? No, but it does force a case by case analysis Worst case: The array is in reverse order as in the following: 20 15 13 10 9 7 5 3 2 0 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

35. 35 Improved Bubblesort Analysis Worst case: Here each value is bubbled to the bottom of the array and the full n 2 / 2 - n / 2 iterations must be done. Time complexity is O(n 2 ) 20 15 13 10 9 7 5 3 2 0 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

36. 36 Improved Bubblesort Analysis Best case: Values are already sorted. Here the outer loop will iterate only once, since no swaps will be done sorted will never get set false and the outer loop will exit. Time complexity for this case is O(n) 0 2 3 5 7 9 10 13 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

37. 37 Improved Bubblesort Analysis Average case: Values are random. Outer loop iteratations depend on the specific data and although it may exit early it is just as likely not to. Time complexity for this case is still O(n 2 ) 15 9 13 20 5 0 7 10 3 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]

38. 38 Selection Sort public static void selectionSort(Comparable[ ] theArray, int n) { // Sorts the items in an array into ascending order. // Precondition: theArray is an array of n items. // Postcondition: theArray is sorted into ascending order. // Calls: indexOfLargest. // --------------------------------------------------- // last = index of the last item in the subarray of items yet to be sorted // largest = index of the largest item found for (int last = n-1; last >= 1; last--) { // Invariant: theArray[last+1..n-1] is sorted and > theArray[0..last] // select largest item in theArray[0..last] int largest = indexOfLargest(theArray, last+1); // swap largest item theArray[largest] with theArray[last] Comparable temp = theArray[largest]; theArray[largest] = theArray[last]; theArray[last] = temp; } // end for } // end selectionSort

39. 39 Selection Sort : indexOfLargest private static int indexOfLargest(Comparable[ ] theArray, int size) { // --------------------------------------------------- // Finds the largest item in an array. // Precondition: theArray is an array of size items; size >= 1. // Postcondition: Returns the index of the largest item in the array. // -------------------------------------------------- int indexSoFar = 0; // index of largest item found so far // Invariant: theArray[indexSoFar]>=theArray[0..currIndex-1] for (int currIndex = 1; currIndex < size; ++currIndex) { if (theArray[currIndex].compareTo( theArray[indexSoFar] ) > 0) { indexSoFar = currIndex; } // end if } // end for return indexSoFar; // index of largest item } // end indexOfLargest --NOTE: some versions of Selection Sort include this code as part of the Selection Sort method itself

40. 40 Selection Sort : Trace 15 913 20 50 7103 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] last = 9 largest = 3 15 9 13 2 5 07 10 3 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] last = 8 largest = 0 3 9 13 2 5 0 7 10 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] last = 7 largest = 2 3 9 10 2 50 7 13 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] last = 6 largest = 2 3 9 7 2 5 0 10 13 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] last = 5 largest = 1 3 0 7 2 5 9 10 13 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] last = 4 largest = 2

41. 41 Selection Sort : Trace 3 0 5 2 7 9 10 13 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] last = 3 largest = 2 3 0 2 5 7 9 10 13 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] last = 2 largest = 0 2 0 3 5 7 9 10 13 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] last = 1 largest = 0 0 2 3 5 7 9 10 13 15 20 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] outer loop exits

42. 42 Selection Sort : Analysis The outer loop: for (int last = n-1; last >= 1; last--) Iterates n -1 times and each iteration calls indexOfLargest: int largest = indexOfLargest(theArray, last+1); The inner loop (in indexOfLargest): Number of iterations depends on which outer loop call (last + 1 is passed into size : for (int currIndex = 1; currIndex < size; ++currIndex)

43. 43 Selection Sort : Analysis (2) Make a table of inner loop iterations for each outer loop iteration: Value of last Number of inner loop iterations n -1 n -2 n -3 n -4... 1 n -1 n -2 n -3 n -4... 1

44. 44 Selection Sort : Analysis (3) Now sum the column containing the number of inner loop iterations: (n -1) + (n -2) + (n -3) +... + 1 = n (n -1) / 2 And thus the growth rate function f(n) is: f(n) = n 2 / 2 - n / 2 and the dominant term is n 2 So Selection Sort has time complexity O (n 2 ) Are there any special cases (best, worst, average)?

45. 45 Insertion Sort public static void insertionSort(Comparable[] theArray, int n) { // --------------------------------------------------- // Sorts the items in an array into ascending order. // Precondition: theArray is an array of n items. // Postcondition: theArray is sorted into ascending order. // --------------------------------------------------- // unsorted = first index of the unsorted region, // loc = index of insertion in the sorted region, // nextItem = next item in the unsorted region for (int unsorted = 1; unsorted < n; ++unsorted) { // Invariant: theArray[0..unsorted-1] is sorted Comparable nextItem = theArray[unsorted]; int loc = unsorted; while ((loc > 0) && (theArray[loc-1].compareTo(nextItem) > 0)) { // shift theArray[loc-1] to the right theArray[loc] = theArray[loc-1]; loc--; //correction from book’s code (both printed and electronic) } // end while // insert nextItem into sorted region theArray[loc] = nextItem; } // end for } // end insertionSort

46. 46 Insertion Sort : Trace 15 9 13 20 5 0 7103 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] unsorted = 1 nextitem = 9 loc = 0 9 15 13 20 507103 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] unsorted = 2 nextitem = 13 loc = 1 9 13 15 20 5 0 7 10 3 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] unsorted = 3 nextitem = 20 loc = 3 9 13 15 20 5 0 7 10 3 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] unsorted = 4 nextitem = 5 loc = 0 5 9 13 15 20 0 7 10 3 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] unsorted = 5 nextitem = 0 loc = 0

47. 47 Insertion Sort : Trace (inner loop) 5 9 13 15 20 0 7 10 3 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] unsorted = 5 nextitem = 0 loc = 5 5 9 13 15 20 7 10 3 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] unsorted = 5 nextitem = 0 loc = 4 5 9 13 15 20 7 10 3 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] unsorted = 5 nextitem = 0 loc = 3 5 9 13 15 20 7 10 3 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] unsorted = 5 nextitem = 0 loc = 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] 5 9 9 13 15 20 7 10 3 2 unsorted = 5 nextitem = 0 loc = 1 5 0 9 13 15 20 7 10 3 2 [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] unsorted = 5 nextitem = 0 loc = 0 5

48. 48 Insertion Sort : Analysis The outer loop: for (int unsorted = 1; unsorted < n; ++unsorted) Iterates n -1 times and each iteration selects nextItem to be moved into position. The inner loop: Iterates until nextItem is in its correct position or until the postion into which nextItem will be placed (loc) is at the top of the array: while ((loc > 0) && (theArray[loc-1].compareTo(nextItem) > 0))

49. 49 Insertion Sort : Analysis (2) The number of inner loop iterations depends on the value of nextItem: To analyze use worst case, average case, best case. Worst case: array data is in reverse order. The inner loop will always iterate until nextItem is placed at the top of the array since nextItem is always smaller than the values above.

50. 50 Insertion Sort : Analysis (2) Worst case inner loop iterations: Value of unsorted Number of inner loop iterations 1 2 3 4... n -1 1 2 3 4... n -1

51. 51 Insertion Sort : Analysis (3) Now sum the column containing the number of inner loop iterations: (n -1) + (n -2) + (n -3) +... + 1 = n (n -1) / 2 And thus the growth rate function f(n) is: f(n) = n 2 / 2 - n / 2 and the dominant term is n 2 So Insertion Sort has time complexity O (n 2 ) in the worst case.

52. 52 Insertion Sort : Analysis (4) The number of inner loop iterations depends on the value of nextItem: To analyze use worst case, average case, best case. Best case: array data is in sorted order. The inner loop will never iterate since nextItem is always larger than the values above.

53. 53 Insertion Sort : Analysis (5) Now the outer loop controls time complexity: The outer loop still executes n -1 times. So Insertion Sort has time complexity O (n) in the best case. The average case lies somewhere between O(n) and O(n 2 ) and depends on the specific data values in the array. So - as the array is more nearly sorted the time complexity of Insertion Sort approaches O(n).

54. 54 Shell Sort - Introduction More properly, Shell’s Sort Created in 1959 by Donald Shell Link to a local copy of the article: Donald Shell, “A High-Speed Sorting Procedure”, Communications of the ACM Vol 2, No. 7 (July 1959), 30-32 Originally Shell built his idea on top of Bubble Sort (link to article flowchart), but it has since been transported over to Insertion Sort.link to article flowchart

55. 55 Shell Sort -General Description Essentially a segmented insertion sort Divides an array into several smaller non- contiguous segments The distance between successive elements in one segment is called a gap. Each segment is sorted within itself using insertion sort. Then resegment into larger segments (smaller gaps) and repeat sort. Continue until only one segment (gap = 1) - final sort finishes array sorting.

56. 56 Shell Sort -Background General Theory: Makes use of the intrinsic strengths of Insertion sort. Insertion sort is fastest when: The array is nearly sorted. The array contains only a small number of data items. Shell sort works well because: It always deals with a small number of elements. Elements are moved a long way through array with each swap and this leaves it more nearly sorted.

57. 57 Shell Sort - example 80 93 60 68 1285 42 30 10 Initial Segmenting Gap = 4 10 30 60 68 1285 42 93 80

58. 58 Shell Sort - example (2) 103060681285429380 Resegmenting Gap = 2 101242683093608580

59. 59 Shell Sort - example (3) 101230804285606893 101242683093608580 Resegmenting Gap = 1

60. 60 Gap Sequences for Shell Sort The sequence h 1, h 2, h 3,..., h t is a sequence of increasing integer values which will be used as a sequence (from right to left) of gap values. Any sequence will work as long as it is increasing and h 1 = 1. For any gap value h k we have A[i] <= A[i + h k ] An array A for which this is true is h k sorted. An array which is h k sorted and is then h k-1 sorted remains h k sorted.

61. 61 Shell Sort - Ideal Gap Sequence Although any increasing sequence will work ( if h 1 = 1): Best results are obtained when all values in the gap sequence are relatively prime (sequence does not share any divisors). Obtaining a relatively prime sequence is often not practical in a program so practical solutions try to approximate relatively prime sequences.

62. 62 Shell Sort - Practical Gap Sequences Three possibilities presented: 1) Shell's suggestion - first gap is N/2 - successive gaps are previous value divided by 2. Odd gaps only - like Shell method except if division produces an even number add 1. better performance than 1) since all odd values eliminates the factor 2. 2.2 method - like Odd gaps method (add 1 to even division result) but use a divisor of 2.2 and truncate. best performance of all - most nearly a relatively prime sequence.

63. 63 Shell Sort - Added Gap Sequence Donald Knuth, in his discussion of Shell’s Sort, recommended another sequence of gaps. h 0 = 1 h j+1 = h j * 3 + 1 Find the h j > n, then start with h j /3

64. 64 Link to the Java program that generated the above data.

65. 65 Shell Sort - Time Complexity Time complexity: O(n r ) with 1 < r < 2 This is better than O(n 2 ) but generally worse than O(n log 2 n).

66. 66 Shellsort - Code public static void shellSort( Comparable[ ] theArray, int n ) { // shellSort: sort first n items in array theArray for( int gap = n / 2; gap > 0; gap = gap / 2 ) for( int i = gap; i < n; i++ ) { Comparable tmp = theArray[ i ]; int j = i; for( ; j >= gap && tmp.compareTo(theArray[ j - gap ]) < 0 ; j -= gap ) theArray[ j ] = theArray[ j - gap ]; theArray[ j ] = tmp; }

67. 67 ShellSort -Trace (gap = 4) for( int gap = n / 2; gap > 0; gap = gap / 2 ) for( int i = gap; i < n; i++ ) { Comparable tmp = theArray[ i ]; int j = i; for( ; j >= gap && tmp.compareTo(theArray[ j - gap ]) < 0 ; j -= gap ) theArray[ j ] = theArray[ j - gap ]; theArray[ j ] = tmp; } 8093 60 681285423010 [0] [1] [2] [3] [4] [5] [6] [7] [8] n: 9 gap: 4 i: j: theArray

68. 68 ShellSort -Trace (gap = 2) for( int gap = n / 2; gap > 0; gap = gap / 2 ) for( int i = gap; i < n; i++ ) { Comparable tmp = theArray[ i ]; int j = i; for( ; j >= gap && tmp.compareTo(theArray[ j - gap ]) < 0 ; j -= gap ) theArray[ j ] = theArray[ j - gap ]; theArray[ j ] = tmp; } [0] [1] [2] [3] [4] [5] [6] [7] [8] n: 9 gap: 2 i: j: theArray 103060681285429380

69. 69 ShellSort -Trace (gap = 1) for( int gap = n / 2; gap > 0; gap = gap / 2 ) for( int i = gap; i < n; i++ ) { Comparable tmp = theArray[ i ]; int j = i; for( ; j >= gap && tmp.compareTo(theArray[ j - gap ]) < 0 ; j -= gap ) theArray[ j ] = theArray[ j - gap ]; theArray[ j ] = tmp; } [0] [1] [2] [3] [4] [5] [6] [7] [8] n: 9 gap: 1 i: j: theArray 101242683093608580