1. Searching and Sorting Searching: Sequential, Binary Sorting: Selection, Insertion, Shell

2. Outline n Searching  unsorted lists  sorted lists  Java’s Comparable interface n Sorting  insertion  selection  shell

3. Searching an Unsorted List n Finding out whether an item is in a list  contains method n Linear search:  loop/recur thru list looking at each item »stop when item found or no more list to look at to iterLinS(arr, item): for i = 0..arr.len-1: if arr[i]=item: return true return false to recLinS(arr, len, item): if len = 0: return false if arr[len-1] = item: return true; return recLinS(arr, len-1, item)

4. Linear Search n Worst case? It’s not there!  total of N comparisons  O(N) n But if it is there?  best case: it’s first (1 comparison): O(1)  worst case: it’s last (N comparisons): O(N)  average case: any position equally likely! »1 comparison, 2 comparisons, 3, 4, …, N »average = (1 + 2 + 3 + … + N) / N = (N(N+1)/2) / N = (N+1)/2: O(N)

5. Best, Worst, Average n Number of operations often variable  depends on specific values being used  no single formula that states amount of work! n Formulas for best, worst may be easy to get n Formula for average usually a bit harder n Most interested in worst and average  best is nice, but we don’t expect/worry about it  expect average, prepare for worst

6. Searching a Sorted List n Improved linear search:  stop when we pass where it should have been should have been before 17 so it must not be there should have been before 17 so it must not be there n Saves very little time, actually  best & worst still the same  average still O(N) 5791213172225272842 15

7. Cutting in Half n Binary search better for sorted lists:  look at middle item  if it is bigger than what we’re looking for, then we only need to look in the lower half  if it’s smaller than what we’re looking for, then we only need to look in the upper half  if it’s what we’re looking for – return the index

8. Binary Search n Find midpoint, compare it to the item  too big  search lower part of array  too small  search upper part of array  otherwise  just right! 5791213172225272842 9

9. Binary Search n Repeat until found…  too big  search lower part of array  too small  search upper part of array  otherwise  just right! 5791213172225272842 9

10. Binary Search n Find midpoint, compare it to the item  too big  search lower part of array  too small  search upper part of array  otherwise  just right! 5791213172225272842 4

11. Binary Search n Repeat until found…  too big  search lower part of array  too small  search upper part of array  otherwise  just right! 5791213172225272842 4

12. Binary Search n Repeat until found…  or until nowhere left to look  (return fail) 5791213172225272842 4

13. Binary Search int binaryFind(T item, T v[], int lo, int hi) if (lo > hi) return –1;// not found int mid = lo + (hi – lo)/2; if (item<v[mid]) return binaryFind(item, v, lo, mid–1); if (v[mid]<item) return binaryFind(item, v, mid+1, hi); return mid;// found

14. Binary Search in Java n Want it to work for any kind of array  need a generic method »similar to generic class: public class Pair »similar to generic class: public class Pair  add before return type public static int binaryFind(T item, T[] v, int lo, int hi)  problem: doesn’t compile if (item < v[mid]) bad operand types for binary operator ‘<’ !

15. Comparison in Java n The < operator only works for numbers: if (“this string” < “that string”) n Need to use compareTo method if (“this string”.compareTo(“that string”) < 0) n The T for binaryFind must also have the compareTo method  and must have told Java that it does »recall: interfaces bad operand types for binary operator ‘<’ !

16. The Comparable Interface n Tells Java that this object can be compared  generic interface: what can it be compared to?  usually to other objects of same class public class Person implements Comparable public class Person implements Comparable n Only method required is compareTo public int compareTo(Person other) »returns < 0 if this < other »returns 0 if this == other »returns > 0 if this > other

17. The Comparable Interface n Suppose sorting by height (short to tall) public int compareTo(Person other) { return this.height – other.height; return this.height – other.height;} »returns < 0 if this.height < other.height »returns 0 if this.height == other.height »returns > 0 if this.height > other.height n Suppose sorting by name public int compareTo(Person other) { return this.name.compareTo(other.name); return this.name.compareTo(other.name);}

18. Binary Search in Java n Problem was item < v[mid]  needed to change it to item.compareTo(v[mid]) < 0  item must compare itself to objects of type T  item’s type must be Comparable  item’s type must be Comparable public static int binaryFind(Comparable item, T[] v, int lo, int hi) n That works  works for any type that can compare itself to T »so we could look for a Student in a Person[]

19. binaryFind in Java public static int binaryFind(Comparable item, T[] v, int lo, int hi) { if (lo > hi) { return -1; } if (lo > hi) { return -1; } int mid = lo + (hi - lo) / 2; int mid = lo + (hi - lo) / 2; if (item.compareTo(v[mid]) < 0) { if (item.compareTo(v[mid]) < 0) { return binaryFind(item, v, lo, mid - 1); return binaryFind(item, v, lo, mid - 1); } else if (item.compareTo(v[mid]) > 0) { } else if (item.compareTo(v[mid]) > 0) { return binaryFind(item, v, mid+1, hi); return binaryFind(item, v, mid+1, hi); } else { return mid; }// found it! } else { return mid; }// found it! }

20. Complexity of Binary Search n Worst case, we get half the list every time!  assume N is a power of 2: N == 2 k »1 comparison reduces list from 2 k to 2 k-1 »1 more comparison reduces from 2 k-1 to 2 k-2 »… »1 more comparison reduces from 2 1 to 2 0 (i.e. 1) »1 more comparison reduces from 2 0 to 0 #comparisons = k+1 = 1 + log N = O(log N)#comparisons = k+1 = 1 + log N = O(log N)  shorter lists: 1 + ceiling(log N) = O(log N)

21. Sorted Lists n Sorting a list gives big performance benefit  O(N) vs. O(log N) for N = 1,000,000? »1,000,000 vs. 20 n Sorting lists is a very high priority  lots of different sorting methods  simplest methods not usually very good »bubble sort, anybody?  fastest methods usually hard to explain

22. Some Simple Sorting Methods n Selection sort  “select” items and place in their final position n Insertion sort  “insert” items into a sorted sub-vector n Shell sort  improved insertion sort moves items faster

23. Selection Sort n Based on “copying” a list in order  select the smallest item in the list  add it to end of new list  cross it off the old list n But do it all in one array  front part of list is sorted  won’t need to be changed again – everything left “unsorted” is bigger

24. –23 Selection Sort n Start at low end n For each position in the list…  find the smallest unsorted element  swap it into position 61003–2868 6 100 3 6 8 6 3–28

25. Selection Sort Top-Level Trace lst  [6, 100, 3, -2, 8]; SelectionSort( lst ); 0 1 2 3 4 0 1 2 3 4 lst == 6 100 3 -2 8 1 st pass: find smallest item in list swap it with first item in list i.e. swap -2 with 6

26. Selection Sort Top-Level Trace lst  [6, 100, 3, -2, 8]; SelectionSort( lst ); _0 1 2 3 4 _0 1 2 3 4 lst == 6 100 3 -2 8 lst == [-2] 100 3 6 8 2 nd pass: find smallest remaining item in rest of list swap it into the second position i.e. swap 3 with 100

27. Selection Sort Top-Level Trace lst  [6, 100, 3, -2, 8]; SelectionSort( lst ); 0 1 2 3 4 0 1 2 3 4 lst == 6 100 3 -2 8 lst == [-2] 100 3 6 8 lst == [-2 3] 100 6 8

28. Selection Sort Top-Level Trace lst  [6, 100, 3, -2, 8]; SelectionSort( lst ); 0 1 2 3 4 0 1 2 3 4 lst == 6 100 3 -2 8 lst == [-2] 100 3 6 8 lst == [-2 3] 100 6 8 lst == [-2 3 6] 100 8

29. Selection Sort Top-Level Trace lst  [6, 100, 3, -2, 8]; SelectionSort( lst ); 0 1 2 3 4 0 1 2 3 4 lst == 6 100 3 -2 8 lst == [-2] 100 3 6 8 lst == [-2 3] 100 6 8 lst == [-2 3 6] 100 8 lst == [-2 3 6 8] 100

30. Exercise n Show the evolution of the following array under selection sort. Show the result after each (top-level) pass  [15, 3, 21, 45, 7]

31. Finding the Smallest n Each pass requires finding the smallest item remaining unsorted  need its location in the array, so we can swap it  on ith pass search locations i–1.. N–1  can start assuming first of those is smallest

32. Selection Sort to SelectionSort( List a ) for i  0.. Length(a) – 2 p  i; for j  i + 1.. Length(a) – 1 if (a[j] < a[p]) p  j; Swap(a[p], a[i]);

33. Exercise n List all the comparisons made between array elements while selection-sorting the following list:  [2, 8, 12, 4, 9]

34. Insertion Sort n Based on inserting into an ordered list  For when you’re creating a list  Keep it sorted right from the start  Insert each item into its proper place... ...shifting other items up as required n But all items already in the array  “split” array into sorted list part and input part

35. Insertion Sort n Start at low end n For each unsorted element…  find its place  make space  insert it 61003–28 61006 3–2861003 6 3–2 36100 8–2 100 8

36. Insertion Sort, Top Level Loop n Multiple passes thru the list n At start of ith pass, first i items are in order  list of length 1 always “in order” n At end of ith pass, first i+1 items in order  thus need only N – 1 passes in all n On ith pass, “insert” i+1 st item into sorted part of list

37. Insertion Sort Top-Level Trace lst  [6, 100, 3, -2, 8]; InsertionSort( lst ); _0 _1 2 3 4 _0 _1 2 3 4 lst == [ 6] 100 3 -2 8 1 st pass: list from 0 to 0 is sorted insert item 1 into the sorted part of the list i.e. insert 100 into [6]

38. Insertion Sort Top-Level Trace lst  [6, 100, 3, -2, 8]; InsertionSort( lst ); _0 _1 2 3 4 _0 _1 2 3 4 lst == [ 6] 100 3 -2 8 lst == [ 6 100] 3 -2 8 2 nd pass: list from 0 to 1 is sorted insert item 2 into the sorted part of the list i.e. insert 3 into [6, 100]

39. Insertion Sort Top-Level Trace lst  [6, 100, 3, -2, 8]; InsertionSort( lst ); _0 _1 2 3 4 _0 _1 2 3 4 lst == [ 6] 100 3 -2 8 lst == [ 6 100] 3 -2 8 lst == [ 3 6 100] -2 8

40. Insertion Sort Top-Level Trace lst  [6, 100, 3, -2, 8]; InsertionSort( lst ); _0 _1 2 3 4 _0 _1 2 3 4 lst == [ 6] 100 3 -2 8 lst == [ 6 100] 3 -2 8 lst == [ 3 6 100] -2 8 lst == [-2 3 6 100] 8

41. Insertion Sort Top-Level Trace lst  [6, 100, 3, -2, 8]; InsertionSort( lst ); _0 _1 2 3 4 _0 _1 2 3 4 lst == [ 6] 100 3 -2 8 lst == [ 6 100] 3 -2 8 lst == [ 3 6 100] -2 8 lst == [-2 3 6 100] 8 lst == [-2 3 6 8 100]

42. Exercise n Show the evolution of the following array under insertion sort. Show the result after each (top-level) pass  [15, 3, 21, 45, 7]  [17, 4, 8, 3]

43. Insertion on Pass i n Move i+1 st item into temporary storage n Starting at the “end” of the sorted part… n …move items up until you find where “new” item goes  find a smaller item  fall off the front of the list

44. Insertion on Pass 2 Trace lst == [6, 100], 3, -2, 8 Insert 3 into [6, 100] 0 1 2 temp 0 1 2 temp lst == [ 6 100 3] 3 lst == [ 6 100 100] 3 lst == [ 6 6 100] 3 lst == [ 3 6 100] 3

45. Insertion on Pass 2 Trace lst == [-2, 3, 6, 100], 8 Insert 8 into [-2, 3, 6, 100] 0 1 2 3 4 temp 0 1 2 3 4 temp [ -2 3 6 100 8] 8 [ -2 3 6 100 100] 8 [ -2 3 6 8 100] 8

46. Insertion Sort Pseudo-Code to InsertionSort( List a ) for i  0.. Length(a) – 2 p  i + 1 temp  a[p]; while (p > 0 && a[p – 1] > temp) a[p]  a[p – 1]; p  p – 1; a[p]  temp;

47. Exercise n Show the evolution of the following array under insertion sort. Show the result after each assignment into the array (that is, inner as well as outer loops):  [17, 4, 8, 3]

48. Work in Selection Sort to SelectionSort( List a ) for i  1.. Length(a) – 1 p  i; for j  i + 1.. Length(a) if (a[j] < a[p])p  j; temp  a[p]; a[p]  a[i]; a[i]  temp; Assignment Comparison

49. Counting Ops in Selection Sort n Outer loop runs i from 1 to N – 1  inner runs j from i+1 to N (always) i=1  N–1 (3 + j=i+1  N 1) i=1  N–1 (3 + j=i+1  N 1) = i=1  N–1 (3 + j=1  N 1 – j=1  i 1) = i=1  N–1 (3 + j=1  N 1 – j=1  i 1) = i=1  N–1 (3 + N – i) = i=1  N–1 (3 + N – i) n = (3 + N)(N – 1) – (N – 1)(N)/2 n = N 2 + 2N – 3 – (N 2 – N)/2 = (N 2 + 5N – 6)/2

50. Work in Insertion Sort to InsertionSort( List a ) for i  1.. Length(a) – 1 p  i + 1 temp  a[p]; while (p > 1 && a[p – 1] > temp) a[p]  a[p – 1]; p  p – 1; a[p]  temp; Assignment Comparison

51. Counting Ops in Insertion Sort n Let N be the size of the array/vector  number of elements in the list n Outer loop iterates N – 1 times  contains two assignments n Inner loop iterates at most i times  list was in reverse order – need to compare all  contains one comparison & one assignment

52. Counting Ops in Insertion Sort n Outer loop runs i from 1 to N – 1  inner runs p from i+1 down to 2 (“worst case”) i=1  N–1 (2 + p=2  i+1 2) = i=1  N–1 (2 + 2i) i=1  N–1 (2 + p=2  i+1 2) = i=1  N–1 (2 + 2i) n = 2(N – 1) + 2(N – 1)(N)/2 n = 2N – 2 + N 2 – N n = N 2 + N – 2

53. Comparing Selection & Insertion n T Selection sort (N) = (N 2 + 5N – 6)/2 n T Insertion sort (N) = N 2 + N – 2 n N = 1000 gives:  Selection sort: 502,987 operation  Insertion sort: 1,000,998 operations n Selection looks faster n But this is worst-case time

54. Worst vs. Average n When we did Insertion sort, we said “at most” for the inner loop  the most work it could do – “worst case”  usually will do less n For Selection, we said “always”  never does any less

55. Measuring Complexity n Worst case  imagine that everything goes badly for us  longest possible time for this algorithm to run n Average case  how much work would we expect to do n Best case  everything goes well – rarely considered

56. Average Case Analysis n Sometimes hard to do  not this time n On average, we expect inner loop of insertion to only go ½ way to the front i=1  N–1 (2 + ½(2i)) = 2(N–1) + (N–1)(N)/2 i=1  N–1 (2 + ½(2i)) = 2(N–1) + (N–1)(N)/2 n = 2N – 2 + (N 2 – N)/2 n = (N 2 + 3N – 4)/2

57. Average Selection & Insertion n T Selection average (N) = (N 2 + 5N – 6)/2 n T Insertion average (N) = (N 2 + 3N – 4)/2 n N = 1000 gives:  Selection sort: 502,987 operation  Insertion sort: 501,498 operations n They do about the same amount of work  and both are O(N 2 )

58. Exercise n Show the evolution of each of the following arrays under both insertion and selection sorts  [99, 14, 21, 12, 5]  [3, 7, 14, 5, 2, 8]  [9, 8, 7, 6, 5, 4, 3, 2, 1]

59. Shell Sort n Sort far apart items first  then sort the ones that are closer together n Sort items separated by a given “gap”  every fifth element, for example n Use any sorting method  insertion sort, e.g. n Reduce the gap size & repeat  stop when you’ve sorted with a gap of 1

60. Shell Sort: Gap = 5 1724123211948115757741182829 2481 24 Swap 1794 17 Swap 2911 OK 2832 28 Swap 1812 OK Start at location 6 Up by 1 each time Use insertion sort

61. Shell Sort: Gap = 5 (cont) 9481122811172415757741183229 4181 41 Swap 7794 77 Swap 7529 OK 1532 15 Swap 24 OK 17 OK 28 15 Swap

62. Shell Sort: Gap = 5 (cont) 7741121511172432759481182829 n List is now “5-sorted” 7717944124811218153228117529

63. Shell Sort: Gap = 3 7741121511172432759481182829 n Continue with a smaller gap 77 41 12 15 11 17 24 32 75 94 81 18 28 29 Swap OK Swap (41)

64. Shell Sort: Gap = 3 (cont) 7741121511172432759481182829 n List is now 3-sorted n Note: it’s still 5-sorted 24 28 17 18 11 12 15 81 77 94 32 75 41 29 281532 241294 2428171811121581779432754129

65. Shell Sort: Gap = 1 n Now it’s just normal insertion sort  but everything’s “pretty close” to where it’s going to end up n Number of assignments:  shell sort:13 + 13 + 15 = 41  insertion sort:57 2428171811121581779432754129 15 1215121118 1728 2441297594753294779481

66. Gap Sizes n Any descending sequence will do  so long as it ends at 1 n Some sequences:  N/2, N/4, N/8, …, 1 not especially good  A = 2  log N  – 1, A/2, A/4, …, 1  N/3, N/9, N/27, …, 1  …other sequences may be even better

67. Gap Sizes n N = 13 insertion sort = 52 comps, 65 stores  1 st gap sequence: 6, 3, 141 comps, 77 stores  2 nd gap sequence: 7, 3, 146 comps, 81 stores  3 rd gap sequence: 4, 137 comps, 63 stores n N = 130 (random values) 4487c, 4618s  1 st : 65, 32, 16, 8, 4, 2, 11383c, 2236s  2 nd : 127, 63, 31, 15, 7, 3, 11059c, 1792s  3 rd : 43, 14, 4, 11082c, 1592s 17241232119481157741182829

68. Shell Sort to ShellSort( List a ) gap  Length(a)  2;/* or … */ while (gap  1) if (gap % 2 == 0) ++gap;/* much better! */ for i  gap.. Length(a)–1 p  i,temp  a[p]; while (p  gap && a[p–gap] > temp) a[p]  a[p–gap], p  p–gap; a[p]  temp; gap  gap  2;/* or … */

69. Exercise n Show the evolution of the following arrays under shell sort. Use length/3 as the starting gap:  [15, 3, 21, 45, 7, 17, 4]  [13, 11, 20, 15, 16, 6, 5, 8]

70. Complexity of Shell Sort n Depends on the gap sequence  O(N 2 ) for N/2 version »O(N 3/2 ) if we do the only-odd-numbers version  O(N 3/2 ) for 2  log N  – 1 version »…, 63, 31, 15, 7, 3, 1 »some others have this as well  some versions have O(N 4/3 )  one version has O(N 1+sqrt(8ln(5/2)/ln(N)) )  others we don’t even know!

71. Comments n All O(N 2 ) in worst case  Shell sort can be O(N 3/2 ), but still not good enuf  doubling size of array quadruples time n Want something better! n Recursion offers a way up  next time: merge sort, quick sort