1. Chapter 7: Sorting Algorithms Insertion Sort

2. Sorting Algorithms  Insertion Sort  Shell Sort  Heap Sort  Merge Sort  Quick Sort 2

3. Assumptions  Elements to sort are placed in arrays of length N  Can be compared  Sorting can be performed in main memory 3

4. Complexity Simple sorting algorithms : O(N 2 ) Shellsort: o(N 2 ) Advanced sorting algorithms: O(NlogN) In general: Ω(NlogN) 4

5. Insertion Sort 5 PRE: array of N elements (from 0 to N-1) POST: array sorted 1. An array of one element is sorted 2. Assume that the first p elements are sorted. for j = p to N-1 Take the j-th element and find a place for it among the first j sorted elements

6. Insertion Sort 6 int j, p; comparable tmp; for ( p = 1; p < N ; p++) { tmp = a[p]; for ( j=p; j>0 && tmp < a[ j-1 ] ; j- - ) a[ j ] = a[ j-1 ]; a[ j ] = tmp; } http://www.cs.auckland.ac.nz/software/AlgAnim/sorting.html#bubble

7. Analysis of the Insertion Sort 7 Insert the N-th el.: at most N-1 comparisons N-1 movements Insert the N-1 st el. at most N-2 comparisons N-2 movements Insert the 2 nd el.: 1 comparison 1 movement 2*(1 + 2 +… N - 1) = 2*N * (N-1) / 2 = N(N-1) = Θ (N 2 ) Almost sorted array: O(N) Average complexity: Θ (N 2 )

8. A lower bound for simple sorting algorithms 8 An inversion : an ordered pair (A i, A j ) such that i A j Example: 10, 6, 7, 15, 3,1 Inversions are: (10,6), (10,7), (10,3), (10,1), (6,3), (6,1) (7,3), (7,1) (15,3), (15,1), (3,1)

9. Swapping 9 Swapping adjacent elements that are out of order removes one inversion. A sorted array has no inversions. Sorting an array that contains i inversions requires at least i swaps of adjacent elements How many inversions are there in an average unsorted array?

10. Theorems 10 Theorem 1: The average number of inversions in an array of N distinct elements is N (N - 1) / 4 Theorem 2: Any algorithm that sorts by exchanging adjacent elements requires Ω (N 2 ) time on average. For a sorting algorithm to run in less than quadratic time it must do something other than swap adjacent elements

11. Shell Sort

12.  Improves on insertion sort  Compares elements far apart, then less far apart, finally compares adjacent elements (as an insertion sort). 12

13. Idea  arrange the data sequence in a two-dimensional array  sort the columns of the array  repeat the process each time with smaller number of columns 13

14. Example 14 3 7 9 0 5 1 6 8 4 2 0 6 1 5 7 3 4 9 8 2 it is arranged in an array with 7 columns (left), then the columns are sorted (right): 3 7 9 0 5 1 6 3 3 2 0 5 1 5 8 4 2 0 6 1 5  7 4 4 0 6 1 6 7 3 4 9 8 2 8 7 9 9 8 2

15. Example (cont) 15 3 3 20 0 1 0 5 11 2 2 5 7 43 3 4 4 0 6  4 5 6 1 6 85 6 8 7 9 97 7 9 8 2 8 9 one column in the last step – it is only a 6, an 8 and a 9 that have to move a little bit to their correct position

16. Implementation 16 one-dimensional array that is indexed appropriately. an increment sequence to determine how far apart elements to be sorted are

17. Increment sequence 17 Determines how far apart elements to be sorted are: h 1, h 2,..., h t with h 1 = 1 h k -sorted array - all elements spaced a distance h k apart are sorted relative to each other.

18. Correctness of the algorithm 18 Shellsort only works because an array that is h k -sorted remains h k -sorted when h k- 1 -sorted. Subsequent sorts do not undo the work done by previous phases. The last step (with h = 1) - Insertion Sort on the whole array

19. Java code for Shell sort 19 int j, p, gap; comparable tmp; for (gap = N/2; gap > 0; gap = gap/2) for ( p = gap; p < N ; p++) { tmp = a[p]; for ( j = p ; j >= gap && tmp < a[ j- gap ]; j = j - gap) a[ j ] = a[ j - gap ]; a[j] = tmp; }

20. Increment sequences 20 1. Shell's original sequence: N/2, N/4,..., 1 (repeatedly divide by 2). 2. Hibbard's increments: 1, 3, 7,..., 2 k - 1 ; 3. Knuth's increments: 1, 4, 13,..., ( 3 k - 1) / 2 ; 4. Sedgewick's increments: 1, 5, 19, 41, 109,.... 9 ·4 k - 9 ·2 k + 1 or 4 k - 3 ·2 k + 1.

21. Analysis 21 Shellsort's worst-case performance using Hibbard's increments is Θ(n 3/2 ). The average performance is thought to be about O(n 5/4 ) The exact complexity of this algorithm is still being debated. for mid-sized data : nearly as well if not better than the faster (n log n) sorts.

22. Heap Sort

23.  Basic Idea  Complexity  Example http://www.cs.auckland.ac.nz/software/AlgAnim/heapsort.html 23

24. Idea  Store N elements in a binary heap tree.  Perform delete_Min operation N times, storing each element deleted from the heap into another array.  Copy back the array. Not very efficient to use two arrays. Improvement – use one array for the binary heap and the sorted elements 24

25. Improvements Use the same array to store the deleted elements instead of using another array After each deletion we get a vacant position in the array - the last cell. There we store the deleted element, which becomes part of the sorted sequence. 25

26. Improvements When all the elements are deleted and stored in the same array following the above method, the elements will be there in reversed order. What is the remedy for this? Store the elements in the binary heap tree in reverse order of priority - then at the end the elements in the array will be in correct order. 26

27. Complexity Sorts in O(NlogN) time by performing N times deleteMax operations. - Each deleteMax operation takes log N running time. - N times performing deleteMax  NlogN running time Used for general purpose sorting, guarantees O(N logN) 27

28. Example 28 15191071716 1. Consider the values of the elements as priorities and build the heap tree. 2. Start deleteMax operations, storing each deleted element at the end of the heap array.

29. Example (cont) 29 Note that we use only one array, treating its parts differently: when sorting, part of the array will be the heap, and the rest part - the sorted array

30. Build the Heap 30 We start with the element at position SIZE/2 comparing the item with the children. The hole is percolated down to position 6 and the item is inserted there. 151971716 10 Result: 15191671710 holechild

31. Build the Heap 31 Next we compare position 2 with its children. 151671710 19 holechild1child2 19 is greater than 7 and 17, and we continue with position 1 15191671710

32. Build the Heap 32 Percolate down the hole at position 1 191671710 15 The hole at position 1 is percolated down to position 2 -the greater child. 191671710 15

33. Build the Heap 33 Percolate down the hole at position 2 191671710 15 One of the children of the hole at position 2 - item 17, is greater than 15. So we percolate the hole to position 5. 19171671510

34. Build the Heap 34 19171671510 19 1716 71510 the heap is built

35. Sorting 35 DeleteMax the top element 19 171671519 1716 715 Store the last heap element (10) in a temporary place. Move the DeletedMax element (19) to the place where the last heap element was - the current available position in the sorted portion of the array. A hole is created at the top 10

36. Sorting 36 Percolate down the hole 1671519 16 715 10 17

37. Sorting 37 Percolate down the hole 1671519 16 7 15 10 17

38. Sorting 38 Fill the hole 1671519 10 16 7 15 1017

39. Sorting 39 DeleteMax the top element 17 1671519 16 7 15 17 10 Store the last heap element (10) in a temporary place. Move the DeletedMax element (17) to the place where the last heap element was - the current available position in the sorted portion of the array. A hole is created at the top

40. Sorting 40 Percolate down the hole 1671519 16 7 15 17 10

41. Sorting 41 Fill the hole 1671519 10 16 7 15 1710

42. Sorting 42 DeleteMax the top element 16 161519 1015 1710 Store the last heap element (7) in a temporary place. Move the DeletedMax element (16) to the place where the last heap element was - the current available position in the sorted portion of the array. A hole is created at the top 7

43. Sorting 43 Percolate down the hole 161519 10 15 1710 7

44. Sorting 44 Fill the hole 161519 10 7 15 17107

45. Sorting 45 DeleteMax the top element 15 161519 7 17 10 7 Store the last heap element (10) in a temporary place. Move the DeletedMax element (15) to the place where the last heap element was - the current available position in the sorted portion of the array. A hole is created at the top

46. Sorting 46 Percolate down the hole 161519 7 17 10 7 Since 10 is greater than the children of the hole, It has to be inserted in the hole

47. Sorting 47 Fill the hole 161519 7 10 17107

48. Sorting 48 DeleteMax the top element 10 16151917 7 Store the last heap element (7) in a temporary place. Move the DeletedMax element (10) to the place where the last heap element was - the current available position in the sorted portion of the array. A hole is created at the top 10

49. Sorting 49 Fill the hole 161519 7 17710 The hole has no children and so it has to be filled.

50. Sorted array 50 16151917710 7 is the last element from the heap, so now the array is sorted