1. HKOI 2006 Intermediate Training Searching and Sorting 1/4/2006

2. Sorting - Introduction Sorting is the ordering of your data in a consistent manner. Each element is usually part of a collection of data called a record. Each record contains a key, which is the value to be sorted

3. Sorting - Introduction Ordering--We require the existence of the “ ” operators, which can be used to place a consistent ordering on the input. Eg. Sort the elements: Dog, Cat, Mouse, Tiger, Turtoise

4. Sorting - Introduction There are several easy algorithms to sort in O(n 2 ) There are slightly more complicated O(n log n) sorting algorithms. Any binary comparison based sorting algorithm has complexity of at least O(n log n)

5. Sorting - Introduction Different Sorting Methods: Bubble Sort Insertion Sort Selection Sort Quick Sort Heap Sort Shell Sort Merge Sort Radix Sort

6. Bubble Sort This is probably the simplest way sort an array of objects. The basic idea is to compare two neighboring objects, and to swap them if they are in the wrong order In each pass, the largest key in the list will be bubbled to the end, but the earlier keys may still be out of order.

7. Bubble Sort Example Original 34 8 64 51 32 21 # of swaps ---------------------------------------- After p = 1, 8 34 51 32 21 64 4 After p = 2, 8 34 32 21 51 64 2 After p = 3, 8 32 21 34 51 64 2 After p = 4, 8 21 32 34 51 64 1 After p = 5, 8 21 32 34 51 64 0 After p = 6, 8 21 32 34 51 64 0

8. Bubble Sort Algorithm: for i = 1 to N-1 do for j = 1 to N-i do if A[j].key > A[j+1].key then swap(A[j],A[j+1])

9. Bubble Sort Improved version: for i = 1 to N-1 do begin for j = 1 to N-i do if A[j].key > A[j+1].key then swap(A[j],A[j+1]) if no swapping was done, then stop here end

10. Bubble Sort Complexity? O(n 2 ) If the file is initially sorted in the reverse order, the total number of comparisons is (n-1) + (n-2) +... + 2 + 1 = (n-1) * n/2

11. Bubble Sort Advantage: It requires little additional space (one additional record to swap and few simple integer variables to process) It is O(n) in the case that the file is completely sorted (or almost completely sorted)

12. Insertion Sort One of the simplest methods An example of an insertion sort occurs in everyday life while playing cards. To sort the cards in your hand you extract a card, shift the remaining cards, and then insert the extracted card in the correct place. This process is repeated until all the cards are in the correct sequence

13. Insertion Sort Example Original 34 8 64 51 32 21 # of Moves ------------------------------------------ After p = 1, 8 34 64 51 32 21 1 After p = 2, 8 34 64 51 32 21 0 After p = 3, 8 34 51 64 32 21 1 After p = 4, 8 32 34 51 64 21 3 After p = 5, 8 21 32 34 51 64 4

14. Insertion Sort Algorithm for p = 2 to N temp = A[p]; for j = p downto 2 if A[j-1] > temp A[j] = A[j-1]; j = j – 1; A[j] = temp;

15. Insertion Sort Complexity? O(n 2 )

16. Insertion Sort Advantages: Similar to those mentioned in Bubble Sort

17. Additional Information on Insertion Sort: Improvement: using a binary search. This reduces the total number of comparisons from O(n 2 ) to O(n log n). However, the moving operation still requires O(n 2 ) time. So the binary search does not significantly improves the overall time bound. Another improvement is to use the linked-list insertion. This reduces the time required for insertion but not the time for searching for the proper position.

18. Selection Sort It is also called Straight Selection or Push- Down Sort. A selection sort is one in which successive elements are selected in order and placed into their proper sorted positions. The elements of the input may have to be preprocessed to make the ordered selection possible.

19. Selection Sort Example Original 34 8 64 51 32 21 # of swaps ---------------------------------------- After p = 1, 8 34 64 51 32 21 1 After p = 2, 8 21 64 51 32 34 1 After p = 3, 8 21 32 51 64 34 1 After p = 4, 8 21 32 34 64 51 1 After p = 5, 8 21 32 34 51 64 1

20. Selection Sort Algorithm: for i = 1 to N-1 lowindex = i; lowkey = A[i]; for j = i+1 to N if A[j] < lowkey lowindex = j; lowkey = A[j] swap (A[i], A[lowindex]);

21. Selection Sort Complexity? O(n 2 ) Advantage?

22. Stability A stable sorting algorithm is any sorting algorithm that preserves the relative ordering of items with equal values Before: (a,3),(b,5),(c,2),(d,5),(e,4) After: (c,2),(a,3),(e,4),(b,5),(d,5)

23. Stability Which of the discussed sorting methods are stable? It depends on your implementation

24. Merge Sort Merge sort is good for external sorting (data are stored in disks or tapes) The basic operation of merge sort is to merge 2 sorted lists Technique of divide-and-conquer

25. Merge Sort Merge 2 sorted list: 11324262152738 Aptr Bptr Cptr 11324262152738 1 11324262152738 12 11324262152738 1213

26. Merge Sort 11324262152738 121315 11324262152738 12131524 11324262152738 1213152426 11324262152738 12131524262738 Array A is exhausted! Copy remainder of array B

27. Merge Sort Merge sort follows the divide-and-conquer approach Divide: Divide the n-element sequence into two (n/2)-element subsequences Conquer: Sort the two subsequences recursively Combine: Merge the two sorted subsequence to produce the answer

28. Merge Sort Complexity? O(n log n)

29. Quick Sort Divide-and-conquer process for sorting an array A[p..r] Divide: A[p..r] is partitioned into two nonempty subarrays A[p..q] and A[q+1..r] such that each element of A[p..q] is less than each element of A[q+1..r] Conquer: The two subarrays A[p..q] and A[q+1..r] are sorted by recursive calls to quicksort. Combine: Since the subarrays are sorted in place, no work is needed to combine them

30. Quick Sort Quicksort(S) { if size of S is 0 or 1 then return; pick a pivot v divide S - {v} into S 1 and S 2 such that every element in S 1 is <= v every element in S 2 is >= v return { Quicksort(S 1 ) v Quicksort(S 2 ) } }

31. Quick Sort Questions: How to pick a pivot? How to divide S into S 1 and S 2 ? S v S1S1 S2S2 v

32. Quick Sort How to pick a pivot? First element in S For example: 42 34 8 2 6 21 5 32 1 Median-of-Three median of first, center and last element For example: 42 34 8 2 6 21 5 32 1 Pivot = median of 42, 6 and 1 = 6 Pivot = 42

33. Quick Sort How to divide S into S 1 and S 2 ? Based on the value of pivot, we put elements that are <= pivot to the left of pivot elements that are >= pivot to the right of pivot For example, On the left of pivot: 10, 11, 9, 15 are <= 25 On the right of pivot: 31, 43, 62, 81 are >= 25 15259314362118110 Pivot

34. Quick Sort One of the methods Steps: (1) Swap the first, center and last element in S such that first element <= center element <= last element (2) The pivot v is the center element, swap it with the element before the last element (3) Make two pointers i, j to point to first element and the element before the last elements respectively (4) Move i pointer in right direction to point to a number >= pivot v (5) Move j pointer in left direction to point to a number <= pivot v (6) If i pointer is right of j pointer, goto step (7), else swap the elements pointed by i and j and goto step (4) (7) Swap the element pointed to i with the pivot v (the element before the last element in S)

35. Quick Sort Example:10 11 9 15 5 1 3 2 8 155913211810 Step 1: Swap the first, center and last element in S such that first element <= center element <= last element

36. Quick Sort Example:10 11 9 15 5 1 3 2 8 158913211105 Step 1: After swapping. Therefore the pivot is 8

37. Quick Sort Example:10 11 9 15 5 1 3 2 8 152913811105 Step 2: Swap the pivot with the element before the last element.

38. Quick Sort Example:10 11 9 15 5 1 3 2 8 152913811105 Step 3: Make two pointers i, j to point to first element and the element just before the last elements respectively ij

39. Quick Sort Example:10 11 9 15 5 1 3 2 8 152913811105 Step 4: Move i pointer in right direction to point to a number >= pivot v ij

40. Quick Sort Example:10 11 9 15 5 1 3 2 8 152913811105 Step 5: Move j pointer in left direction to point to a number <= pivot v ij

41. Quick Sort Example:10 11 9 15 5 1 3 2 8 152911183105 Step 6: As i pointer is NOT right of j pointer, swap the elements pointed by i and j and goto step (4) ij

42. Quick Sort Example:10 11 9 15 5 1 3 2 8 152911183105 Step 4: Move i pointer in right direction to point to a number >= pivot v i j

43. Quick Sort Example:10 11 9 15 5 1 3 2 8 152911183105 Step 5: Move j pointer in left direction to point to a number <= pivot v i j

44. Quick Sort Example:10 11 9 15 5 1 3 2 8 152191183105 Step 6: As i pointer is NOT right of j pointer, swap the elements pointed by i and j and goto step (4) i j

45. Quick Sort Example:10 11 9 15 5 1 3 2 8 152191183105 Step 4: Move i pointer in right direction to point to a number >= pivot v i j

46. Quick Sort Example:10 11 9 15 5 1 3 2 8 152191183105 Step 5: Move j pointer in left direction to point to a number <= pivot v ij

47. Quick Sort Example:10 11 9 15 5 1 3 2 8 215191183105 Step 6: As i pointer is NOT right of j pointer, swap the elements pointed by i and j and goto step (4) ij

48. Quick Sort Example:10 11 9 15 5 1 3 2 8 215191183105 Step 4: Move i pointer in right direction to point to a number >= pivot v ij

49. Quick Sort Example:10 11 9 15 5 1 3 2 8 215191183105 Step 5: Move j pointer in left direction to point to a number <= pivot v ij

50. Quick Sort Example:10 11 9 15 5 1 3 2 8 215191183105 Step 6: As i pointer is right of j pointer, goto step (7) ij

51. Quick Sort Example:10 11 9 15 5 1 3 2 8 281911153105 Step 7: Swap the element pointed to i with the pivot (the element before the last element in S) ij

52. Quick Sort Example:10 11 9 15 5 1 3 2 8 281911153105 Done!!! You can see that all elements in S 1 is = pivot v Pivot v S1S1 S2S2

53. Quick Sort Complexity? Best: O(n log n) Worst: O(n 2 ) Average: O(n log n) Not a stable sort

54. Bucket Sort Assume 0<A[i]<=M Algorithm: initialize an array Count of size M to all 0's for i=1 to N inc(Count[A[i]]) Scan the Count array, and printout the sorted list

55. Bucket Sort Complexity: O(M+N) It can be used for small integers and limited by M

56. Radix Sort It is also called card sort. Algorithm: initialize an array of 10 buckets to empty for i=1 to N read A[i] and place it into the bucket the last digit Use the same process to sort the second last digit repeat until the first digit

57. Radix Sort Sort into buckets on the least significant digit first, then proceed to the most significant digit Initial: 64, 8, 216, 512, 27, 729, 0, 1, 343, 125 0123456789 By the least significant digit: 01 512343 64 125216 278 729 After First Pass: 0, 1, 512, 343, 64, 125, 216, 27, 8, 729

58. Radix Sort 1st Pass Result: 0, 1, 512, 343, 64, 125, 216, 27, 8, 729 0123456789 By the tens digit: 810810 216 512 729 27 125 343 64 After Second Pass: 0, 1, 8, 512, 216, 125, 27, 729, 343, 64 Second Pass:

59. Radix Sort 2nd Pass Result: 0, 1, 8, 512, 216, 125, 27, 729, 343, 64 0123456789 By the most significant digit: 0, 1, 8, 27, 64, 125, 216, 343, 512, 729 Third Pass: After Last Pass: 64 27 8 1 0 125 216512729343

60. Radix Sort Complexity: O(dn), d is the number of digits Stable sort