1. Algorithms Sorting

2. Sorting Definition It is a process for arranging a collection of items in an order. Sorting can arrange both numeric and alphabetic data in increasing or decreasing order. 2 135246 123456 Unsorted array sorted array

3. 3 1. Bubble Sort it is one of the oldest sorts known. The bubble sort got its name because of the way the biggest elements "bubble" to the top. It based on the property of a sorted list that any two adjacent elements are in sorted order.

4. Bubble sort Compare each element (except the last one) with its neighbor to the right – If they are out of order, swap them – This puts the largest element at the very end – The last element is now in the correct and final place Compare each element (except the last two) with its neighbor to the right – If they are out of order, swap them – This puts the second largest element next to last – The last two elements are now in their correct and final places Compare each element (except the last three) with its neighbor to the right – Continue as above until you have no unsorted elements on the left 4

5. 5 Example Consider a set of data: 5 9 2 8 4 6 3. Bubble sort first compares the first two elements, the 5 and the 9. Because they are already in sorted order, nothing happens. The next pair of numbers, the 9 and the 2 are compared. Because they are not in sorted order, they are swapped and the data becomes: 5 2 9 8 4 6 3.

6. 6 To better understand the "bubbling" nature of the sort, watch how the largest number, 9, "bubbles" to the top in the first iteration of the sort. First iteration (5 9) 2 8 4 6 3 --> compare 5 and 9, no swap 5 (9 2) 8 4 6 3 --> compare 9 and 2, swap 5 2 (9 8) 4 6 3 --> compare 9 and 8, swap 5 2 8 (9 4) 6 3 --> compare 9 and 4, swap 5 2 8 4 (9 6) 3 --> compare 9 and 6, swap 5 2 8 4 6 (9 3) --> compare 9 and 3, swap 5 2 8 4 6 3 9 --> first iteration complete Example (cont)

7. 7 Notice that in the example, the largest element, 9, got swapped all the way into its correct position at the end of the list. This happens because in each comparison, the larger element is always pushed towards its place at the end of the list. Example (cont)

8. 8 Second iteration In the second iteration, the second-largest element will be bubbled up to its correct place in the same manner: (5 2) 8 4 6 3 9 --> compare 5 and 2, swap 2 (5 8) 4 6 3 9 --> compare 5 and 8, no swap 2 5 (8 4) 6 3 9 --> compare 8 and 4, swap 2 5 4 (8 6) 3 9 --> compare 8 and 6, swap 2 5 4 6 (8 3) 9 --> compare 8 and 3, swap 2 5 4 6 3 8 9 --> no need to compare last two because the last element is known to be the largest. Example (cont)

9. Third iteration (2 5) 4 6 3 8 9 -->compare 2 and 5 no swap, 2 (5 4) 6 3 8 9 -->compare 5 and 4 swap, 2 4 (5 6) 3 8 9 -->compare 5 and 6 no swap, 2 4 5 (6 3) 8 9 -->compare 6 and 3 swap, 2 4 5 3 6 8 9 -->no need to compare the last three elements 9

10. 10 Fourth iteration (2 4) 5 3 6 8 9 --> compare 2 and 4, no swap 2 (4 5) 3 6 8 9 --> compare 4 and 5, no swap 2 4 (5 3) 6 8 9 --> compare 5 and 3, swap 2 4 3 5 6 8 9 --> no need to compare the last four elements. Fifth iteration (2 4) 3 5 6 8 9 --> compare 2 and 4, no swap 2 (4 3) 5 6 8 9 --> compare 4 and 3, swap 2 3 4 5 6 8 9 --> compare 4 and 5, no swap 2 3 4 5 6 8 9 --> no need to compare the last five elements. Sixth iteration (2 3) 4 5 6 8 9 --> compare 2 and 3, no swap. no need to compare the last six elements. Example (cont)

11. Example of bubble sort 7 2854 2 7854 2 7854 2 7584 2 7548 2 7548 2 5748 2 5478 2 7548 2 5478 2 4578 2 5478 2 4578 2 4578 (done) 11

12. Bubble Sorting Algorithm Algorithm: Input array of size N for(i=n-1;i>=1;i--) { for(y=1;y<=i; y++) { if(x[y-1]>x[y]) { temp=x[y-1]; x[y-1]=x[y]; x[y]=temp; } Output sorted array of Size N

13. Code for bubble sort public static void bubbleSort(int[] a) { int outer, inner; for (outer = a.length - 1; outer > 0; outer--) { // counting down for (inner = 0; inner a[inner + 1]) { // if out of order... int temp = a[inner]; //...then swap a[inner] = a[inner + 1]; a[inner + 1] = temp; } } } } 13

14. Selection sort Given an array of length n, – Search elements 0 through n-1 and select the smallest Swap it with the element in location 0 – Search elements 1 through n-1 and select the smallest Swap it with the element in location 1 – Search elements 2 through n-1 and select the smallest Swap it with the element in location 2 – Search elements 3 through n-1 and select the smallest Swap it with the element in location 3 – Continue in this fashion until there’s nothing left to search

15. 15 Selection Sort 2954816 1954826 1254896 1245896 1245698 1245896 1245689 int[] myList = {2, 9, 5, 4, 8, 1, 6}; // Unsorted

16. Selection Sort outmin = 1 3 out 16

17. Example of selection sort The selection sort might swap an array element with itself--this is harmless, and not worth checking for 7 2854 2 7854 2 4857 2 4587 2 4578 17

18. Chapter 10: Sorting18 Selection Sort Example 3565306020 scan 0-4, smallest 20 swap 35 and 20 2065306035scan 1-4, smallest 30 swap 65 and 30 2030656035scan 2-4, smallest 35 swap 65 and 35 2030356065scan 3-4, smallest 60 swap 60 and 60 2030356065done

19. Chapter 10: Sorting19 Selection Sort Algorithm 1.for fill = 0 to n-1 do 2.set posMin to fill for next = fill+1 to n-1 do 3. if item at next < item at posMin 4. set posMin to next 5. Exchange item at posMin with one at fill

20. Code for selection sort public static void selectionSort(int[] a) { int outer, inner, min; for (outer = 0; outer < a.length - 1; outer++) { min = outer; for (inner = outer + 1; inner < a.length; inner++) { if (a[inner] < a[min]) { min = inner; } // Invariant: for all i, if outer <= i <= inner, then a[min] <= a[i] } // a[min] is least among a[outer]..a[a.length - 1] int temp = a[outer]; a[outer] = a[min]; a[min] = temp; // Invariant: for all i <= outer, if i < j then a[i] <= a[j] } } 20

21. I NSERTION S ORT One of the simplest methods to sort an array is an insertion sort, If the first few objects are already sorted, an unsorted object can be inserted in the sorted set in proper place. An example of an insertion sort occurs in everyday life while playing cards. To sort the cards in your hand you extract a card, shift the remaining cards, and then insert the extracted card in the correct place. This process is repeated until all the cards are in the correct sequence.

22. Chapter 10: Sorting22 Insertion Sort Based on technique of card players to arrange a hand – Player keeps cards picked up so far in sorted order – When the player picks up a new card Makes room for the new card Then inserts it in its proper place

23. Insertion sort Example int[] myList = {2, 9, 5, 4, 8, 1, 6}; // Unsorted The insertion sort algorithm sorts a list of values by repeatedly inserting an unsorted element into a sorted sublist until the whole list is sorted.

24. 24 Insertion sort Example 2954816 2954816 2594816 2458916 1245896 2459816 1245689 int[] myList = {2, 9, 5, 4, 8, 1, 6}; // Unsorted

25. 25 Starting near the top of the array in the Figure (a) we extract the 3. Then the above elements are shifted down until we find the correct place to insert the 3. This process repeats in Figure (b) with the next number. Finally, in Figure (c), we complete the sort by inserting 2 in the correct place.

26. for i = 2 to N do newElement = list[ i ] location = i - 1 while (location ≥ 1) and (list[ location ] > newElement) do list[ location + 1 ] = list[ location ] location = location - 1 end while list[ location + 1 ] = newElement end for I NSERTION S ORT A LGORITHM 26

27. Conclusion and Summary Bubble sort is useful for small amounts of data. Selection sort can be used when amount of data is small and swapping is time- consuming. Insertion sort is most versatile and the best when the list is almost sorted. 27

28. Quick Sort As the name implies quicksort is the fastest known sorting algorithm in practice. Quick-sort is a randomized sorting algorithm based on the divide-and-conquer prototype Select a pivot and split the data into two groups: ( pivot): Recursively apply Quicksort to the subgroups

29. 29 Quick-Sort Quick-sort is a randomized sorting algorithm based on the divide-and-conquer prototype: Divide: pick a random element x (called pivot) and partition S into L elements less than x E elements equal x G elements greater than x Recur: sort L and G Conquer: join L, E and G x x LG E x

30. 30 Quicksort – Basic Algorithm Sorting an array S consists of 4 easy steps: quicksort(S) { 1. If the number of elements in S is 0 or 1, then return S 2. Pick any element v in S. This is called the pivot. 3. Partition S-{v} into 2 disjoint groups: S 1 ={xєS-{v}| x =v} 4. return {quicksort(S 1 ) {v}, quicksort(S 2 )} } Partition step ambiguous for elements equal to the pivot.

31. 31 Picking the Pivot

32. Quicksort Start Unsorted Array Start with all data in an array, and consider it unsorted

33. Quicksort Step 1 26 Step 1, select a pivot (it is arbitrary) We will select the first element, as presented in the original algorithm. 33352919 pivot 1222

34. Quicksort Step 2 26 Step 2, start process of dividing data into LEFT and RIGHT groups: The LEFT group will have elements less than the pivot. The RIGHT group will have elements greater that the pivot. Use markers left and right 33352919 left pivot 1222 right

35. Quicksort Step 3 26 Step 3, If left element belongs to LEFT group, then increment left index. If right index element belongs to RIGHT, then decrement right. Exchange when you find elements that belong to the other group. 33352919 left pivot 1222 right

36. Quicksort Step 4 26 Step 4: Element 33 belongs to RIGHT group. Element 22 belongs to LEFT group. Exchange the two elements. 33352919 left pivot 1222 right 2622352919 left pivot 1233 right

37. Quicksort Step 5 26 Step 5: After the exchange, increment left marker, decrement right marker. 22352919 left pivot 1233 right

38. Quicksort Step 6 26 Step 6: Element 35 belongs to RIGHT group. Element 12 belongs to LEFT group. Exchange, increment left, and decrement right. 22352919 left pivot 1233 right 2622122919 left pivot 3533 right

39. Quicksort Step 7 26 Step 7: Element 29 belongs to RIGHT. Element 19 belongs to LEFT. Exchange, increment left, decrement right. 22122919 left pivot 3533 right 2622121929 left pivot 3533 right

40. Quicksort Step 8 26 Step 8: When the left and right markers pass each other, we are done with the partition task. Swap the right with pivot. 22121929 left pivot 3533 right 26 19221229 pivot 3533 LEFTRIGHT

41. Quicksort Step 9 Step 9: Apply Quicksort to the LEFT and RIGHT groups, recursively. Assemble parts when done pivot 26 19221229 previous pivot 3533 Quicksort pivot 121922293335 26 121922293335

42. Example Recursive implementation with the left most array entry selected as the pivot element.

43. Quicksort code p: first element r: last element Quicksort(A, p, r) { if (p < r) { q = Partition(A, p, r) Quicksort(A, p, q-1) Quicksort(A, q+1, r) } Initial call is Quicksort(A, 1, n), where n in the length of A 43

44. Partition Clearly, all the action takes place in the partition() function – Rearranges the subarray in place – End result: Two subarrays All values in first subarray  all values in second – Returns the index of the “pivot” element separating the two subarrays 44

45. Partition Code Partition(A, p, r) { x = A[r]// x is pivot i = p - 1 for j = p to r – 1 { do if A[j] <= x then { i = i + 1 exchange A[i]  A[j] } exchange A[i+1]  A[r] return i+1 } 45

46. Heap A heap is a data structure that stores a collection of objects (with keys), and has the following properties: – Complete Binary tree – Heap Order It is implemented as an array where each node in the tree corresponds to an element of the array. H EAP S ORT 46

47. Binary Tree a binary tree is a tree data structure in which each node has at most two children (referred to as the left child and the right child).treedata structurechildren In a binary tree, the degree of each node can be at most two. Binary trees are used to implement binary search and heap sortbinary search heapsort 4 5 3 Parent node Left child node Right child node

48. Binary Tree Each node of the binary tree corresponds to an element of the array. The array is completely filled on all levels except possibly lowest PARENT (i) return floor(i/2)-1 LEFT (i) return 2i+1 RIGHT (i) return 2i + 2 19 1216 4 1 7 1914127 Array A

49. Binary Tree 2945353121 4 12 5 26251415 I 0 1 2 3 4 5 6 7 8 9 10 11 array 4 5 12 26 25 14 15 29 45 35 31 21 size = 12 Level=0 Level=1 Level=2 Level=3 Binary tree of Size 12 and depth 3

50. Binary tree A complete binary tree is a binary tree in which every level, except possibly the last, is completely filled, and all nodes are as far left as possible(left justified). This type of tree is used as a specialized data structure called a heap.heap 19 1216 4 1 7 19 1216 4 1 7 12 Compete or not??

51. What is a Binary Heap? A binary heap is a complete binary tree with one (or both) of the following heap order properties: MinHeap property: Each node must have a key that is less or equal to the key of each of its children. MaxHeap property: Each node must have a key that is greater or equal to the key of each of its children. A binary heap satisfying the MinHeap property is called a MinHeap. A binary heap satisfying the MaxHeap property is called a MaxHeap. Recall: A complete binary tree may have missing nodes only on the right side of the lowest level.

52. Max Heap Example 161914127 Array A 19 1216 4 1 7 52

53. Min heap example 127191641 Array A 1 416 12 7 19 53

54. siftUp Given a node that does not have the heap property, you can give it the heap property by exchanging its value with the value of the larger child This is sometimes called sifting up Notice that the child may have lost the heap property 14 812 Blue node has Maxheap property 12 814 Blue node does not have Maxheap property 54

55. Plan of attack First, we will learn how to turn a binary tree into a heap Next, we will learn how to turn a binary tree back into a heap after it has been changed in a certain way Finally (this is the cool part) we will see how to use these ideas to sort an array 55

56. Constructing a heap I A tree consisting of a single node is automatically a heap We construct a heap by adding nodes one at a time: – Add the node just to the right of the rightmost node in the deepest level – If the deepest level is full, start a new level Examples: Add a new node here 56

57. Constructing a heap II Each time we add a node, we may destroy the heap property of its parent node To fix this, we sift up But each time we sift up, the value of the topmost node in the sift may increase, and this may destroy the heap property of its parent node We repeat the sifting up process, moving up in the tree, until either – We reach nodes whose values don’t need to be swapped (because the parent is still larger than both children), or – We reach the root 57

58. Constructing a heap III 8 8 10 8 85 85 12 10 125 8 105 8 123 4 58

59. Other children are not affected The node containing 8 is not affected because its parent gets larger, not smaller The node containing 5 is not affected because its parent gets larger, not smaller The node containing 8 is still not affected because, although its parent got smaller, its parent is still greater than it was originally 12 105 814 12 145 810 14 125 810 59

60. A sample heap Here’s a sample binary tree after it has been heapified Notice that heapified does not mean sorted Heapifying does not change the shape of the binary tree; 19 1418 22 321 14 119 15 25 1722 60

61. Removing the root Notice that the largest number is now in the root Suppose we discard the root: How can we fix the binary tree so it is once again ? Solution: remove the rightmost leaf at the deepest level and use it for the new root 19 1418 22 321 14 119 15 1722 11 61

62. The reHeap method I Our binary tree is no longer have the Max heap property However, only the root lacks the Maxheap property We can siftUp() the root After doing this, one and only one of its children may have lost the Maxheap property 19 1418 22 321 14 9 15 1722 11 62

63. The reHeap method II Now the left child of the root (still the number 11 ) lacks the Maxheap property We can siftUp() this node After doing this, one and only one of its children may have lost the Maxheap property 19 1418 22 321 14 9 15 1711 22 63

64. The reHeap method III Now the right child of the left child of the root (still the number 11 ) lacks the Maxheap property: We can siftUp() this node After doing this, one and only one of its children may have lost the Maxheap property —but it doesn’t, because it’s a leaf 19 1418 11 321 14 9 15 1722 64

65. The reHeap method IV Our tree is once again a heap, because every node in it has the Maxheap property Once again, the largest (or a largest) value is in the root We can repeat this process until the tree becomes empty This produces a sequence of values in order largest to smallest 19 1418 21 311 14 9 15 1722 65

66. Sorting What do heaps have to do with sorting an array? Here’s the neat part: – Because the compete binary tree can be represented as an array – All our operations on the complete binary trees can be represented as operations on arrays – To sort: heapify the array; while the array isn’t empty { remove and replace the root; reheap the new root node; } 66

67. Mapping into an array Notice: – The left child of index i is at index 2*i+1 – The right child of index i is at index 2*i+2 – Example: the children of node 3 (19) are 7 (18) and 8 (14) 19 1418 22 321 14 119 15 25 1722 252217192214151814213911 0 1 2 3 4 5 6 7 8 9 10 11 12 67

68. Removing and replacing the root The “root” is the first element in the array The “rightmost node at the deepest level” is the last element Swap them......And pretend that the last element in the array no longer exists—that is, the “last index” is 11 (9) 252217192214151814213911 0 1 2 3 4 5 6 7 8 9 10 11 12 112217192214151814213925 0 1 2 3 4 5 6 7 8 9 10 11 12 68

69. Array Representation of a Binary Heap A heap is a dynamic data structure that is represented and manipulated more efficiently using an array. Since a heap is a complete binary tree, its node values can be stored in an array, without any gaps, in a breadth-first order, where: Value(node i+1 ) array[ i ], for i > 0 21 24 6526 32 3119 16 68 13 32266568193124162113 9876543210 The root is array[0] The parent of array[i] is array[(i – 1)/2], where i > 0 The left child, if any, of array[i] is array[2i+1]. The right child, if any, of array[i] is array[2i+2].

70. Array Representation of a Binary Heap (contd.) We shall use an implementation in which the heap elements are stored in an array starting at index 1. Value(node i )array[i], for i > 1 21 24 6526 32 3119 16 68 13 32266568193124162113 9876543210 10 The root is array[1]. The parent of array[i] is array[i/2], where i > 1 The left child, if any, of array[i] is array[2i]. The right child, if any, of array[i] is array[2i+1].

71. Insertion Algorithm 1.Add the new element to the next available position at the lowest level 2.Restore the max-heap property if violated General strategy is percolate up (or bubble up): if the parent of the element is smaller than the element, then interchange the parent and child. OR Restore the min-heap property if violated General strategy is percolate up (or bubble up): if the parent of the element is larger than the element, then interchange the parent and child. 71

72. 19 1216 4 1 7 19 1216 4 1 717 19 1217 4 1 716 Insert 17 swap Percolate up to maintain the heap property 72

73. Deletion Delete max – Copy the last number to the root ( overwrite the maximum element stored there ). – Restore the max heap property by percolate down. Delete min – Copy the last number to the root ( overwrite the minimum element stored there ). – Restore the min heap property by percolate down. 73

74. Heap Sort A sorting algorithm that works by first organizing the data to be sorted into a special type of binary tree called a heap Heapify Build Heap Heap Sort P ROCEDURES ON H EAP 74

75. Heapify Heapify picks the largest child key and compare it to the parent key. If parent key is larger than heapify quits, otherwise it swaps the parent key with the largest child key. So that the parent is now becomes larger than its children. Heapify(A, i) { l  left(i) (using 2i+1 r  right(i) (using 2i+1 if l A[i] then largest  l else largest  i if r A[largest] then largest  r if largest != i then swap A[i]  A[largest] Heapify(A, largest) } 75

76. BUILD HEAP We can use the procedure 'Heapify' in a bottom-up fashion to convert an array A[1.. n] into a heap. Since the elements in the subarray A[n/2 +1.. n] are all leaves, the procedure BUILD_HEAP goes through the remaining nodes of the tree and runs 'Heapify' on each one. The bottom-up order of processing node guarantees that the subtree rooted at children are heap before 'Heapify' is run at their parent. Buildheap(A) { heapsize[A]  length[A] for i   length[A]/2  down to 1 do Heapify(A, i) } 76

77. Heap Sort Algorithm The heap sort algorithm starts by using procedure BUILD-HEAP to build a heap on the input array A[1.. n]. Since the maximum element of the array stored at the root A[1], it can be put into its correct final position by exchanging it with A[n] (the last element in A). If we now discard node n from the heap than the remaining elements can be made into heap. Note that the new element at the root may violate the heap property. All that is needed to restore the heap property. Heapsort(A) { Buildheap(A) for i  length[A] down to 2 do swap A[1]  A[i] heapsize[A]  heapsize[A] - 1 Heapify(A, 1) } 77

78. Example: Convert the following array to a heap 164711219 Picture the array as a complete binary tree: 16 47 12 1 19 78

79. 16 47 12 1 19 16 419 12 1 7 16 1219 4 1 7 1216 4 1 7 swap 79

80. Heap Sort The heapsort algorithm consists of two phases: - build a heap from an arbitrary array - use the heap to sort the data To sort the elements in the decreasing order, use a min heap To sort the elements in the increasing order, use a max heap 19 1216 4 1 7 80

81. Example of Heap Sort 19 1216 4 1 7 19 12 16 1 4 7 Array A Sorted: Take out biggest Move the last element to the root 81

82. 1216 4 1 7 19 12 16 1 4 7 Array A Sorted: HEAPIFY() swap 82

83. 12 16 4 1 7 19 12 16 1 4 7 Array A Sorted: 83

84. 12 16 4 1 7 19 12 16 1 4 7 Array A Sorted: Take out biggest Move the last element to the root 84

85. 12 4 1 7 19 12 16 1 4 7 Array A Sorted: 85

86. 12 4 1 7 19 12 16 1 4 7 Array A Sorted: HEAPIFY() swap 86

87. 12 4 1 7 19 12 16 1 4 7 Array A Sorted: 87

88. 12 4 1 7 19 12 16 1 4 7 Array A Sorted: Take out biggest Move the last element to the root 88

89. 4 1 7 19 12 16 1 4 7 Array A Sorted: swap 89

90. 4 1 7 19 12 16 1 4 7 Array A Sorted: 90

91. 4 1 7 19 12 16 1 4 7 Array A Sorted: Move the last element to the root Take out biggest 91

92. 4 1 19 12 16 1 4 7 Array A Sorted: HEAPIFY() swap 92

93. 4 1 19 12 16 1 4 7 Array A Sorted: Move the last element to the root Take out biggest 93

94. 1 19 12 16 1 4 7 Array A Sorted: Take out biggest 94