1. Sorting Cont

2. Quick Sort As the name implies quicksort is the fastest known sorting algorithm in practice. Quick-sort is a randomized sorting algorithm based on the divide-and-conquer prototype Select a pivot and split the data into two groups: ( pivot): Recursively apply Quicksort to the subgroups

3. 3 Quick-Sort Quick-sort is a randomized sorting algorithm based on the divide-and-conquer prototype: Divide: pick a random element x (called pivot) and partition S into L elements less than x E elements equal x G elements greater than x Recur: sort L and G Conquer: join L, E and G x x LG E x

4. 4 Quicksort – Basic Algorithm Sorting an array S consists of 4 easy steps: quicksort(S) { 1. If the number of elements in S is 0 or 1, then return S 2. Pick any element v in S. This is called the pivot. 3. Partition S-{v} into 2 disjoint groups: S 1 ={xєS-{v}| x =v} 4. return {quicksort(S 1 ) {v}, quicksort(S 2 )} } Partition step ambiguous for elements equal to the pivot.

5. 5 Picking the Pivot

6. Quicksort Start Unsorted Array Start with all data in an array, and consider it unsorted

7. Quicksort Step 1 26 Step 1, select a pivot (it is arbitrary) We will select the first element, as presented in the original algorithm. 33352919 pivot 1222

8. Quicksort Step 2 26 Step 2, start process of dividing data into LEFT and RIGHT groups: The LEFT group will have elements less than the pivot. The RIGHT group will have elements greater that the pivot. Use markers left and right 33352919 left pivot 1222 right

9. Quicksort Step 3 26 Step 3, If left element belongs to LEFT group, then increment left index. If right index element belongs to RIGHT, then decrement right. Exchange when you find elements that belong to the other group. 33352919 left pivot 1222 right

10. Quicksort Step 4 26 Step 4: Element 33 belongs to RIGHT group. Element 22 belongs to LEFT group. Exchange the two elements. 33352919 left pivot 1222 right 2622352919 left pivot 1233 right

11. Quicksort Step 5 26 Step 5: After the exchange, increment left marker, decrement right marker. 22352919 left pivot 1233 right

12. Quicksort Step 6 26 Step 6: Element 35 belongs to RIGHT group. Element 12 belongs to LEFT group. Exchange, increment left, and decrement right. 22352919 left pivot 1233 right 2622122919 left pivot 3533 right

13. Quicksort Step 7 26 Step 7: Element 29 belongs to RIGHT. Element 19 belongs to LEFT. Exchange, increment left, decrement right. 22122919 left pivot 3533 right 2622121929 left pivot 3533 right

14. Quicksort Step 8 26 Step 8: When the left and right markers pass each other, we are done with the partition task. Swap the right with pivot. 22121929 left pivot 3533 right 26 19221229 pivot 3533 LEFTRIGHT

15. Quicksort Step 9 Step 9: Apply Quicksort to the LEFT and RIGHT groups, recursively. Assemble parts when done pivot 26 19221229 previous pivot 3533 Quicksort pivot 121922293335 26 121922293335

16. Example Recursive implementation with the left most array entry selected as the pivot element.

17. Quicksort code p: first element r: last element Quicksort(A, p, r) { if (p < r) { q = Partition(A, p, r) Quicksort(A, p, q-1) Quicksort(A, q+1, r) } Initial call is Quicksort(A, 1, n), where n in the length of A 17

18. Partition Clearly, all the action takes place in the partition() function – Rearranges the subarray in place – End result: Two subarrays All values in first subarray  all values in second – Returns the index of the “pivot” element separating the two subarrays 18

19. Partition Code Partition(A, p, r) { x = A[r]// x is pivot i = p - 1 for j = p to r – 1 { do if A[j] <= x then { i = i + 1 exchange A[i]  A[j] } exchange A[i+1]  A[r] return i+1 } 19

20. Heap A heap is a data structure that stores a collection of objects (with keys), and has the following properties: – Complete Binary tree – Heap Order It is implemented as an array where each node in the tree corresponds to an element of the array. H EAP S ORT 20

21. Binary Tree a binary tree is a tree data structure in which each node has at most two children (referred to as the left child and the right child).treedata structurechildren In a binary tree, the degree of each node can be at most two. Binary trees are used to implement binary search and heap sortbinary search heapsort 4 5 3 Parent node Left child node Right child node

22. Binary Tree Each node of the binary tree corresponds to an element of the array. The array is completely filled on all levels except possibly lowest PARENT (i) return floor(i/2)-1 LEFT (i) return 2i+1 RIGHT (i) return 2i + 2 19 1216 4 1 7 1914127 Array A

23. Binary Tree 2945353121 4 12 5 26251415 I 0 1 2 3 4 5 6 7 8 9 10 11 array 4 5 12 26 25 14 15 29 45 35 31 21 size = 12 Level=0 Level=1 Level=2 Level=3 Binary tree of Size 12 and depth 3

24. Binary tree A complete binary tree is a binary tree in which every level, except possibly the last, is completely filled, and all nodes are as far left as possible(left justified). This type of tree is used as a specialized data structure called a heap.heap 19 1216 4 1 7 19 1216 4 1 7 12 Compete or not??

25. What is a Binary Heap? A binary heap is a complete binary tree with one (or both) of the following heap order properties: MinHeap property: Each node must have a key that is less or equal to the key of each of its children. MaxHeap property: Each node must have a key that is greater or equal to the key of each of its children. A binary heap satisfying the MinHeap property is called a MinHeap. A binary heap satisfying the MaxHeap property is called a MaxHeap. Recall: A complete binary tree may have missing nodes only on the right side of the lowest level.

26. Max Heap Example 161914127 Array A 19 1216 4 1 7 26

27. Min heap example 127191641 Array A 1 416 12 7 19 27

28. siftUp Given a node that does not have the heap property, you can give it the heap property by exchanging its value with the value of the larger child This is sometimes called sifting up Notice that the child may have lost the heap property 14 812 Blue node has Maxheap property 12 814 Blue node does not have Maxheap property 28

29. Plan of attack First, we will learn how to turn a binary tree into a heap Next, we will learn how to turn a binary tree back into a heap after it has been changed in a certain way Finally (this is the cool part) we will see how to use these ideas to sort an array 29

30. Constructing a heap I A tree consisting of a single node is automatically a heap We construct a heap by adding nodes one at a time: – Add the node just to the right of the rightmost node in the deepest level – If the deepest level is full, start a new level Examples: Add a new node here 30

31. Constructing a heap II Each time we add a node, we may destroy the heap property of its parent node To fix this, we sift up But each time we sift up, the value of the topmost node in the sift may increase, and this may destroy the heap property of its parent node We repeat the sifting up process, moving up in the tree, until either – We reach nodes whose values don’t need to be swapped (because the parent is still larger than both children), or – We reach the root 31

32. Constructing a heap III 8 8 10 8 85 85 12 10 125 8 105 8 123 4 32

33. Other children are not affected The node containing 8 is not affected because its parent gets larger, not smaller The node containing 5 is not affected because its parent gets larger, not smaller The node containing 8 is still not affected because, although its parent got smaller, its parent is still greater than it was originally 12 105 814 12 145 810 14 125 810 33

34. A sample heap Here’s a sample binary tree after it has been heapified Notice that heapified does not mean sorted Heapifying does not change the shape of the binary tree; 19 1418 22 321 14 119 15 25 1722 34

35. Removing the root Notice that the largest number is now in the root Suppose we discard the root: How can we fix the binary tree so it is once again ? Solution: remove the rightmost leaf at the deepest level and use it for the new root 19 1418 22 321 14 119 15 1722 11 35

36. The reHeap method I Our binary tree is no longer have the Max heap property However, only the root lacks the Maxheap property We can siftUp() the root After doing this, one and only one of its children may have lost the Maxheap property 19 1418 22 321 14 9 15 1722 11 36

37. The reHeap method II Now the left child of the root (still the number 11 ) lacks the Maxheap property We can siftUp() this node After doing this, one and only one of its children may have lost the Maxheap property 19 1418 22 321 14 9 15 1711 22 37

38. The reHeap method III Now the right child of the left child of the root (still the number 11 ) lacks the Maxheap property: We can siftUp() this node After doing this, one and only one of its children may have lost the Maxheap property —but it doesn’t, because it’s a leaf 19 1418 11 321 14 9 15 1722 38

39. The reHeap method IV Our tree is once again a heap, because every node in it has the Maxheap property Once again, the largest (or a largest) value is in the root We can repeat this process until the tree becomes empty This produces a sequence of values in order largest to smallest 19 1418 21 311 14 9 15 1722 39

40. Sorting What do heaps have to do with sorting an array? Here’s the neat part: – Because the compete binary tree can be represented as an array – All our operations on the complete binary trees can be represented as operations on arrays – To sort: heapify the array; while the array isn’t empty { remove and replace the root; reheap the new root node; } 40

41. Mapping into an array Notice: – The left child of index i is at index 2*i+1 – The right child of index i is at index 2*i+2 – Example: the children of node 3 (19) are 7 (18) and 8 (14) 19 1418 22 321 14 119 15 25 1722 252217192214151814213911 0 1 2 3 4 5 6 7 8 9 10 11 12 41

42. Removing and replacing the root The “root” is the first element in the array The “rightmost node at the deepest level” is the last element Swap them......And pretend that the last element in the array no longer exists—that is, the “last index” is 11 (9) 252217192214151814213911 0 1 2 3 4 5 6 7 8 9 10 11 12 112217192214151814213925 0 1 2 3 4 5 6 7 8 9 10 11 12 42

43. Array Representation of a Binary Heap A heap is a dynamic data structure that is represented and manipulated more efficiently using an array. Since a heap is a complete binary tree, its node values can be stored in an array, without any gaps, in a breadth-first order, where: Value(node i+1 ) array[ i ], for i > 0 21 24 6526 32 3119 16 68 13 32266568193124162113 9876543210 The root is array[0] The parent of array[i] is array[(i – 1)/2], where i > 0 The left child, if any, of array[i] is array[2i+1]. The right child, if any, of array[i] is array[2i+2].

44. Array Representation of a Binary Heap (contd.) We shall use an implementation in which the heap elements are stored in an array starting at index 1. Value(node i )array[i], for i > 1 21 24 6526 32 3119 16 68 13 32266568193124162113 9876543210 10 The root is array[1]. The parent of array[i] is array[i/2], where i > 1 The left child, if any, of array[i] is array[2i]. The right child, if any, of array[i] is array[2i+1].

45. Insertion Algorithm 1.Add the new element to the next available position at the lowest level 2.Restore the max-heap property if violated General strategy is percolate up (or bubble up): if the parent of the element is smaller than the element, then interchange the parent and child. OR Restore the min-heap property if violated General strategy is percolate up (or bubble up): if the parent of the element is larger than the element, then interchange the parent and child. 45

46. 19 1216 4 1 7 19 1216 4 1 717 19 1217 4 1 716 Insert 17 swap Percolate up to maintain the heap property 46

47. Deletion Delete max – Copy the last number to the root ( overwrite the maximum element stored there ). – Restore the max heap property by percolate down. Delete min – Copy the last number to the root ( overwrite the minimum element stored there ). – Restore the min heap property by percolate down. 47

48. Heap Sort A sorting algorithm that works by first organizing the data to be sorted into a special type of binary tree called a heap Heapify Build Heap Heap Sort P ROCEDURES ON H EAP 48

49. Heapify Heapify picks the largest child key and compare it to the parent key. If parent key is larger than heapify quits, otherwise it swaps the parent key with the largest child key. So that the parent is now becomes larger than its children. Heapify(A, i) { l  left(i) (using 2i+1 r  right(i) (using 2i+1 if l A[i] then largest  l else largest  i if r A[largest] then largest  r if largest != i then swap A[i]  A[largest] Heapify(A, largest) } 49

50. BUILD HEAP We can use the procedure 'Heapify' in a bottom-up fashion to convert an array A[1.. n] into a heap. Since the elements in the subarray A[n/2 +1.. n] are all leaves, the procedure BUILD_HEAP goes through the remaining nodes of the tree and runs 'Heapify' on each one. The bottom-up order of processing node guarantees that the subtree rooted at children are heap before 'Heapify' is run at their parent. Buildheap(A) { heapsize[A]  length[A] for i   length[A]/2  down to 1 do Heapify(A, i) } 50

51. Heap Sort Algorithm The heap sort algorithm starts by using procedure BUILD-HEAP to build a heap on the input array A[1.. n]. Since the maximum element of the array stored at the root A[1], it can be put into its correct final position by exchanging it with A[n] (the last element in A). If we now discard node n from the heap than the remaining elements can be made into heap. Note that the new element at the root may violate the heap property. All that is needed to restore the heap property. Heapsort(A) { Buildheap(A) for i  length[A] down to 2 do swap A[1]  A[i] heapsize[A]  heapsize[A] - 1 Heapify(A, 1) } 51

52. Example: Convert the following array to a heap 164711219 Picture the array as a complete binary tree: 16 47 12 1 19 52

53. 16 47 12 1 19 16 419 12 1 7 16 1219 4 1 7 1216 4 1 7 swap 53

54. Heap Sort The heapsort algorithm consists of two phases: - build a heap from an arbitrary array - use the heap to sort the data To sort the elements in the decreasing order, use a min heap To sort the elements in the increasing order, use a max heap 19 1216 4 1 7 54

55. Example of Heap Sort 19 1216 4 1 7 19 12 16 1 4 7 Array A Sorted: Take out biggest Move the last element to the root 55

56. 1216 4 1 7 19 12 16 1 4 7 Array A Sorted: HEAPIFY() swap 56

57. 12 16 4 1 7 19 12 16 1 4 7 Array A Sorted: 57

58. 12 16 4 1 7 19 12 16 1 4 7 Array A Sorted: Take out biggest Move the last element to the root 58

59. 12 4 1 7 19 12 16 1 4 7 Array A Sorted: 59

60. 12 4 1 7 19 12 16 1 4 7 Array A Sorted: HEAPIFY() swap 60

61. 12 4 1 7 19 12 16 1 4 7 Array A Sorted: 61

62. 12 4 1 7 19 12 16 1 4 7 Array A Sorted: Take out biggest Move the last element to the root 62

63. 4 1 7 19 12 16 1 4 7 Array A Sorted: swap 63

64. 4 1 7 19 12 16 1 4 7 Array A Sorted: 64

65. 4 1 7 19 12 16 1 4 7 Array A Sorted: Move the last element to the root Take out biggest 65

66. 4 1 19 12 16 1 4 7 Array A Sorted: HEAPIFY() swap 66

67. 4 1 19 12 16 1 4 7 Array A Sorted: Move the last element to the root Take out biggest 67

68. 1 19 12 16 1 4 7 Array A Sorted: Take out biggest 68