1. Computer Science and Software Engineering University of Wisconsin - Platteville 12. Heap Yan Shi CS/SE 2630 Lecture Notes Partially adopted from C++ Plus Data Structure textbook slides

2. What is a Heap?  Shape: complete binary tree  Order: for each node in the heap, the value stored in that node is greater than or equal to the value in each of its children. —Root always has the largest element  By default, when we say “heap”, we mean “maximum heap” —minimum heap: for each node in the heap, the value stored in that node is less than or equal to the value in each of its children

3. Are these Heaps? C A T treePtr 50 20 18 30 10

4. Are these Heaps? 35 25 18 30 22 6 50 20 50

5. Store a Heap in Array 70 0 60 1 40 3 30 4 12 2 8 5 tree [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 70 60 12 40 30 8 tree.nodes For any tree node nodes[index] left child: nodes[index*2 + 1] right child: nodes[index*2 + 2] parent: nodes[(index – 1)/2]

6. Construct a Heap  How to guarantee it is a complete binary tree? —treat the data as an array/sequence, add nodes to the tree in a top-down, left-to-right manner  How to guarantee the order? —bubble the values as high as it can go

7. Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 10 20 12 40 20 15 tree.nodes

8. Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 10 20 12 40 20 15 nodes 10 Add nodes[0] to the tree: it has no parent, so we have a heap.

9. Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 10 20 12 40 20 15 tree.nodes 10 20 Add nodes[1] to the tree: 20 > 10, so we need to bubble 20 up. Exchange 20 and 10 in both the tree and the array.

10. Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 20 10 12 40 20 15 tree.nodes 20 10 Add nodes[1] to the tree: now we have a heap.

11. Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 20 10 12 40 20 15 tree.nodes 20 10 Add nodes[2] to the tree: 12 < 20. It cannot go higher. We have a heap. 12

12. Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 20 10 12 40 20 15 tree.nodes 20 10 Add nodes[3] to the tree: 40 < 10. We need to bubble 40 up. Exchange 40 and 10 in both tree and array. 12 40

13. Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 20 40 12 10 20 15 tree.nodes 20 40 Add nodes[3] to the tree: 40 < 20. We need to bubble 40 up. Exchange 40 and 20 in both tree and array. 12 10

14. Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 40 20 12 10 20 15 tree.nodes 40 20 Add nodes[3] to the tree: 40 cannot go higher. We have a heap. 12 10

15. Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 40 20 12 10 20 15 tree.nodes 40 20 Add nodes[4] to the tree: 20 = 20. It cannot go higher. We have a heap. 12 1020

16. Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 40 20 12 10 20 15 tree.nodes 40 20 Add nodes[5] to the tree: 15 < 12. We need to bubble 15 up. Exchange 15 and 12 in both tree and array. 12 1020 15

17. Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 40 20 15 10 20 12 tree.nodes 40 20 Add nodes[5] to the tree: 15 < 12. We need to bubble 15 up. Exchange 15 and 12 in both tree and array. 15 1020 12

18. Example: Done! [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 40 20 15 10 20 12 tree.nodes 40 20 Add nodes[5] to the tree: 15 cannot go higher. We have a heap. All nodes are added. Done! 15 1020 12

19. Construct a Heap: Bubble Up Algorithm  Given nodes[] and size  For each node at index i: —Leaf = i; Done = false —while Leaf is not 0 AND Done == false  Parent = (Leaf-1)/2  if nodes[Leaf] > nodes[Parent] Swap nodes[Leaf] and nodes[Parent] Leaf = Parent;  otherwise Done = true  What is the time complexity? O(nlog 2 n)

20. ReheapUp Algorithm  Based on the previous algorithm.  What if we want to add a new item to an existing heap? —add the item as the last element in the array  it is the only item that violates the order —restore the order in a bottom-up manner —we can do it recursively! void ReheapUp( int root, int bottom ) { int parent ; if ( bottom > root ){ parent = ( bottom - 1 ) / 2; if ( nodes [ parent ] < nodes [ bottom ] ){ Swap ( nodes [ parent ], nodes [ bottom ] ); ReheapUp ( root, parent ); }

21. Sorting Using Heap  Heap is perfect for sorting: the root is always the largest!  Basic algorithm: —use heap in which top node contains largest value —repeat until heap is empty:  take root (largest value) off heap, put it in its place  reconstruct the heap  How to reconstruct the heap?

22. Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 40 20 15 10 20 12 tree.nodes 40 20 Take root off heap 15 1020 12

23. Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 40 20 15 10 20 12 tree.nodes 20 Put the last one in the array (12) as the new root: Only the root violates the order. Its left and right subtrees are still heaps! We also have space to store the old root 40. 15 1020 12 What should we put here?

24. Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 12 20 15 10 20 40 tree.nodes 12 20 12 < 20 and 12 < 15 Swap 12 with its child: which one? 15 1020 The larger one so that we guarantee the new parent >= both children

25. Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 20 12 15 10 20 40 tree.nodes 20 12 12 < 20: Swap 12 with 20 15 1020

26. Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 20 15 10 12 40 tree.nodes 20 12 < 20: Swap 12 with 20 It is already a leaf node. We stop. 15 1012

27. ReheapDown Algorithm  To facilitate heapSort.  Root is the only node that violates the order: —reorder the heap by pushing down the root as low as possible  This procedure is also known as Heapify.  How to use Heapify to remove a node? ReheapDown ( root, bottom ) maxChild = index of the larger child of nodes[root] if nodes[root] < nodes[maxChild] Swap( nodes[root], nodes[maxChild] ReheapDown( maxChild, bottom )

28. Example: Continued [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 20 15 10 12 40 tree.nodes 20 Swap the root (20) and bottom (12) Remove 20 from the heap 15 1012

29. Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 12 20 15 10 20 40 tree.nodes 12 20 ReheapDown 15 10

30. Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 20 12 15 10 20 40 tree.nodes 20 12 Swap the root (20) and bottom (10). It is a heap now. Remove 20 from the heap 15 10

31. Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 10 12 15 20 40 tree.nodes 10 12 ReheapDown 15

32. Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 10 12 15 20 40 tree.nodes 15 12 Swap the root (15) and bottom (10) Remove 15 from the heap 10

33. Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 10 12 15 20 40 tree.nodes 10 12 Reheap

34. Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 12 10 15 20 40 tree.nodes 12 10 Swap the root (12) and bottom (10) Remove 12 from the heap

35. Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 10 12 15 20 40 tree.nodes 10 We have only one node in the heap. Simply remove 10 from the heap and we are done!

36. Example: [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 10 12 15 20 40 tree.nodes Note that now the array is sorted in ascending order! This is called Heap Sort!

37. Heap Sort  Algorithm: 1.Construct a heap H of n nodes 2.for i: length(H)-1 to 1 1. Swap H[0] and H[i] 2. Reduce H’s size by 1 3. ReheapDown( 0, i - 1 ) What is the time complexity?  Construct a heap using ReheapUp takes O(nlog 2 n)  ReheapDown takes O(log 2 n) which repeats n-1 times  O(nlog 2 n)

38. A Better Algorithm to Construct a Heap  Inspired by the Heapify process, we have another algorithm to build a heap:  How to guarantee it is a complete binary tree? —treat the data as an array/sequence, and convert it into a complete binary tree in a top-down, left-to-right manner  How to guarantee the order? —Leaf nodes are already heaps (single node) —Starting from the first non-leaf node till the root, make sure each node is larger than its children  if not, do ReheapDown

39. Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 10 20 12 40 20 15 tree.nodes 10 2012 4020 15 Leaf nodes are already heaps. So we start from Level 1: ReheapDown( 2, 5);

40. Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 10 20 15 40 20 12 tree.nodes 10 2015 4020 12 ReheapDown( 1, 5);

41. Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 10 40 15 20 12 tree.nodes 10 4015 20 12 ReheapDown( 0, 5);

42. Example [ 0 ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] 40 20 15 10 20 12 tree.nodes 40 2015 1020 12 Now we have a heap!

43. Construct a Heap by Heapify  Algorithm: 1.Elements are stored in array A 2.for i: from (size(A)/2 – 1) to 0 1. ReheapDown( i, size(A)-1)  What is the time complexity? O(n) !!!

44. Why O(n) to build a heap?  Assume it is a full tree.  Total work to build a heap is:  Since, we can get  So we have T(n) = 2 h+1  Since h = log 2 n, we have T(n) = n+1  O(n) http://www.cs.umd.edu/~meesh/351/mount/lectures/lect14-heapsort-analysis-part.pdf

45. Summary  Insert a new item to a heap: —The last element violates the order —Use ReheapUp algorithm  Heap Sort: —Keep taking off the root which is the largest —replace the root with the last element in the heap —The new root violates the order —Use ReheapDown algorithm —O(nlog 2 n)  Build a heap: —by ReheapUp: O(nlog 2 n) —by ReheapDown: O(n)