1. 1 CSE 326: Data Structures Priority Queues (Heaps) Lecture 9: Monday, Jan 27, 2003

2. 2 Not Quite Queues Consider applications –ordering CPU jobs –searching for the exit in a maze –emergency room admission processing Problems? –short jobs should go first –most promising nodes should be searched first –most urgent cases should go first

3. 3 Priority Queue ADT Priority Queue operations Priority Queue property: for two elements in the queue, x and y, if x has a lower priority value than y, x will be deleted before y F(7) E(5) D(100) A(4) B(6) insert deleteMin G(9) C(3) create :  heap insert : heap  value  heap findMin : heap  value deleteMin : heap  heap is_empty : heap  boolean create :  heap insert : heap  value  heap findMin : heap  value deleteMin : heap  heap is_empty : heap  boolean

4. 4 Applications of the Priority Q Hold jobs for a printer in order of length Store packets on network routers in order of urgency Simulate events Anything greedy

5. 5 Discrete Event Simulation An event is a pair (x,t) where x describes the event and t is time it should occur A discrete event simulator (DES) maintains a set S of events which it intends to simulate in time order repeat { Find and remove (x 0,t 0 ) from S such that t 0 is minimum; Do whatever x 0 says to do; in the process new events (x 1,t 1 )…(x k,t k ) may be generated; Insert the new events into S; } repeat { Find and remove (x 0,t 0 ) from S such that t 0 is minimum; Do whatever x 0 says to do; in the process new events (x 1,t 1 )…(x k,t k ) may be generated; Insert the new events into S; }

6. 6 Emergency Room Simulation Patient arrive at time t with injury of criticality C –If no patients waiting and a free doctor, assign them to doctor and create a future departure event; else put patient in the Criticality priority queue Patient departs at time t –If someone in Criticality queue, pull out most critical and assign to doctor; create a future departure event time queue arrive(t,c) criticality (triage) queue assign patient to doctor patient generator depart(t) arrive(t,c)

7. 7 Naïve Priority Queue Data Structures Unsorted list: –insert: –findMin: –deleteMin: Sorted list: –insert: –findMin: –deleteMin:

8. 8 BST Tree Priority Queue Data Structure 4 121062 115 8 14 13 79 Regular BST: –insert: –findMin: –deleteMin: AVL Tree: –insert: –findMin: –deleteMin: Can we do better?

9. 9 Binary Heap Priority Q Data Structure 201412911 81067 54 2 Heap-order property –parent’s key is less than children’s keys –result: minimum is always at the top Structure property –complete tree with fringe nodes packed to the left –result: depth is always O(log n); next open location always known How do we find the minimum?

10. 10 201412911 81067 54 2 24576 811912142012 123456789101112 1 23 4 56 7 8 9 1011 12 Nifty Storage Trick Calculations: –child: –parent: –root: –next free: 0

11. 11 201412911 81067 54 2 24576 811912142012 123456789101112 1 23 4 56 7 8 9 1011 12 Nifty Storage Trick Calculations: –child: left = 2*node right=2*node+1 –parent: floor(node/2) –root: 1 –next free: length+1 0

12. 12 findMin 201412911 81067 54 2 pqueue.findMin() 2 Time = O(1)

13. 13 DeleteMin 201412911 81067 54 20 2 1412911 81067 54 2 pqueue.deleteMin()

14. 14 Percolate Down 1412911 81067 54 20 1412911 81067 520 4 1412911 810207 56 4 1420911 810127 56 4

15. 15 DeleteMin Code Comparable deleteMin(){ x = A[1]; A[1]=A[size--]; percolateDown(1); return x; } percolateDown(int hole) { tmp=A[hole]; while (2*hole <= size) { left = 2*hole; right = left + 1; if (right <= size && A[right] < A[left]) target = right; else target = left; if (A[target] < tmp) { A[hole] = A[target]; hole = target; } else break; } A[hole] = tmp; } Trick to avoid repeatedly copying the value at A[1] Move down Time = O(log n) (why ?)

16. 16 Insert 201412911 81067 54 2 201412911 81067 54 2 pqueue.insert(3) 3

17. 17 Percolate Up 201412911 81067 54 2 3201412911 8367 54 2 10 201412911 8567 34 2 10

18. 18 Insert Code void insert(Comparable x) { // Efficiency hack: we won’t actually put x // into the heap until we’ve located the position // it goes in. This avoids having to copy it // repeatedly during the percolate up. int hole = ++size; // Percolate up for( ; hole>1 && x < A[hole/2] ; hole = hole/2) A[hole] = A[hole/2]; A[hole] = x; } Time = O(log n) (why ?)

19. 19 Performance of Binary Heap In practice: binary heaps much simpler to code, lower constant factor overhead Binary heap worst case Binary heap avg case AVL tree worst case BST tree avg case Insert O(log n)O(1) percolates 1.6 levels O(log n) Delete Min O(log n)

20. 20 Changing Priorities In many applications the priority of an object in a priority queue may change over time –if a job has been sitting in the printer queue for a long time increase its priority –unix “renice” Must have some (separate) way of find the position in the queue of the object to change (e.g. a hash table)

21. 21 Other Priority Queue Operations decreaseKey –Given the position of an object in the queue, increase its priority (lower its key). Fix heap property by: increaseKey –given the position of an an object in the queue, decrease its priority (increase its key). Fix heap property by: remove –given the position of an an object in the queue, remove it. Do increaseKey to infinity then …

22. 22 BuildHeap Task: Given a set of n keys, build a heap all at once Approach 1: Repeatedly perform Insert(key) Complexity:

23. 23 BuildHeap Floyd’s Method 511310694817212 pretend it’s a heap and fix the heap-order property! 27184 96103 115 12 buildHeap(){ for (i=size/2; i>0; i--) percolateDown(i); }

24. 24 Build(this)Heap 67184 92103 115 12 671084 9213 115 12 1171084 9613 25 12 1171084 9653 21 12

25. 25 Finally… 11710812 9654 23 1

26. 26 Complexity of Build Heap Note: size of a perfect binary tree doubles (+1) with each additional layer At most n/4 percolate down 1 level at most n/8 percolate down 2 levels at most n/16 percolate down 3 levels… O(n)

27. 27 Thinking about Heaps Observations –finding a child/parent index is a multiply/divide by two –operations jump widely through the heap –each operation looks at only two new nodes –inserts are at least as common as deleteMins Realities –division and multiplication by powers of two are fast –looking at one new piece of data terrible in a cache line –with huge data sets, disk accesses dominate

28. 28 4 9654 23 1 81012 7 11 Solution: d-Heaps Each node has d children Still representable by array Good choices for d: –optimize performance based on # of inserts/removes –choose a power of two for efficiency –fit one set of children in a cache line –fit one set of children on a memory page/disk block 3728512111069112

29. 29 New Operation: Merge Merge(H1,H2): Merge two heaps H1 and H2 of size O(N). –E.g. Combine queues from two different sources to run on one CPU. 1.Can do O(N) Insert operations: O(N log N) time 2.Better: Copy H2 at the end of H1 (assuming array implementation) and use Floyd’s Method for BuildHeap. Running Time: O(N) Can we do even better? (i.e. Merge in O(log N) time?)

30. 30 Binomial Queues All in O(log n) time Recursive Definition of Binomial Tree B k of height k: –B 0 = single root node –B k = Attach B k-1 to root of another B k-1 Idea: a binomial heap H is a forest of binomial trees: H = B 0 B 1 B 2... B k where each B i may be present, or may be empty insert : heap  value  heap findMin : heap  value deleteMin : heap  heap merge : heap  heap  heap insert : heap  value  heap findMin : heap  value deleteMin : heap  heap merge : heap  heap  heap

31. 31 Building a Binomial Tree To construct a binomial tree B k of height k: 1.Take the binomial tree B k-1 of height k-1 2.Place another copy of B k-1 one level below the first 3.Attach the root nodes Binomial tree of height k has exactly 2 k nodes (by induction) B 0 B 1 B 2 B 3

32. 32 Building a Binomial Tree To construct a binomial tree B k of height k: 1.Take the binomial tree B k-1 of height k-1 2.Place another copy of B k-1 one level below the first 3.Attach the root nodes Binomial tree of height k has exactly 2 k nodes (by induction) B 0 B 1 B 2 B 3

33. 33 Building a Binomial Tree To construct a binomial tree B k of height k: 1.Take the binomial tree B k-1 of height k-1 2.Place another copy of B k-1 one level below the first 3.Attach the root nodes Binomial tree of height k has exactly 2 k nodes (by induction) B 0 B 1 B 2 B 3

34. 34 Building a Binomial Tree To construct a binomial tree B k of height k: 1.Take the binomial tree B k-1 of height k-1 2.Place another copy of B k-1 one level below the first 3.Attach the root nodes Binomial tree of height k has exactly 2 k nodes (by induction) B 0 B 1 B 2 B 3

35. 35 Building a Binomial Tree To construct a binomial tree B k of height k: 1.Take the binomial tree B k-1 of height k-1 2.Place another copy of B k-1 one level below the first 3.Attach the root nodes Binomial tree of height k has exactly 2 k nodes (by induction) B 0 B 1 B 2 B 3

36. 36 Building a Binomial Tree To construct a binomial tree B k of height k: 1.Take the binomial tree B k-1 of height k-1 2.Place another copy of B k-1 one level below the first 3.Attach the root nodes Binomial tree of height k has exactly 2 k nodes (by induction) B 0 B 1 B 2 B 3

37. 37 Building a Binomial Tree To construct a binomial tree B k of height k: 1.Take the binomial tree B k-1 of height k-1 2.Place another copy of B k-1 one level below the first 3.Attach the root nodes Binomial tree of height k has exactly 2 k nodes (by induction) B 0 B 1 B 2 B 3 Recall: a binomial heap may have any subset of these trees

38. 38 Why Binomial? Why are these trees called binomial? –Hint: how many nodes at depth d? B 0 B 1 B 2 B 3

39. 39 Why Binomial? Why are these trees called binomial? –Hint: how many nodes at depth d? Number of nodes at different depths d for B k = [1], [1 1], [1 2 1], [1 3 3 1], … Binomial coefficients of (a + b) k : k!/((k-d)!d!) B 0 B 1 B 2 B 3

40. 40 Binomial Queue Properties Suppose you are given a binomial queue of N nodes 1.There is a unique set of binomial trees for N nodes 2.What is the maximum number of trees that can be in an N-node queue? –1 node  1 tree B 0 ; 2 nodes  1 tree B 1 ; 3 nodes  2 trees B 0 and B 1 ; 7 nodes  3 trees B 0, B 1 and B 2 … –Trees B 0, B 1, …, B k can store up to 2 0 + 2 1 + … + 2 k = 2 k+1 – 1 nodes = N. –Maximum is when all trees are used. So, solve for (k+1). –Number of trees is  log(N+1) = O(log N)

41. 41 Definition of Binomial Queues 3 Binomial Queue = “forest” of heap-ordered binomial trees 1 7 2 1 3 8 11 5 6 5 9 6 7 21 B 0 B 2 B 0 B 1 B 3 Binomial queue H 1 5 elements = 101 base 2  B 2 B 0 Binomial queue H 2 11 elements = 1011 base 2  B 3 B 1 B 0

42. 42 findMin() In each B i, the minimum key is at the root So scan sequentially B 1, B 2,..., B k, compute the smallest of their keys: Time: O(log n) (why ?) B 0 B 1 B 2 B 3

43. 43 Binomial Queues: Merge Main Idea: Merge two binomial queues by merging individual binomial trees –Since B k+1 is just two B k ’s attached together, merging trees is easy Steps for creating new queue by merging: 1.Start with B k for smallest k in either queue. 2.If only one B k, add B k to new queue and go to next k. 3.Merge two B k ’s to get new B k+1 by making larger root the child of smaller root. Go to step 2 with k = k + 1.

44. 44 Example: Binomial Queue Merge 3 1 7 2 1 3 8 11 5 6 5 9 6 7 21 H1: H2:

45. 45 Example: Binomial Queue Merge 3 1 7 2 1 3 8 11 5 6 5 9 6 7 21 H1: H2:

46. 46 Example: Binomial Queue Merge 3 1 7 2 1 3 8 11 5 6 5 9 6 7 21 H1: H2:

47. 47 Example: Binomial Queue Merge 3 1 7 2 1 3 8 11 5 6 5 9 6 7 21 H1: H2:

48. 48 Example: Binomial Queue Merge 3 1 7 2 1 3 8 11 5 6 5 9 6 7 21 H1: H2:

49. 49 Example: Binomial Queue Merge 3 1 7 2 1 3 8 11 5 6 5 9 6 7 21 H1: H2:

50. 50 Binomial Queues: Merge and Insert What is the run time for Merge of two O(N) queues? How would you insert a new item into the queue?

51. 51 Binomial Queues: Merge and Insert What is the run time for Merge of two O(N) queues? –O(number of trees) = O(log N) How would you insert a new item into the queue? –Create a single node queue B 0 with new item and merge with existing queue –Again, O(log N) time Example: Insert 1, 2, 3, …,7 into an empty binomial queue

52. 52 Insert 1,2,…,7 1

53. 53 Insert 1,2,…,7 1 2

54. 54 Insert 1,2,…,7 1 2 3

55. 55 Insert 1,2,…,7 1 2 3 4

56. 56 Insert 1,2,…,7 1 2 3 4

57. 57 Insert 1,2,…,7 1 2 3 4 5

58. 58 Insert 1,2,…,7 1 2 3 4 5 6

59. 59 Insert 1,2,…,7 1 2 3 4 5 6 7

60. 60 Binomial Queues: DeleteMin Steps: 1.Find tree B k with the smallest root 2.Remove B k from the queue 3.Delete root of B k (return this value); You now have a new queue made up of the forest B 0, B 1, …, B k-1 4.Merge this queue with remainder of the original (from step 2) Run time analysis: Step 1 is O(log N), step 2 and 3 are O(1), and step 4 is O(log N). Total time = O(log N) Example: Insert 1, 2, …, 7 into empty queue and DeleteMin

61. 61 Insert 1,2,…,7 1 2 3 4 5 6 7

62. 62 DeleteMin 2 3 4 5 6 7

63. 63 Merge 2 3 4 5 6 7

64. 64 Merge 2 3 4 5 6 7

65. 65 Merge 2 3 4 5 6 7

66. 66 Merge 2 3 4 5 6 7 DONE!

67. 67 Implementation of Binomial Queues Need to be able to scan through all trees, and given two binomial queues find trees that are same size –Use array of pointers to root nodes, sorted by size –Since is only of length log(N), don’t have to worry about cost of copying this array –At each node, keep track of the size of the (sub) tree rooted at that node Want to merge by just setting pointers –Need pointer-based implementation of heaps DeleteMin requires fast access to all subtrees of root –Use First-Child/Next-Sibling representation of trees

68. 68 Efficient BuildHeap for Binomial Queues Insert one at a time - O(n log n) Better algorithm: –Start with each element as a singleton tree –Merge trees of size 1 –Merge trees of size 2 –Merge trees of size 4 Complexity: