1. CS 261 – Data Structures Priority Queues & Heaps

2. Priority Queues Not really a FIFO queue – misnomer!! Associates a “priority” with each object: –First element has the highest priority A data structure that supports the PQueue interface: public void add(Object val); public Object getFirst(); public void removeFirst(); Examples of priority queues: –To do list with priorities –Active processes in an OS

3. Use of Priority Queues in Simulations The original, and one of the most important uses of Priority queues Discrete Event Driven Simulation Actions are represented by “events” - things that have (or will) happen at a given time The PQ maintains list of pending events. Highest priority is next event. Event is pulled from list, executed, usually spawns more events, which are then inserted into the PQ. Loop until everything happens, or until fixed time is reached.

4. Priority Queue Applications Discrete event-driven simulation – i.e., a simulation of a process where events occur Example: Ice cream store –People arrive –People order –People leave Simulation Algorithm: 1.Determine the times of each event using a random number generator with some distribution: 2.Put all events in a priority queue based on when it happens 3.Simulation framework pulls the minimum (next to happen) and executes the event

5. Priority Queue Applications Many types of events … but they all have a time when they occur Can we store various types of events in a priority queue? –Heterogeneous collection –Keep a pointer to action, or a flag to tell what action to perform –So, have your comparison based on the time

6. Priority Queues: Implementations We can definitely do better than these!!! SortedVectorSortedListLinkedList add O(n) Binary search Slide data up O(n) Linear search O(1) addLast(obj) getFirst O(1) elementAt(0) O(1) Returns head.obj O(n) Linear search for smallest value removeFirst O(n) Slide data down O(1)  Reverse Order O(n) head.remove() O(n) Linear search then remove smallest

7. Priority Queues: Heaps Heap: has 2 completely different meanings 1.Classic data structure used to implement priority queues 2.Memory space used for dynamic allocation We will study the data structure (not dynamic memory allocation) Heap data structure: a complete binary tree in which every node’s value is less than or equal to the values of its children Review: a complete binary tree is a tree in which 1.Every node has at most two children (binary) 2.The tree is entirely filled except for the bottom level which is filled from left to right (complete) –Longest path is ceiling(log n) for n nodes

8. Heap: Example 2 5 8 3 7910 14121116 Root = Smallest element Next open spot Last filled position (not necessarily the last object added)

9. Maintaining the Heap: Addition 2 5 8 3 7910 14121116 4 Place new element in next available position, then fix it by “percolating up” Add element: 4 New element in next open spot.

10. Maintaining the Heap: Addition (cont.) Percolating up: while new value is less than parent, swap value with parent 2 5 8 3 4910 141211167 2 4 8 3 5910 141211167 After first iteration (swapped with 7) After second iteration (swapped with 5) New value not less than parent  Done

11. Maintaining the Heap: Removal Since each node’s value is less than or equal to the values of its children, the root is always the smallest element Thus, the PQueue methods getFirst and removeFirst access and remove the root node, respectively Heap removal ( removeFirst ) : 1.Replace root with the element in the last filled position 2.Fix heap by “percolating down”

12. Maintaining the Heap: Removal 2 5 8 3 7910 14121116 Root = Smallest element removeFirst : 1.Move value in last element into root 2.Percolate down Last filled position

13. Maintaining the Heap: Removal (cont.) Percolating down: while greater than smallest child swap with smallest child 16 5 8 3 7910 141211 3 5 8 16 7910 141211 Root object removed (16 copied to root and last node removed) After first iteration (swapped with 3)

14. Maintaining the Heap: Removal (cont.) 3 9 16 9 12 16 After second iteration (moved 9 up) After third iteration (moved 12 up) Reached leaf node  Stop percolating Percolating down: while greater than smallest child swap with smallest child 5 8710 141211 8 3 5 8710 1411

15. Maintaining the Heap: Removal (cont.) 3 5 8 9 71210 141611 Root = New smallest element New last filled position

16. Heap Representation Recall that a complete binary tree can be efficiently represented by an array or a vector: –Children of node at index i are stored at indices –Parent of node at index i is at index 0202 1313 2525 3939 4 10 5757 6868 7 14 8 12 9 11 10 16 2 5 8 3 7910 14121116 2i + 1 and 2i + 2 floor((i - 1) / 2)

17. Heap Implementation: add (cont.) void addElement(DyArray da, EleType val) { int pos = dyArraySize(da); // Get index of next open spot. dyArrayAdd(da, val); // add element at end int up = (pos – 1) / 2; // Get parent index (up the tree).... } Example: add 4 to the heap Prior to addition, size = 11 0202 1313 2525 3939 4 10 5757 6868 7 14 8 12 9 11 10 16 11 4 Next open spot ( pos ) Parent position ( up )

18. A couple of Useful Routines void swap (dyArray * da, int i, in j) { // swap elements at I and j EleType temp = dyArrayGet(da, i); dyArrayPut(da, i, dyArrayGet(da, j)); dyArrayPut(da, j, temp); } int indexSmallest (dyArray * da, int i, int j) { // return index of smallest element if (LT(dyArrayGet(da, i), dyArrayGet(da, j))) return i; return j; }

19. Heap Implementation: add (cont.) public void add(dyArray * da, EleType val) {... // While not at root and new value is less than parent. while ((pos != 0) && (indexSmallest(da, pos, up) == pos)) { swap(da, pos, up); // swap values of parent and child pos = up; up = (pos - 1) / 2; // Move position and compute parent idx.... } 0202 1313 2525 3939 4 10 5454 6868 7 14 8 12 9 11 10 16 11 7 After first iteration: “swapped” new value (4) with parent (7) New parent value: 5 posup

20. Heap Implementation: add (cont.) public void add(dyArray * da, EleType val) {... // While not at root and new value is less than parent. while ((pos != 0) && (indexSmallest(da, pos, up) == pos)) { swap(da, pos, up); // swap values of parent and child pos = up; up = (pos - 1) / 2; // Move position and compute parent idx.... } 0202 1313 2424 3939 4 10 5555 6868 7 14 8 12 9 11 10 16 11 7 After second iteration: “swapped” new value (4) with parent (5) New parent value: 2 posup

21. Heap Implementation: add (cont.) public void add(Dyarray * da, EleType val) {... // While not at root and new value is less than parent. while ((pos != 0) && (indexSmallest(da, pos, up) == pos)) {... } // End while: reached root or parent is smaller than new value. } // Done. 0202 1313 2424 3939 4 10 5555 6868 7 14 8 12 9 11 10 16 11 7 Second part of while test fails: stop iteration and add new value posup

22. Heap Implementation: add (cont.) 2 5 8 3 7910 14121116 4 0202 1313 2525 3939 4 10 5757 6868 7 14 8 12 9 11 10 16 11 4 pos up

23. Heap Implementation: add (cont.) 4 7 5454 11 7 pos up 2 5 8 3 910 14121116 0202 1313 2525 3939 4 10 6868 7 14 8 12 9 11 10 16

24. Heap Implementation: add (cont.) 4 5 2424 5555 pos up 7 11 7 2 8 3 910 14121116 0202 1313 3939 4 10 6868 7 14 8 12 9 11 10 16

25. Heap Implementation: add (cont.) 4 5 2424 5555 pos 7 11 7 2 8 3 910 14121116 0202 1313 3939 4 10 6868 7 14 8 12 9 11 10 16

26. 10 16 Heap Implementation: removeFirst void removeFirst(dyArray * da) { int last = dyArraySize(da) – 1; if (last != 0) // Copy the last element to dyArrayPut(da,0, dyArrayGet(da, last)); // the first position. dyArrayRemoveAt(da, last); // Remove last element. adjustHeap(dyArraySize(da), 0); // Rebuild heap property. } 0202 1313 2424 3939 4 10 5555 6868 7 14 8 12 9 11 Last element ( last ) First element ( elementAt(0) ) 1313 2424 3939 4 10 5555 6868 7 14 8 12 9 11 0707 7 10 16 11

27. Heap Implementation: adjustHeap void adjustHeap(EleType * data, int max, int idx) { val = 7 idxmax child (left child smallest) min = 3 What’s next: val > min  Copy min to parent spot ( idx ) Move idx to child 1313 2424 3939 4 10 5555 6868 7 14 8 12 9 11 0707 10 16 11

28. Heap Implementation: adjustHeap (cont.) 2 4 8 3 5910 14121116 7 last 1313 2424 3939 4 10 5555 6868 7 14 8 12 9 11 0202 7 10 16

29. Heap Implementation: adjustHeap (cont.) 7 4 8 3 5910 14121116 1313 2424 3939 4 10 5555 6868 7 14 8 12 9 11 0707 10 16 New root max

30. Heap Implementation: adjustHeap (cont.) 7 4 8 3 5910 14121116 1313 2424 3939 4 10 5555 6868 7 14 8 12 9 11 0707 10 16 Smallest child ( min = 3 ) val = 7 11 max val > min  Copy min to parent spot ( idx ) Move idx to child

31. Heap Implementation: adjustHeap (cont.) 3 4 8 7 5910 14121116 1717 2424 3939 4 10 5555 6868 7 14 8 12 9 11 0303 10 16 val = 7 11 max val > min  Copy min to parent spot ( idx ) Move idx to child idx

32. Heap Implementation: adjustHeap (cont.) 3 4 8 7 5910 14121116 1717 2424 3939 4 10 5555 6868 7 14 8 12 9 11 0303 10 16 val = 7 11 max If larger than both children Done idx Smallest child ( min = 9 )

33. AdjustHeap - Write recursively void adjustHeap (EleType * data, int max, int pos) { int leftChild = pos * 2 + 1; int rightChild = pos * 2 + 2; if (rightChild < max) { // have two children // get index of smallest // compare smallest child to pos // if necessary, swap and call adjustHeap(max, child) } else if (leftChild < max) { // have only one child // compare child to parent // if necessary, swap and call adjustHeap(max, child) } // else no children, we are at bottom }

34. Priority Queues: Performance Evaluation So, which is the best implementation of a priority queue? SortedVectorSortedListHeap addElement O(n) Binary search Slide data up O(n) Linear search O(log n) Percolate up getFirst O(1) elementAt(0) O(1) Returns head.obj O(1) Get root node removeFirst O(n) Slide data down O(1): Reverse Order O(1) head.remove() O(log n) Percolate down

35. Priority Queues: Performance Evaluation Remember that a priority queue’s main purpose is rapidly accessing and removing the smallest element! Consider a case where you will insert (and ultimately remove) n elements: –ReverseSortedVector and SortedList: Insertions: n * n = n 2 Removals: n * 1 = n Total time: n 2 + n = O(n 2 ) –Heap: Insertions: n * log n Removals: n * log n Total time: n * log n + n * log n = 2n log n = O(n log n)