Priority Queues, Heaps & Leftist Trees CSE, POSTECH

Priority Queues A priority queue is a collection of zero or more elements  each element has a priority or value Unlike the FIFO queues, the order of deletion from a priority queue (e.g., who gets served next) is determined by the element priority Elements are deleted by increasing or decreasing order of priority rather than by the order in which they arrived in the queue

Priority Queues Operations performed on priority queues 1) Find an element, 2) insert a new element, 3) delete an element, etc. Two kinds of (Min, Max) priority queues exist In a Min priority queue, find/delete operation finds/deletes the element with minimum priority In a Max priority queue, find/delete operation finds/deletes the element with maximum priority Two or more elements can have the same priority

Priority Queues See ADT 12.1 & Program 12.1 for max priority queue specification What would be different for min priority queue specification? Read Examples 12.1, 12.2 What are other examples in our daily lives that utilize the priority queue concept?

Implementation of Priority Queues Implemented using heaps and leftist trees Heap is a complete binary tree that is efficiently stored using the array-based representation Leftist tree is a linked data structure suitable for the implementation of a priority queue

Max (Min) Tree A max tree (min tree) is a tree in which the value in each node is greater (less) than or equal to those in its children (if any) See Figure 12.1, 12.2 for examples Nodes of a max or min tree may have more than two children (i.e., may not be binary tree)

Max Tree Example

Min Tree Example

Heaps - Definitions A max heap (min heap) is a max (min) tree that is also a complete binary tree Figure 12.1 (a) & (c) are max heap Figure 12.2 (a) & (c) are min heap Why aren’t Figure 12.1 (b) & 12.2 (b) max/min heap? How can you change the Figure 12.1 (b) & 12.2 (b) so that they are max/min heap? 30 25 14 12 7 10 8 6 9 5

Max Heap with 9 Nodes

Min Heap with 9 Nodes

Array Representation of Heap A heap is efficiently represented as an array.

Heap Operations When n is the number of elements (heap size), Insertion  O(log2n) Deletion  O(log2n) Initialization  O(n)

Insertion into a Max Heap 9 8 6 7 2 5 1 New element is 5 Are we finished?

Insertion into a Max Heap 9 8 6 7 2 5 1 20 New element is 20 Are we finished?

Insertion into a Max Heap 9 8 6 7 2 5 1 20 Exchange the positions with 7 Are we finished?

Insertion into a Max Heap 9 6 7 2 5 1 8 20 Exchange the positions with 8 Are we finished?

Insertion into a Max Heap 6 7 2 5 1 8 9 20 Exchange the positions with 9 Are we finished?

Complexity of Insertion See also Figure 12.3 for another insertion example At each level, we do (1) work Thus the time complexity is O(height) = O(log2n), where n is the heap size

Deletion from a Max Heap 20 6 7 2 5 1 15 8 9 Max element is in the root What happens when we delete an element?

Deletion from a Max Heap 6 7 2 5 1 15 8 9 After the max element is removed. Are we finished?

Deletion from a Max Heap 6 7 2 5 1 15 8 9 Heap with 10 nodes. Reinsert 8 into the heap.

Deletion from a Max Heap 6 7 2 5 1 15 9 8 Reinsert 8 into the heap. Are we finished?

Deletion from a Max Heap 6 7 2 5 1 9 15 8 Exchange the position with 15 Are we finished?

Deletion from a Max Heap 6 7 2 5 1 8 15 9 Exchange the position with 9 Are we finished?

Complexity of Deletion See also Figure 12.4 for another deletion example The time complexity of deletion is the same as insertion At each level, we do (1) work Thus the time complexity is O(height) = O(log2n), where n is the heap size

Max Heap Initialization Heap initialization means to construct a heap by adjusting the tree if necessary Example: input array = [-,1,2,3,4,5,6,7,8,9,10,11]

Max Heap Initialization - Start at rightmost array position that has a child. - Index is floor(n/2).

Max Heap Initialization

Max Heap Initialization

Max Heap Initialization

Max Heap Initialization

Max Heap Initialization Are we finished? Done!

Complexity of Initialization See Figure 12.5 for another initialization example Height of heap = h. Number of nodes at level j is <= 2j-1. Time for each node at level j is O(h-j+1). Time for all nodes at level j is <= 2j-1(h-j+1) = t(j). Total time is t(1) + t(2) + … + t(h) = O(2h) = O(n).

The Class MaxHeap See Program 12.2 for Insertion into a MaxHeap See Program 12.3 for Deletion from a MaxHeap See Program 12.4 for Initializing a nonempty MaxHeap

PUSH OPERATION template<class T> void maxHeap<T>::push(const T& theElement) {// Add theElement to heap. // increase array length if necessary if (heapSize == arrayLength - 1) {// double array length changeLength1D(heap, arrayLength, 2 * arrayLength); arrayLength *= 2; } // find place for theElement // currentNode starts at new leaf and moves up tree int currentNode = ++heapSize; while (currentNode != 1 && heap[currentNode / 2] < theElement) { // cannot put theElement in heap[currentNode] heap[currentNode] = heap[currentNode / 2]; // move element down currentNode /= 2; // move to parent heap[currentNode] = theElement;

POP OPERATION template<class T> void maxHeap<T>::pop() {// Remove max element. // if heap is empty return null if (heapSize == 0) // heap empty throw queueEmpty(); // Delete max element heap[1].~T(); // Remove last element and reheapify T lastElement = heap[heapSize--]; // find place for lastElement starting at root int currentNode = 1, child = 2; // child of currentNode while (child <= heapSize) { // heap[child] should be larger child of currentNode if (child < heapSize && heap[child] < heap[child + 1]) child++; // can we put lastElement in heap[currentNode]? if (lastElement >= heap[child]) break; // yes // no heap[currentNode] = heap[child]; // move child up currentNode = child; // move down a level child *= 2; } heap[currentNode] = lastElement;

INITIALIZE template<class T> void maxHeap<T>::initialize(T *theHeap, int theSize) {// Initialize max heap to element array theHeap[1:theSize]. delete [] heap; heap = theHeap; heapSize = theSize; // heapify for (int root = heapSize / 2; root >= 1; root--) { T rootElement = heap[root]; // find place to put rootElement int child = 2 * root; // parent of child is target // location for rootElement while (child <= heapSize) // heap[child] should be larger sibling if (child < heapSize && heap[child] < heap[child + 1]) child++; // can we put rootElement in heap[child/2]? if (rootElement >= heap[child]) break; // yes // no heap[child / 2] = heap[child]; // move child up child *= 2; // move down a level } heap[child / 2] = rootElement;

Exercise 12.7 Do Exercise 12.7 theHeap = [-, 10, 2, 7, 6, 5, 9, 12, 35, 22, 15, 1, 3, 4]

Exercise 12.7 (a) 12.7 (a) – complete binary tree

Exercise 12.7 (b) 12.7 (b) – The heapified tree

Exercise 12.7 (c) 12.7 (c) – The heap after 15 is inserted is:

Exercise 12.7 (c) 12.7 (c) – The heap after 20 is inserted is:

Exercise 12.7 (c) 12.7 (c) – The heap after 45 is inserted is:

Exercise 12.7 (d) 12.7 (d) – The heap after the first remove max operation is:

Exercise 12.7 (d) 12.7 (d) – The heap after the second remove max operation is:

Exercise 12.7 (d) 12.7 (d) – The heap after the third remove max operation is:

Leftist Trees Despite heap structure being both space and time efficient, it is NOT suitable for all applications of priority queues Leftist tree structures are useful for applications to meld (i.e., combine) pairs of priority queues using multiple queues of varying size Leftist tree is a linked data structure suitable for the implementation of a priority queue A tree which tends to “lean” to the left.

Leftist Trees External node – a special node that replaces each empty subtree Internal node – a node with non-empty subtrees Extended binary tree – a binary tree with external nodes added (see Figure 12.6)

Extended Binary Tree Figure 12.6 s and w values

Height-Biased Leftist Tree (HBLT) Let s(x) be the length (height) of a shortest path from node x to an external node in its subtree If x is an external node, s(x) = 0 If x is an internal node, s(x) = min {s(L), s(R)} + 1, where L and R are left and right children of x A binary tree is a height-biased leftist tree (HBLT) iff at every internal node, the s value of the left child is greater than or equal to the s value of the right child Is Figure 12.6(a) an HBLT? If not, how can we change it to become an HBLT?

Max/Min HBLT A max HBLT is an HBLT that is also a max tree Are the trees of Figure 12.1 are also max HBLTs? YES! A min HBLT is an HBLT that is also a min tree Are the trees of Figure 12.2 are also min HBLTs?

Weight-Biased Leftist Tree (WBLT) Let the weight, w(x), of node x to be the number of internal nodes in the subtree with root x If x is an external node, w(x) = 0 If x is an internal node, its weight is one more than the sum of the weights of its children A binary tree is a weight-biased leftist tree (WBLT) iff at every internal node, the w value of the left child is greater than or equal to the w value of the right child

Weight-Biased Leftist Tree (WBLT) A max (min) WBLT is a max (min) tree that is also a WBLT Is Figure 12.6(a) an WBLT? If not, how can we change it to become an WBLT?

Operations on a Max HBLT Read Section 12.5.2 for Insertion into a Max HBLT Read Section 12.5.3 for Deletion from a Max HBLT Read Section 12.5.4 for Melding Two Max HBLTs See Figure 12.7 and read Example 12.3 for Melding max HBLTs Read Section 12.5.5 for Initialization of a Max HBLT See Figure 12.8 for Initializing a max HBLT

Figure 12.7 Melding (combining) max HBLTs Melding max HBLTs Figure 12.7 Melding (combining) max HBLTs

The Class maxHBLT See Program 12.5 for Melding of two leftist trees See Program 12.6 for meld, push and pop methods See Program 12.7 for Initializing a max HBLT Do Exercise 12.19

Exercise 12.19 (a) The first six calls to meld create the following six max leftist trees. 5 3 7 6 20 8 9 2 15 12 30 17 The next three calls to meld combine pairs of these trees to create the following three trees: 5 3 7 6 9 2 20 8 15 12 30 17 What would be next?

Applications of Heaps Sort (heap sort) Machine scheduling Huffman codes

Heap Sort use element key as priority Algorithm put elements to be sorted into a priority queue (i.e., initialize a heap) extract (delete) elements from the priority queue if a min priority queue is used, elements are extracted in non-decreasing order of priority if a max priority queue is used, elements are extracted in non-increasing order of priority

Heap Sort Example After putting into a max priority queue

Sorting Example After first remove max operation

Sorting Example After second remove max operation

Sorting Example After third remove max operation

Sorting Example After fourth remove max operation

Sorting Example After fifth remove max operation

Complexity Analysis of Heap Sort See Program 12.8 for Heap Sort See Figure 12.9 for another Heap Sort example Heap sort n elements. Initialization operation takes O(n) time Each deletion operation takes O(log n) time Thus, the total time is O(n log n) - Why?  The heap has to be reinitialized (melded) after each delete operation compare with O(n2) for sort methods of Chapter 2

Machine Scheduling Problem m identical machines n jobs to be performed The machine scheduling problem is to assign jobs to machines so that the time at which the last job completes is minimum

Machine Scheduling Example 3 machines and 7 jobs job times are [6,2,3,5,10,7,14] What are some possible schedules? A possible schedule: What algorithm did we use for the above scheduling? What are other scheduling algorithms

Machine Scheduling Example What is the finish time (length) of the schedule?  21 Objective: Find schedules with minimum finish time Minimum finish time scheduling is NP-hard.

NP-hard Problems The class of problems for which no one has developed a polynomial time algorithm. No algorithm whose complexity is O(nk ml) is known for any NP-hard problem (for any constants k and l) NP stands for Nondeterministic Polynomial NP-hard problems are often solved by heuristics (or approximation algorithms), which do not guarantee optimal solutions Longest Processing Time (LPT) rule is a good heuristic for minimum finish time scheduling.

LPT Schedule & Example finish time is 16! Longest Processing Time (LPT) first Jobs are scheduled in the descending order 14, 10, 7, 6, 5, 3, 2 Each job is scheduled on the machine on which it finishes earliest finish time is 16!

LPT Schedule & Example What is the minimum finish time with thee machines for jobs (2, 14, 4, 16, 6, 5, 3)? See Figure 12.10

LPT using a Min Heap Min Heap has the finish times of the m machines. Initial finish times are all 0. To schedule a job, remove the machine with minimum finish time from the heap. Update the finish time of the selected machine and put the machine back into the min heap. See Program 12.9 for LPT scheduler

Complexity Analysis of LPT When n  m (i.e., more machines than jobs), LPT takes (1) time When n  m, (i.e., more jobs than machines), the heap sort takes O(n log n) time Heap initialization takes O(m) time DeleteMin operation takes O(log m) time Insert operation takes O(log m) time n DeleteMin and n Insert takes O(n log m) time Thus, the total time is O(n log n + n log m) = O(n log n) time (as n > m)

Huffman Codes For text compression, the LZW method relies on the recurrence of substrings in a text Huffman codes is another text compression method, which relies on the relative frequency (i.e., the number of occurrences of a symbol) with which different symbols appear in a text Uses extended binary trees Variable-length codes that satisfy the property, where no code is a prefix of another Huffman tree is a binary tree with minimum weighted external path length for a given set of frequencies (weights)

Huffman Codes READ Section 12.6.3 READ all of Chapter 12