1. Priority Queues, Heaps & Leftist Trees
CSE, POSTECH

2. Priority Queues
A priority queue is a collection of zero or more elements  each element has a priority or value
Unlike the FIFO queues, the order of deletion from a priority queue (e.g., who gets served next) is determined by the element priority
Elements are deleted by increasing or decreasing order of priority rather than by the order in which they arrived in the queue

3. Priority Queues Operations performed on priority queues
1) Find an element, 2) insert a new element, 3) delete an element, etc.
Two kinds of (Min, Max) priority queues exist
In a Min priority queue, find/delete operation finds/deletes the element with minimum priority
In a Max priority queue, find/delete operation finds/deletes the element with maximum priority
Two or more elements can have the same priority

4. Priority Queues
See ADT 12.1 & Program 12.1 for max priority queue specification
What would be different for min priority queue specification?
Read Examples 12.1, 12.2
What are other examples in our daily lives that utilize the priority queue concept?

5. Implementation of Priority Queues
Implemented using heaps and leftist trees
Heap is a complete binary tree that is efficiently stored using the array-based representation
Leftist tree is a linked data structure suitable for the implementation of a priority queue

6. Max (Min) Tree
A max tree (min tree) is a tree in which the value in each node is greater (less) than or equal to those in its children (if any)
See Figure 12.1, 12.2 for examples
Nodes of a max or min tree may have more than two children (i.e., may not be binary tree)

7. Max Tree Example

8. Min Tree Example

9. Heaps - Definitions
A max heap (min heap) is a max (min) tree that is also a complete binary tree
Figure 12.1 (a) & (c) are max heap
Figure 12.2 (a) & (c) are min heap
Why aren’t Figure 12.1 (b) & 12.2 (b) max/min heap?
How can you change the Figure 12.1 (b) & 12.2 (b) so that they are max/min heap? 30 25 14 12 7 10 8 6 9 5

10. Max Heap with 9 Nodes

11. Min Heap with 9 Nodes

12. Array Representation of Heap
A heap is efficiently represented as an array.

13. Heap Operations When n is the number of elements (heap size),
Insertion  O(log2n)
Deletion  O(log2n)
Initialization  O(n)

14. Insertion into a Max Heap9 8 6 7 2 5 1
New element is 5
Are we finished?

15. Insertion into a Max Heap9 8 6 7 2 5 1 20
New element is 20
Are we finished?

16. Insertion into a Max Heap9 8 6 7 2 5 1 20
Exchange the positions with 7
Are we finished?

17. Insertion into a Max Heap9 6 7 2 5 1 8 20
Exchange the positions with 8
Are we finished?

18. Insertion into a Max Heap6 7 2 5 1 8 9 20
Exchange the positions with 9
Are we finished?

19. Complexity of Insertion
See also Figure 12.3 for another insertion example
At each level, we do (1) work
Thus the time complexity is O(height) = O(log2n), where n is the heap size

20. Deletion from a Max Heap20 6 7 2 5 1 15 8 9
Max element is in the root
What happens when we
delete an element?

21. Deletion from a Max Heap6 7 2 5 1 15 8 9
After the max element is removed.
Are we finished?

22. Deletion from a Max Heap6 7 2 5 1 15 8 9
Heap with 10 nodes.
Reinsert 8 into the heap.

23. Deletion from a Max Heap6 7 2 5 1 15 9 8
Reinsert 8 into the heap.
Are we finished?

24. Deletion from a Max Heap6 7 2 5 1 9 15 8
Exchange the position with 15
Are we finished?

25. Deletion from a Max Heap6 7 2 5 1 8 15 9
Exchange the position with 9
Are we finished?

26. Complexity of Deletion
See also Figure 12.4 for another deletion example
The time complexity of deletion is the same as insertion
At each level, we do (1) work
Thus the time complexity is O(height) = O(log2n), where n is the heap size

27. Max Heap Initialization
Heap initialization means to construct a heap by adjusting the tree if necessary
Example: input array = [-,1,2,3,4,5,6,7,8,9,10,11]

28. Max Heap Initialization
- Start at rightmost array position that has a child.
- Index is floor(n/2).

29. Max Heap Initialization

30. Max Heap Initialization

31. Max Heap Initialization

32. Max Heap Initialization

33. Max Heap Initialization
Are we finished?
Done!

34. Complexity of Initialization
See Figure 12.5 for another initialization example
Height of heap = h.
Number of nodes at level j is <= 2j-1.
Time for each node at level j is O(h-j+1).
Time for all nodes at level j is <= 2j-1(h-j+1) = t(j).
Total time is t(1) + t(2) + … + t(h) = O(2h) = O(n).

35. The Class MaxHeap See Program 12.2 for Insertion into a MaxHeap
See Program 12.3 for Deletion from a MaxHeap
See Program 12.4 for Initializing a nonempty MaxHeap

36. PUSH OPERATION template<class T>
void maxHeap<T>::push(const T& theElement)
{// Add theElement to heap.
// increase array length if necessary
if (heapSize == arrayLength - 1)
{// double array length
changeLength1D(heap, arrayLength, 2 * arrayLength);
arrayLength *= 2; }
// find place for theElement
// currentNode starts at new leaf and moves up tree
int currentNode = ++heapSize;
while (currentNode != 1 && heap[currentNode / 2] < theElement) {
// cannot put theElement in heap[currentNode]
heap[currentNode] = heap[currentNode / 2]; // move element down
currentNode /= 2; // move to parent
heap[currentNode] = theElement;

37. POP OPERATION
template<class T> void maxHeap<T>::pop() {// Remove max element. // if heap is empty return null if (heapSize == 0) // heap empty throw queueEmpty(); // Delete max element heap[1].~T(); // Remove last element and reheapify T lastElement = heap[heapSize--]; // find place for lastElement starting at root int currentNode = 1, child = 2; // child of currentNode while (child <= heapSize) { // heap[child] should be larger child of currentNode if (child < heapSize && heap[child] < heap[child + 1]) child++; // can we put lastElement in heap[currentNode]? if (lastElement >= heap[child]) break; // yes // no heap[currentNode] = heap[child]; // move child up currentNode = child; // move down a level child *= 2; } heap[currentNode] = lastElement;

38. INITIALIZE
template<class T> void maxHeap<T>::initialize(T *theHeap, int theSize) {// Initialize max heap to element array theHeap[1:theSize]. delete [] heap; heap = theHeap; heapSize = theSize; // heapify for (int root = heapSize / 2; root >= 1; root--) { T rootElement = heap[root]; // find place to put rootElement int child = 2 * root; // parent of child is target // location for rootElement while (child <= heapSize) // heap[child] should be larger sibling if (child < heapSize && heap[child] < heap[child + 1]) child++; // can we put rootElement in heap[child/2]? if (rootElement >= heap[child]) break; // yes // no heap[child / 2] = heap[child]; // move child up child *= 2; // move down a level } heap[child / 2] = rootElement;

39. Exercise 12.7
Do Exercise 12.7
theHeap = [-, 10, 2, 7, 6, 5, 9, 12, 35, 22, 15, 1, 3, 4]

40. Exercise 12.7 (a)
12.7 (a) – complete binary tree

41. Exercise 12.7 (b)
12.7 (b) – The heapified tree

42. Exercise 12.7 (c)
12.7 (c) – The heap after 15 is inserted is:

43. Exercise 12.7 (c)
12.7 (c) – The heap after 20 is inserted is:

44. Exercise 12.7 (c)
12.7 (c) – The heap after 45 is inserted is:

45. Exercise 12.7 (d)
12.7 (d) – The heap after the first remove max operation is:

46. Exercise 12.7 (d)
12.7 (d) – The heap after the second remove max operation is:

47. Exercise 12.7 (d)
12.7 (d) – The heap after the third remove max operation is:

48. Leftist Trees
Despite heap structure being both space and time efficient, it is NOT suitable for all applications of priority queues
Leftist tree structures are useful for applications
to meld (i.e., combine) pairs of priority queues
using multiple queues of varying size
Leftist tree is a linked data structure suitable for the implementation of a priority queue
A tree which tends to “lean” to the left.

49. Leftist Trees
External node – a special node that replaces each empty subtree
Internal node – a node with non-empty subtrees
Extended binary tree – a binary tree with external nodes added (see Figure 12.6)

50. Extended Binary Tree
Figure 12.6 s and w values

51. Height-Biased Leftist Tree (HBLT)
Let s(x) be the length (height) of a shortest path from node x to an external node in its subtree
If x is an external node, s(x) = 0
If x is an internal node, s(x) = min {s(L), s(R)} + 1, where L and R are left and right children of x
A binary tree is a height-biased leftist tree (HBLT) iff at every internal node, the s value of the left child is greater than or equal to the s value of the right child
Is Figure 12.6(a) an HBLT?
If not, how can we change it to become an HBLT?

52. Max/Min HBLT A max HBLT is an HBLT that is also a max tree
Are the trees of Figure 12.1 are also max HBLTs?
YES!
A min HBLT is an HBLT that is also a min tree
Are the trees of Figure 12.2 are also min HBLTs?

53. Weight-Biased Leftist Tree (WBLT)
Let the weight, w(x), of node x to be the number of internal nodes in the subtree with root x
If x is an external node, w(x) = 0
If x is an internal node, its weight is one more than the sum of the weights of its children
A binary tree is a weight-biased leftist tree (WBLT) iff at every internal node, the w value of the left child is greater than or equal to the w value of the right child

54. Weight-Biased Leftist Tree (WBLT)
A max (min) WBLT is a max (min) tree that is also a WBLT
Is Figure 12.6(a) an WBLT?
If not, how can we change it to become an WBLT?

55. Operations on a Max HBLT
Read Section for Insertion into a Max HBLT
Read Section for Deletion from a Max HBLT
Read Section for Melding Two Max HBLTs
See Figure 12.7 and read Example 12.3 for Melding max HBLTs
Read Section for Initialization of a Max HBLT
See Figure 12.8 for Initializing a max HBLT

56. Figure 12.7 Melding (combining) max HBLTs
Melding max HBLTs
Figure 12.7 Melding (combining) max HBLTs

57. The Class maxHBLT See Program 12.5 for Melding of two leftist trees
See Program 12.6 for meld, push and pop methods
See Program 12.7 for Initializing a max HBLT
Do Exercise 12.19

58. Exercise 12.19
(a) The first six calls to meld create the following six max leftist trees. 5 3 7 6 20 8 9 2 15 12 30 17
The next three calls to meld combine pairs of these trees to create the following three trees: 5 3 7 6 9 2 20 8 15 12 30 17
What would be next?

59. Applications of Heaps Sort (heap sort) Machine scheduling
Huffman codes

60. Heap Sort use element key as priority Algorithm
put elements to be sorted into a priority queue (i.e., initialize a heap)
extract (delete) elements from the priority queue
if a min priority queue is used, elements are extracted in non-decreasing order of priority
if a max priority queue is used, elements are extracted in non-increasing order of priority

61. Heap Sort Example
After putting into a max priority queue

62. Sorting Example
After first remove max operation

63. Sorting Example
After second remove max operation

64. Sorting Example
After third remove max operation

65. Sorting Example
After fourth remove max operation

66. Sorting Example
After fifth remove max operation

67. Complexity Analysis of Heap Sort
See Program 12.8 for Heap Sort
See Figure 12.9 for another Heap Sort example
Heap sort n elements.
Initialization operation takes O(n) time
Each deletion operation takes O(log n) time
Thus, the total time is O(n log n) - Why?
 The heap has to be reinitialized (melded) after each delete operation
compare with O(n2) for sort methods of Chapter 2

68. Machine Scheduling Problem
m identical machines
n jobs to be performed
The machine scheduling problem is to assign jobs to machines so that the time at which the last job completes is minimum

69. Machine Scheduling Example
3 machines and 7 jobs
job times are [6,2,3,5,10,7,14]
What are some possible schedules?
A possible schedule:
What algorithm did we use for the above scheduling?
What are other scheduling algorithms

70. Machine Scheduling Example
What is the finish time (length) of the schedule?
 21
Objective: Find schedules with minimum finish time
Minimum finish time scheduling is NP-hard.

71. NP-hard Problems
The class of problems for which no one has developed a polynomial time algorithm.
No algorithm whose complexity is O(nk ml) is known for any NP-hard problem (for any constants k and l)
NP stands for Nondeterministic Polynomial
NP-hard problems are often solved by heuristics (or approximation algorithms), which do not guarantee optimal solutions
Longest Processing Time (LPT) rule is a good heuristic for minimum finish time scheduling.

72. LPT Schedule & Example finish time is 16!
Longest Processing Time (LPT) first
Jobs are scheduled in the descending order 14, 10, 7, 6, 5, 3, 2
Each job is scheduled on the machine on which it finishes earliest
finish time is 16!

73. LPT Schedule & Example
What is the minimum finish time with thee machines for jobs (2, 14, 4, 16, 6, 5, 3)?
See Figure 12.10

74. LPT using a Min Heap Min Heap has the finish times of the m machines.
Initial finish times are all 0.
To schedule a job, remove the machine with minimum finish time from the heap.
Update the finish time of the selected machine and put the machine back into the min heap.
See Program 12.9 for LPT scheduler

75. Complexity Analysis of LPT
When n  m (i.e., more machines than jobs), LPT takes (1) time
When n  m, (i.e., more jobs than machines), the heap sort takes O(n log n) time
Heap initialization takes O(m) time
DeleteMin operation takes O(log m) time
Insert operation takes O(log m) time
n DeleteMin and n Insert takes O(n log m) time
Thus, the total time is O(n log n + n log m) = O(n log n) time (as n > m)

76. Huffman Codes
For text compression, the LZW method relies on the recurrence of substrings in a text
Huffman codes is another text compression method, which relies on the relative frequency (i.e., the number of occurrences of a symbol) with which different symbols appear in a text
Uses extended binary trees
Variable-length codes that satisfy the property, where no code is a prefix of another
Huffman tree is a binary tree with minimum weighted external path length for a given set of frequencies (weights)

77. Huffman Codes
READ Section
READ all of Chapter 12