Outline Priority Queues Binary Heaps Randomized Mergeable Heaps

Priority Queues A priority queue stores a collection of Comparable elements Operations: add(x) : Add x to the collection findMin() : Return the smallest element in the collection deleteMin() : Remove the smallest element in the collection In the JCF Queue interface, these are: add(x)/offer(x) : add(x) peek() : findMind() remove()/poll() : deleteMin()

Priority Queues: Applications We can sort a set of elements using a priority queue. How? public static void sort (T[] a) { Queue pq = new PriorityQueue (); for (T x : a) pq. add(x); for (int i = 0; i < a. length ; i++) a[i] = pq. remove (); } We just need to insert them all into a priority queue and then removing them

Priority Queues: Applications In simulations, priority queues store events ordered by time of occurrence

Priority Queues: Applications In simulations, priority queues store events ordered by time of occurrence

Priority Queues: Applications In simulations, priority queues store events ordered by time of occurrence

Priority Queues: Applications In simulations, priority queues store events ordered by time of occurrence

Priority Queues: Applications In simulations, priority queues store events ordered by time of occurrence

Priority Queues Which structure should we use to implement a priority queue? Priority queues can be implemented as (binary) heaps A binary heap is a binary tree where each node u stores a value u.prio Heap property: u.prio < u.left.prio and u.prio < u.right.prio

Priority Queues Heap property: u.prio < u.left.prio and u.prio < u.right.prio A binary heap is a binary tree where each node u stores a value u.prio

Priority Queues A complete binary tree of height d has up to 2 d nodes To store n nodes, we require 2 d ≥ n A complete heap uses a complete binary tree: d+1 ≥ n + 1 d+1 ≥ log 2 (n + 1) d ≥ log 2 (n + 1) – 1 = O(log n) A complete heap of size n has height O(log n)

Priority Queues How can we maps the nodes of a complete binary tree to the positions of an array? Eytzinger method: a[0] is the root the left child of a[i] is a[2*i+1] the right child of a[i] is a[2*i+2] the parent of a[i] is a[(i-1)/2]

Eytzinger method Eytzinger method:

Eytzinger method Eytzinger method: root Left child Right child a[2*0+1] = a[1] a[2*0+2]= a[2] i = 0

Eytzinger method Eytzinger method: l cde fghijk ab nm o Left child Right child a[2*5+1] = a[11] a[2*0+2]= a[12] i = 5

Implicit Binary Heap An implicit binary heap represents a complete binary heap in an array a using the Eytzinger method No extra pointers, all the data lives in a

Implicit Binary Heap public class BinaryHeap > extends AbstractQueue { protected T[] a; protected int n; protected int left (int i) { return 2*i + 1; } protected int right (int i) { return 2*i + 2; } protected int parent (int i) { return (i -1)/2; }... }

Finding the minimum public T peek () { return a [0]; } Finding the minimum value in a heap is easy It's stored at the root This is a[0] Runs in O(1) time How to find the minimum value in a heap?

Inserting elements To insert x into a (implicit binary) heap, we 1. add x as a leaf 2. while x is smaller than parent(x) swap x with its parent Runs in O(log n) time How to insert an element into a heap?

add x as a leaf 0.2 Inserting elements

add x as a leaf 2. while x is smaller than parent(x) swap x with its parent 0.2

Inserting elements add x as a leaf 2. while x is smaller than parent(x) swap x with its parent 0.2

Inserting elements public boolean offer (T x) { if (n + 1 > a. length ) grow (); a[n++] = x; bubbleUp (n -1); return true ; } protected void bubbleUp (int i) { int p = parent (i); while (i > 0 && a[i]. compareTo (a[p]) < 0) { swap (i,p); i = p; p = parent (i); }

Removing the minimum To delete the minimum from a (implicit binary) heap, we 1. copy x=a[n-1] into a[0] 2. while x is larger than either of its children swap x with the smaller of its children Runs in O(log n) time How can we remove the minimum value from a heap?

copy x=a[n-1] into a[0] Removing the minimum

while x is larger than either of its children swap x with the smaller of its children copy x=a[n-1] into a[0]

Removing the minimum while x is larger than either of its children swap x with the smaller of its children copy x=a[n-1] into a[0]

public T poll () { T x = a [0]; swap (0, --n); trickleDown (0); return x; } Removing the minimum

protected void trickleDown (int i) { do { int j = -1; int r = right (i); if (r < n && a[r]. compareTo (a[i]) < 0) { int l = left (i); if (a[l]. compareTo (a[r]) < 0) { j = l; } else { j = r; } } else { int l = left (i); if (l < n && a[l]. compareTo (a[i]) < 0) { j = l; } } if (j >= 0) swap (i, j); i = j; } while (i >= 0); }

Summary Theorem : A (implicit) binary heap supports the operations add(x) and deleteMin() in O(log n) (amortized) time and supports the findMin() operation in O(1) time.

Building a heap We can insert elements a[0],...,a[n-1] one at a time. This takes O(1 + log(1)) + O(1 + log(2)) + O(1 + log n) Runs in O(nlog n) time How can we build a heap from an unsorted array a? How quickly can we make a into an implicit binary heap? public BinaryHeap (T[] ia) { a = ia; for (int n = 1; n < a. length ; n++) { add(a[n]); } } Can we do better?

Building a heap First build n/2 heaps of size 1 We can do it in O(n) time. How? By working bottom up. public BinaryHeap (T[] ia) { a = ia; n = a. length ; for (int i = n/2; i >= 0; i --) { trickleDown (i); } } Next build n/4 heaps of size 3 Next build n/8 heaps of size 7 build 1 heap of size n …

Building a heap in linear time (analysis) We call trickleDown(i), n/2 j times where j is the root of a heap of size 2 j - 1 (n/4) * O(1) trickleDown(i) then takes O(log(2 j - 1)) = O(j) time Total running time is (n/8) * O(2) (n/16) * O(3) 1 * O(log n) = O(n) …

Heapsort The heapsort algorithm for sorting an array a of length n: Make a into a heap in (O(n) time) Repeat n times Delete the minimum Each deletion takes O(log n) time Runs in O(n log n) time Doesn't require any extra space ( does all work inside of input array a) public T deleteMin () { swap (0, --n); trickleDown (0); }

Merging two heaps We know how to add one element to a heap. What can we do if we want to add another heap? In other words, how can we merge 2 heaps? public Node merge(Node h1, Node h2){ while(h2.size()>0){ T u = h2.peek(); h2.deleteMin(); h1.add(u); } The cost for deleteMin() and add() is O(log n) We perform those operations n times Total cost O(n log n)

Merging two heaps Can we do better? Actually we can do it in O(log n). How? merge(h1,h2) can be defined recursively 1. If either of h1 or h2 is nil, we just return h2 or h1. 2. We take the one with the biggest minimum value and merge it either with the right or left child of the other. 3. We continue the process recursively.

Merging two heaps What is the complexity of this operation? How can we reduce it? Using a heap ordered binary tree structure we can reduce the cost to O(log n). Why? in an implicit binary heap it would be O(n) we need to copy all the elements in a new array

Merging two heaps Node merge(Node h1, Node h2) { if (h1 == nil) return h2; if (h2 == nil) return h1; if (compare(h2.x, h1.x) < 0) return merge(h2, h1); // now we know h1.x <= h2.x if (rand.nextBoolean()) { h1.left = merge(h1.left, h2); h1.left.parent = h1; } else { h1.right = merge(h1.right, h2); h1.right.parent = h1; } return h1; }

Merging two heaps Why we need the random function to merge the heaps? How can we define the add() and remove() operations in a heap ordered binary tree ? public boolean add(T x) { Node u = newNode(); u.x = x; r = merge(u, r); r.parent = nil; n++; return true; } public T remove() { T x = r.x; r = merge(r.left, r.right); if (r != nil) r.parent = nil; n--; return x; }

Analysis of merge(h1,h2) Lemma: The expected length of a random walk in a binary tree with n nodes is at most log(n+1) proof by induction: For n=0 is true log(0+1)=0 Suppose it is true for every n 0 < n n 1 the size of the left sub-tree and n 2 = n - n 1 -1 E[W] = 1 + log(n 1 +1)/2 + log(n 2 +1)/2 E[W] ≤ 1 + log( (n-1)/2 + 1) = 1 + log( (n + 1)/2 ) = log(n + 1)

Summary Lemma: The expected running time of merge(h1,h2) is at most O(log n), where n = h1.size() + h2.size() Theorem: A MeldableHeap implements the (priority) Queue interface and supports the operations add(x), remove() and merge(h1,h2) in O(log n), expected time per operation.

quicksort(S): Recall Quicksort p=5 S<S< S<S< S=S= S=S= S>S> S>S> – Pick a random element p from S – Compare every element to p to get 3 sets S < = {x ϵ S : x < p} S = = {x ϵ S : x = p} S > = {x ϵ S : x > p} – quicksort(S < ) – quicksort(S > )

Quicksort p=5 < < > > i i j j j j i i i i j j i i j j i = j

Quicksort

Recall Heapsort The heapsort algorithm for sorting an array a of length n: Make a into a heap in (O(n) time) Repeat n times Delete the minimum

Recall Heapsort

Mergesort The mergesort algorithm for sorting an array a of length n: Divide a in 2 parts a 1 and a 2 Recursively, sort a 1 and a 2 with mergesort Merge a 1 and a

Merging process i0i0 i0i0 i1i1 i1i1 i 0 < i 1 i 1 < i 0 i 0 ++ i

Analysis of Mergesort The merging process ( step 3 ) costs (O(n) time) The mergesort algorithm for sorting an array a of length n: 1. Divide a in 2 parts a 1 and a 2 2. Recursively, sort a 1 and a 2 with mergesort 3. Merge a 1 and a 2 It is performed log n times. Total costs (O(n log n) time) Theorem: The Mergesort algorithm can sort an array of n items in O(n log n) time

Sorting Algorithms So far, we have seen 3 sorting algorithms: Quicksort: O(n log n) expected time Heapsort: O(n log n) time Mergesort: O(n log n) time Is there a faster sorting algorithm (maybe O(n) time)? Answer: No and yes

Comparision based Sorting Algorithms The algorithm Quicksort, Heapsort and Mergesort are sort by comparing the elements. Every comparison-based sorting algorithm takes (n log n) time for some input A comparison tree is a full binary tree: each internal node u is labelled with a pair u.i and u.j each leaf is labelled with a permutation of { 0,..., n – 1}

Comparision based Sorting Algorithms a[0](>/<)a[1] a[1](>/<)a[2]a[0](>/<)a[2] a[0]<a[1]<a[2]a[0](>/<)a[2] a[1]<a[0]<a[2]a[1](>/<)a[2] a[0]<a[2]<a[1]a[2]<a[0]<a[1] a[1]<a[2]<a[0]a[2]<a[1]<a[0] < < < < < < < < < < > > > > > > > > > >

Comparision based Sorting Algorithms Lemma: Every comparison tree that sorts any input of length n has at least n! Leaves Theorem: Every comparison tree that sorts any input of length n has height at least (n/2)log 2 (n/2) The height is log 2 (n!). n!=O(n n ). log 2 (n n ) = n log 2 (n). The height is O( n log n ).

Sorting in O(n) time In-class problem: Design an algorithm that takes an array a of n integers in the range {0,..., k-1} and sorts them in O(n + k) time

Counting sort int [] countingSort (int [] a, int k) { int c[] = new int [k]; for (int i = 0; i < a. length ; i ++) c[a[i ]]++; for (int i = 1; i < k; i ++) c[i] += c[i -1]; int b[] = new int [a. length ]; for (int i = a.length -1; i >= 0; i --) b[--c[a[i ]]] = a[i]; return b; }

Sorting in O(n) time 1. First count the repetition of any possible number. 2. Compute the starting position of the number 3. Copy the numbers on the output array

3 Recall Heapsort

Radix sort Radix-sort uses the counting sort algorithm to sort integers one “digit" at a time integers have w bits “digit" has d bits uses w/d passes of counting-sort Starts by sorting least-significant digits first works up to most significant digits Correctness depends on fact that counting sort is stable if a[i] = a[j] and i < j then a[i] appears before a[j] in the output

Radix sort Theorem: The radix-sort algorithm can sort an array a of n w-bit integers in O(n + 2 d ) time Theorem: The radix-sort algorithm can sort an array a of n integers in the range {0,..., n c – 1} in O(cn) time.

Summary Comparison based sorting algorithms: Quicksort: O(n log n) expected time Heapsort: O(n log n) time Mergesort: O(n log n) time counting-sort: can sort an array a of n integers in the range {0,..., k-1} in O(n + k) time radix-sort can sort an array a of n integers in the range {0,..., n c – 1} in O(cn) time. No comparison based sorting algorithms: