Chapter 19: Fibonacci Heap Many of the slides are from Prof. Leong Hon Wai’s resources at National University of Singapore

Priority Queue ADT  Priority Queue is an ADT for maintaining a set S of elements, each with a key value and supports the following operations:  I NSERT (S, x) inserts element x into S (also write as S  S  {x}  M INIMUM (S) returns element in S with min key  E XTRACT -M IN (S) removes and returns element in S with min key  D ECREASE -K EY (S, x, k) decreases the value of element x’s key to a new (smaller) value k.

PQ implementations… 1964Binary HeapJ. W. J. Williams 1972Leftist HeapC. A. Crane 1978Binomial HeapJ. Vuillemin 1984Fibonacci Heap M. L. Fredman, R. E. Tarjan 1985Skew Heap D. D. Sleator R. E. Tarjan 1988Relaxed Heap Driscoll, Gabow Shrairman, Tarjan  Many data structures proposed for PQ

Different Priority Queues Data StrI NSERT M IN Extract -M IN D-K EY D ELETE Union Binary HO(lg n)O(1)O(lg n) O(n) Binomial HO(lg n) FibonacciO(1) O(lg n)O(1)O(lg n)O(1)  Time Bounds for different PQ implementations  n is the number of items in the PQ

Binary Min-Heap (as in Heapsort)  Binary min-heap is an array A[1..n] that can be viewed as a nearly complete binary tree  Number the nodes using level order traversal  L EFT (i) = 2i and R IGHT (i) = 2i+1 and  P ARENT (i) =  i/2   Height of tree  lg n  Heap Property: (Each node ≥ its parent node)  A[P ARENT (i)]  A[i]

MinHeap – Two views A[1..10] Left (i) = 2i Right (i) = 2i + 1 Parent (i) =  i/2  Array index ::= level order traversal Heap Property: for all i A[Parent(i)]  A[i]

MinHeap – Insert operation (1) A[1..10] H EAP -I NSERT (A, key): insert at end of array S IFT -U P to restore HP Example: H EAP -I NSERT (A, 5) Restore heap property with S IFT -U P

MinHeap – Insert operation (2) A[1..10] H EAP -I NSERT (A, key): insert at end of array S IFT -U P to restore HP Restore heap property with S IFT -U P Example: H EAP -I NSERT (A, 5)

MinHeap – Insert operation (3) A[1..10] H EAP -I NSERT (A, key): insert at end of array S IFT -U P to restore HP Restore heap property with S IFT -U P Example: H EAP -I NSERT (A, 5)

MinHeap – Insert operation (4) A[1..10] H EAP -I NSERT (A, key): insert at end of array S IFT -U P to restore HP Done! Example: H EAP -I NSERT (A, 5)

MinHeap – Insert operation (5) H EAP -I NSERT (A, key) ⊳ A is a heap (array) heap-size(A)  heap-size(A) + 1 A[heap-size(A)]  key S IFT -U P (A, heap-size(A)) S IFT -U P (A, n) key  A[n] i  n while i > 1 and A[P ARENT (i)] > key do A[i]  A[P ARENT (i)] i  P ARENT (i) A[i]  key T HEAP-INSERT (n) = T SIFT-UP (n) +  (1) = O(lg n) T SIFT-UP (n) = O(h) = O(lg n) (h comparisons, h data-moves)

A[1] is the minimum. H EAP -E XTRACT -M IN (A): H EAP -E XTRACT -M IN (A): Exchange A[1] and A[n] S IFT -D OWN to restore HP MinHeap – Extract-Min operation (1) A[1..10] A[1] is the minimum Exchange A[1] with A[n]

H EAP -E XTRACT -M IN (A): Exchange A[1] and A[n] S IFT -D OWN to restore HP MinHeap – Extract-Min operation (2) A[1..10] S IFT -D OWN to restore heap property return min 4 4

H EAP -E XTRACT -M IN (A): Exchange A[1] and A[n] S IFT -D OWN to restore HP MinHeap – Extract-Min operation (3) A[1..10] S IFT -D OWN to restore heap property

H EAP -E XTRACT -M IN (A): Exchange A[1] and A[n] S IFT -D OWN to restore HP MinHeap – Extract-Min operation (4) A[1..10] Done!

MinHeap – Extract-Min operation (5) H EAP -E XTRACT- M IN (A, key) ⊳ A is a heap (array) min  A [1] A[1]  A[heap-size(A)] heap-size(A)  heap-size(A)  1 S IFT -D OWN (A, 1) S IFT -D OWN (A, i) while i is not a leaf do let j be the index of min child of i if A[i] > A[j] then exchange A[i]  A[j] i  j else i  heap-size(A)+1 ⊳ make i a leaf T SIFT-DOWN (n) = O(h) = O(lg n) (2h comparisons, h data-moves) T HEAP-EXTRACT-MIN (n) = O(lg n)

MinHeap – Decrease-Key (1) A[1..10] H EAP -D ECREASE- K EY (A, i, key): update the key of A[i] S IFT -U P to restore HP H EAP -D ECREASE -K EY (A,9,3)

MinHeap – Decrease-Key (2) A[1..10] H EAP -D ECREASE- K EY (A, i, key): update the key of A[i] S IFT -U P to restore HP Restore heap property with S IFT -U P H EAP -D ECREASE -K EY (A,9,3)

MinHeap – Decrease-Key (3) A[1..10] H EAP -D ECREASE- K EY (A, i, key): update the key of A[i] S IFT -U P to restore HP H EAP -D ECREASE -K EY (A,9,3)

MinHeap – Decrease-Key (4) A[1..10] H EAP -D ECREASE- K EY (A, i, key): update the key of A[i] S IFT -U P to restore HP H EAP -D ECREASE -K EY (A,9,3)

MinHeap – Decrease-Key (5) A[1..10] H EAP -D ECREASE- K EY (A, i, key): update the key of A[i] S IFT -U P to restore HP DONE ! H EAP -D ECREASE -K EY (A,9,3)

MinHeap – Insert operation (6) H EAP -D ECREASE- K EY (A, i, key) ⊳ Update A[i] to key A[i]  key S IFT -U P (A, i) T DECREASE-KEY (n) = T SIFT-UP (n) +  (1) = O(lg n) (h comparisons, h data-moves)

Binary Heap (Summary)  H EAP -M INIMUM (A)O(1)  return root  H EAP -I NSERT (A, x) O(lg n)  append to end, sift-up  H EAP -E XTRACT -M IN (A)O(lg n)  Replace root, sift-down  H EAP -D ECREASE -K EY (A, i, k)O(lg n)  update key, sift-up  What about  H EAP -D ELETE (A, i)  H EAP -M ELD (A, B)

How about Union?  A mergeable heap is any data structure that supports the basic heap operation plus union  Union (H1, H2) creates and returns a new heap

Other Heaps

Binomial Trees B0B0 B1B1 B2B2 B3B3 B4B4 Some examples: Recursive definition: B0B0 Bk1Bk1 Bk1Bk1 BkBk B1B1 Bk1Bk1 BkBk B2B2 B0B0

Properties of Binomial Trees B0B0 B1B1 B2B2 B3B3 B4B4  For a Binomial Tree B k (of order k) 1. there are 2 k nodes, 2. the height of the tree is k, 3. root has degree k and 4. deleting the root gives binomial trees B 0, B 1, …, B k  1 Proof: (DIY, by induction) depth

Defining Property of Binomial Trees B4B4 depth Nodes There are exactly nodes at depth i, (0  i  k)

Binomial Heap (Vuillemin, 1978) B3B3 B0B0 B1B head[H]  A sequence of binomial trees that satisfy  binomial heap property (each tree B k is a min-heap)  0 or 1 binomial tree B k of order k,  There are at most  lg n  + 1 binomial trees.  Eg: A binomial heap H with n = 11 nodes. 11 = (1011) 2

Representing Binomial Heaps (1)  Each node x stores  key[x]  degree[x] p[x]p[x]  child[x]  sibling[x]  (3 pointers per node) 101 parent sibling child key degree

Representing Binomial Heap (2) head[H] Each node x has p[x] child[x] sibling[x] degree[x], key[x]

The Binomial Heap: It’s actual implementation

Operations on a Binomial Heap  M AKE -B INOMIAL -H EAP (H)  Allocate object H, make head[H] = NIL.  (1).  B INOMIAL -H EAP -M INIMUM (H)  Search the root list for minimum. O(lg n). B3B3 B0B0 B1B head[H]

Linking Step: Fundamental Op  B INOMIAL -L INK (y, z) Bk1Bk1 y Bk1Bk1 z Bk1Bk1 y Bk1Bk1 z Assume z  y Linking step B INOMIAL -L INK (y, z) ⊳ Assume z  y p[y]  z sibling[y]  child[z] child[z]  y degree[z]  degree[z] + 1 Constant time O(1)

Binomial Heap Union =

Binomial Heap Union

Binomial Heap Union

Binomial Heap Union  M AKE -B INOMIAL -H EAP -U NION (H 1, H 2 ) :  Create a heap H that is the union of two heaps H 1 and H 2  Analogous to binary addition of n 1 and n 2  Running time.: O(log n) [n = n 1 + n 2 ]  Prop to # of trees in root lists  2(  log 2 n  + 1) = 26

More Operations (1)  B INOMIAL -H EAP -I NSERT (H, x)  Create a one-item (x) binomial heap H 1 and then union H and H 1. O(lg n).  B INOMIAL -H EAP -E XTRACT -M IN (H)  Find minimum, remove root, then union. O(lg n). B3B3 B0B0 B1B head[H]

More Operations (2)  B INOMIAL -H EAP -D ECREASE (H, x, k)  B INOMIAL -H EAP -D ELETE (H, x) B3B3 B0B0 B1B head[H]

Binomial Heaps (Summary)  M INIMUM (H)O(lg n)  U NION (H 1, H 2 )O(lg n)  I NSERT (H, x)O(lg n)  E XTRACT -M IN (H)O(lg n)  D ECREASE- K EY (H, x, k)O(lg n)  D ELETE (H, x)O(lg n)

Binomial  Fibonacci Heaps  Key advantages of Binomial Heap  Elegant and fast – O(lg n)  But, it takes O(lg n) to “maintain” the elegant structure all the time.  Fibonacci Heaps  Want fast I NSERT, D ECREASE -K EY  Be lazy… u Do as little as possible, until there is no choice.  Relax the structure slightly

Fibonacci Heap (Fredman & Tarjan)  Modified from Binomial Heaps  With following relaxations…  Set of heap-ordered trees  Any number of trees of each rank  Trees are doubly-linked, but not ordered H min[H] marked nodes

Representing Fibonacci Heaps (1)  Each node x stores  key[x]  degree[x] (also rank(x))  mark[x] p[x]p[x]  child[x]  left[x]  right [x] (4 pointers per node) parent right child key, degree mark left 10 1 FALSE

Representing Fibonacci Heaps (2) H min[H] min[H]

Properties and Potential Function H min[H]  Properties of F IB -H EAP H:  Mark(x) = TRUE (marked, black in my notes), FALSE (yellow in notes)  n[H] = number of nodes in H  t(H) = number of trees in H  m(H) = number of marked notes in H n[H] = 14 t(H) = 5 m(H) = 3 Potential Function for Amortized Analysis  (H) = t(H) + 2 m(H).

Operations on a Fibonacci Heap  M AKE -F IB -H EAP (H)  Allocate object H, make min[H] = NIL.  (1).  F IB -H EAP -M INIMUM (H)  min[H] points to minimum element. O(1) H min[H] n[H] = 14 t(H) = 5 m(H) = 3

Lazy Operation: I NSERT (1)  F IB -H EAP -I NSERT (H, x)  Be lazy: Create singleton tree with node x H min[H] 21 Eg: Insert 21 n[H] = 14 t(H) = 5 m(H) = 3

Lazy Operation: I NSERT (2)  F IB -H EAP -I NSERT (H, x)  Be lazy: Create singleton tree with node x  Concatenate x with root list of H  Update min[H], n[H] H min[H] 21 Eg: Insert 21 n[H] = 14 t(H) = 5 m(H) = 3 n[H] = 15 t(H) = 6 m(H) = 3

Pseudocode for I NSERT F IB- H EAP -I NSERT (H, x) degree[x]  0 p[x]  NIL child[x]  NIL left[x]  x right[x]  x mark[x]  FALSE concatenate the root list containing x with root list H if min[H] = NIL or key[x] < key[min[H]] then min[H]  x n[H]  n[H] + 1 Worst case time = O(1) Amortized time = O(1) Amortized Time of I NSERT : c’(H) = c(H) +  (H’)   (H) = O(1) + 1 = O(1) O(1)

Lazy Operations: M ELD  F IB -H EAP -M ELD (H 1, H 2 )  Concatenate the two root lists  Update min[H], n[H] n[H 1 ] = 7 t(H 1 ) = 3 m(H 1 ) = min[H 2 ] H2H H1H1 min[H 1 ] n[H 2 ] = 7 t(H 2 ) = 2 m(H 2 ) = 2

Lazy Operations: M ELD  F IB -H EAP -M ELD (H 1, H 2 )  Concatenate the two root lists  Update min[H], n[H] H min[H] n[H] = 14 t(H) = 5 m(H) = 3

What have we got now? With lazy I NSERT and M ELD, We have a doubly-linked list, with pointer to minimum… ( So, when do we need to do real work? )

Linking Step: Fundamental Op  F IB- H EAP -L INK (H, y, x) Linking step F IB- H EAP -L INK (H, y, x) ⊳ Assume x  y remove y from the root list of H make y a child of x, incrementing degree[x] mark[z]  FALSE Constant time O(1) Assume z  y yx y x

D ELETE -M IN Operation (1)  F IB -H EAP -D ELETE -M IN (H)  Delete the min and concatenate its children into root list H min[H] n[H] = 14 t(H) = 5 m(H) = 3

D ELETE -M IN Operation (2)  F IB -H EAP -D ELETE -M IN (H)  Delete the min and concatenate its children into root list  Then what? Where is the new minimum? H min[H] n[H] = 13 t(H) = 7 m(H) = 3 ?

0123 D ELETE -M IN Operation (3)  F IB -H EAP -D ELETE -M IN (H)  Delete the min and concatenate its children into root list  Consolidate trees (so that no two roots of same degree) H n[H] = 13 t(H) = 7 m(H) = 3 current min

0123 D ELETE -M IN Operation (4)  F IB -H EAP -D ELETE -M IN (H)  Delete the min and concatenate its children into root list  Consolidate trees (so that no two roots of same degree) H n[H] = 13 t(H) = 7 m(H) = 3 current min

0123 D ELETE -M IN Operation (5)  F IB -H EAP -D ELETE -M IN (H)  Delete the min and concatenate its children into root list  Consolidate trees (so that no two roots of same degree) H n[H] = 13 t(H) = 7 m(H) = 3 current min

0123 D ELETE -M IN Operation (5)  F IB -H EAP -D ELETE -M IN (H)  Delete the min and concatenate its children into root list  Consolidate trees (so that no two roots of same degree) H n[H] = 13 t(H) = 7 m(H) = 3 min Link 17 and 23 (degree 0) current

0123 D ELETE -M IN Operation (6)  F IB -H EAP -D ELETE -M IN (H)  Delete the min and concatenate its children into root list  Consolidate trees (so that no two roots of same degree) H n[H] = 13 t(H) = 6 m(H) = 3 current min Link 17 and 7 (degree 1)

0123 D ELETE -M IN Operation (7)  F IB -H EAP -D ELETE -M IN (H)  Delete the min and concatenate its children into root list  Consolidate trees (so that no two roots of same degree) H n[H] = 13 t(H) = 5 m(H) = 3 current min Link 7 and 24 (degree 2)

0123 D ELETE -M IN Operation (8)  F IB -H EAP -D ELETE -M IN (H)  Delete the min and concatenate its children into root list  Consolidate trees (so that no two roots of same degree) H n[H] = 13 t(H) = 4 m(H) = 3 current min

0123 D ELETE -M IN Operation (9)  F IB -H EAP -D ELETE -M IN (H)  Delete the min and concatenate its children into root list  Consolidate trees (so that no two roots of same degree) H n[H] = 13 t(H) = 4 m(H) = 3 current min

0123 D ELETE -M IN Operation (10)  F IB -H EAP -D ELETE -M IN (H)  Delete the min and concatenate its children into root list  Consolidate trees (so that no two roots of same degree) H n[H] = 13 t(H) = 4 m(H) = 3 current min

0123 D ELETE -M IN Operation (11)  F IB -H EAP -D ELETE -M IN (H)  Delete the min and concatenate its children into root list  Consolidate trees (so that no two roots of same degree) H n[H] = 13 t(H) = 4 m(H) = 3 current min Link 41 and 18 (degree 1)

0123 D ELETE -M IN Operation (12)  F IB -H EAP -D ELETE -M IN (H)  Delete the min and concatenate its children into root list  Consolidate trees (so that no two roots of same degree) H 52 n[H] = 13 t(H) = 3 m(H) = 3 current min DONE !

D ELETE -M IN Operation (13)  F IB -H EAP -D ELETE -M IN (H)  Delete the min and concatenate its children into root list  Consolidate trees (so that no two roots of same degree) H n[H] = 13 t(H) = 3 m(H) = 3 52 min[H] Final outcome

Analysis of DELETE-MIN  Notations:  Let H be the heap before D ELETE -M IN op;  t(H) is number of trees in H; D(n) is max degree in any n-node FH  k linking steps are done during consolidation  Steps in DELETE-MIN Cost  Remove min node x O(1)  Concat children of x to root list O(D(n))  Initialize ptr data structure A O(D(n))  Consolidation step…O(D(n)+k)  TOTAL: O(D(n)+k) # trees t(H)t(H) t(H)+D(n)  1 t(H)+D(n)  1  k Amortized Time: c’(H) = c(H) +  = O(D(n))  = D(n)  1  k Show later that D(n) = O(lg n)

What have we got now? (2) With Mergeable Heap Operations, i.e., I NSERT and M ELD, D ELETE -M IN can show that all the tree in H are actually Binomial Trees ! ( Now, how about D ECREASE -K EY ? )

D ECREASE -K EY Operation (1-1)  F IB -H EAP -D ECREASE -K EY (H, x, k) [1 st example: case 1]  Decrease key of x to k  Cut off x from its parent, min[H] n[H] = 16 t(H) = 3 m(H) = 3 Decrease 45 to 16

D ECREASE -K EY Operation (1-2)  F IB -H EAP -D ECREASE -K EY (H, x, k) [1 st example: case 1]  Decrease key of x to k  Cut off x from its parent, insert x into root list, update min[H] min[H] n[H] = 16 t(H) = 3 m(H) = 3 Decrease 45 to 16

D ECREASE -K EY Operation (1-3)  F IB -H EAP -D ECREASE -K EY (H, x, k) [1 st example: case 1]  Decrease key of x to k  Cut off x from its parent, insert x into root list, update min[H]  If p[x] is not marked, mark p[x] n[H] = 16 t(H) = 4 m(H) = Decrease 45 to min[H] 24 Mark p[x] 24 Done ! n[H] = 16 t(H) = 4 m(H) = 4

D ECREASE -K EY Operation (2-1)  F IB -H EAP -D ECREASE -K EY (H, x, k) [2 nd example: case 2]  Decrease key of x to k  Cut off x from its parent, insert x into root list, update min[H] n[H] = 16 t(H) = 4 m(H) = Decrease 35 to min[H] 24

D ECREASE -K EY Operation (2-2)  F IB -H EAP -D ECREASE -K EY (H, x, k) [2 nd example: case 2]  Decrease key of x to k  Cut off x from its parent, insert x into root list, update min[H]  If p[x] is marked, then cut off p[x], unmark, insert… n[H] = 16 t(H) = 5 m(H) = Decrease 35 to 4 4 min[H] p[x] is marked

D ECREASE -K EY Operation (2-3)  F IB -H EAP -D ECREASE -K EY (H, x, k) [2 nd example: case 2]  Decrease key of x to k  Cut off x from its parent, insert x into root list, update min[H]  If p[x] is marked, then cut off p[x], unmark, insert… REPEAT n[H] = 16 t(H) = 6 m(H) = Decrease 35 to p[p[x]] is marked min[H]

D ECREASE -K EY Operation (2-4)  F IB -H EAP -D ECREASE -K EY (H, x, k) [2 nd example: case 2]  Decrease key of x to k  Cut off x from its parent, insert x into root list, update min[H]  If p[x] is marked, then cut off p[x], unmark, insert… REPEAT n[H] = 16 t(H) = 7 m(H) = Decrease 35 to Done ! min[H]

Analysis of D ECREASE -K EY  Notations:  Let H be the heap before D ECREASE -K EY op;  t(H) is number of trees in H; m(H) is # marked nodes in H  c cascading cuts are done in total  Steps in D ECREASE -K EY Cost  Decrease key of x O(1)  Cascading cuts (unmark) O(c)  Mark final parent node O(1)  TOTAL: O(c) Amortized Time: c’(H) = c(H) +  = O(1)  = c + 2(2  c) = (4  c ) Viola !! # trees t(H) t(H)+c # mark m(H) m(H)  c+1 m(H)  c+2

D ELETE Operation  F IB -H EAP -D ELETE (H, x)  D ECREASE -K EY (H, x,  )  D ELETE -M IN (H)  Amortized Time: O(D(n)) H min[H] n[H] = 14 t(H) = 5 m(H) = 3

Bounding the Maximum Degree D(n) ::= maximum degree in any F IB -H EAP with n nodes Shall prove that D(n)  lg  n = O(lg n) where  = (1+  5)/2 = [golden ratio] and lg  n = 1.44 lg n

Bounding D(n) (1) Lemma 1: Let x be any node in FIB-HEAP with degree[x] = k. Let y 1, y 2,…, y n be children of x in the order in which they are linked to x (from earliest to latest). Then degree[y 1 ]  0 and degree[y i ]  i  2 for i=2, …,k. Proof: When y i was linked to x, degree[x] = i  1 since all of y 1, y 2,…, y i  1 were already children of x. Thus, degree[y i ] = degree[x] = i  1 at the time of linking. In addition, y i could lose at most 1 child, so degree[y i ]  i  2 x … y1y1 y2y2 yiyi … ykyk

Bounding D(n) (2) Define s k ::= minimum size of a tree of degree k in a FH. & let f k be such a minimal degree-k tree. To bound D(n), two possible approaches: 1. For a given size n, what is the maximum degree[x] ? 2. For a given degree k, what is the smallest size ? f1f1 f0f0 f2f2 f3f3 f4f4 f2f2 f1f1 f0f0 f0f0 x … y1y1 y2y2 yiyi … ykyk Fibonacci numbers

Bounding D(n) (3) Lemma: Let x be node in FH, and degree[x]=k. Then, size(x)  s k  F k+2   k. x … y1y1 y2y2 yiyi … ykyk Proof: First some small values: s 0 = 1, s 1 = 2, s 2 = 3 General Case: f1f1 f0f0 f2f2 Then show: F k+2 = 1 + (F 0 +F 1 +…+F k ) F k+2   k s k  F k+2 Corollary: D(n)  lg  n = 1.44 lg n = O(lg n)

Fibonacci Heaps (Summary)  M INIMUM (H)O(1)  U NION (H 1, H 2 )O(1)  I NSERT (H, x)O(1)  E XTRACT -M IN (H)O(lg n)*  D ECREASE- K EY (H, x, k)O(1)*  D ELETE (H, x)O(lg n)*