Minimum Cost Binary Trees Speaker: Dana Moshkovitz

Outline Minimum Cost Binary Trees: The problem’s description The Garsia-Wachs algorithm Kingston’s proof for the Garsia-Wachs algorithm

Minimum Cost Binary Trees: Description Given a list of n weights (nonnegative real numbers), p 1,...,p n, let us look at the class of binary trees, which these weights are assigned to their leaves in the same order given. we would like to find the tree with the minimum weighted external path length, i.e. that tree T, s.t is minimal (h i, namely the level of p i, is the number of arcs in T along the path from the root to the leaf, whose weight is p i ) Let’s observe an example...

Suppose these are the weights we get: We can construct the following tree: Now let us add to each leave its weighted external path length Minimum Cost Binary Trees Presenting the problem Example Total: 106

Total: 106 Clearly, this tree was not in fact minimal: Total: 101 This is because we can construct a “better” tree. Such tree is this one: Minimum Cost Binary Trees Presenting the problem Example

Minimum Cost Binary Trees Solving the problem Intuition Our first observation is that the smaller the weight, the lower it should be in the tree. The above obvious observation is all we needed in the unordered version. Let us recall Huffman’s simple greedy algorithm. The general idea behind his solution was to construct the tree from the bottom and always choose the two smallest weights to be siblings. Let us demonstrate...

Minimum Cost Binary Trees Solving the problem Intuition - Back to Huffman’s Algorithm 6 Again we have these weights: We choose the two less expensive fatherless weights and create them a father Total: 100

Minimum Cost Binary Trees Solving the problem Intuition The question arising here is whether and how can we apply the same idea to the ordered case. The problem now is that we cannot arbitrarily choose the two cheapest leaves. The two leaves we choose must be neighbors. Unfortunately, this demand makes the naive greedy algorithm incorrect (For example, suppose the weights are 4,3,4,4. The correctness of the result is depended on whether we choose or to be the minimal pair). So, what do we do??

Minimum Cost Binary Trees Solving the problem Toward a Solution The feel is that we have some pretty useful ideas: Constructing the tree from the bottom Finding siblings in each iteration However, as we have already seen, they still need some refinement.

We need an additional observation: Suppose we have weights p 1,...,p n and levels h 1,...,h n. There is only one binary tree (at most) determined by this data, and that tree can be computed efficiently. Let’s demonstrate and explain this: Minimum Cost Binary Trees Solving the problem Toward a Solution

Minimum Cost Binary Trees Solving the problem Toward a Solution Given feasible weights p 1,...,p n and levels h 1,...,h n, let us see how can we construct the binary tree they represent Algorithmically we can obtain this by preserving an array, which will hold in its i-th entry a pointer to a node in the i-th level lacking a right son.

Minimum Cost Binary Trees Solving the problem Toward a Solution We conclude, that it suffices to solve the unordered version under the constraint that the level of each leaf in the resulting tree equals the level of the corresponding leaf of an ordered solution. What we need to explain now is how do we solve the unordered problem under this constraint. The question is how do we choose the right pair of neighbors.

Let us define the following: A pair of leaves p i-1,p i is right minimal (briefly R.M.) if (i) 1<i  n (ii) p i-2 +p i-1  p i-1 +p i (iii) p i-1 +p i i Minimum Cost Binary Trees Solving the problem Right Minimal Pairs In other words, two neighbors are right minimal if their sum is minimal among the sums to their right, but this does not hold for the pairs to their left. Remark: From now on, we shall treat our list of weights as though it consists of additional two sentinels, namely  and  +1, placed in the leftmost and rightmost positions in the list respectively. This is done for the ease of the discussion, proving the modification does not alter the solution is rather trivial (see Gilbert and Moore’s result).

Lemma 1: Suppose we have a sequence of at least three weights p a,p a+1,...,p b, s.t. p j-1 + p j < p j + p j+1, for a<j<b, then h a  h a+1 ...  h b-1 in every minimal tree containing these weights < 8 < 9 < 15 < 16 Minimum Cost Binary Trees Solving the problem Some Interesting Facts about R.M. Pairs

Proof. Suppose h j-1 <h j for some a<j<b. p j must be a left child. Then the transformation: is both legal and less expensive: since p j+1 must be in R, |R|  p j+1 >p j-1. Therefore the lemma holds.  pjpj p j-1 pjpj R R Minimum Cost Binary Trees Solving the problem Some Interesting Facts about R.M. Pairs

Lemma 2: if p i-1,p i is the rightmost R.M. pair, then h i-1  h i ...  h n in every minimal tree. Proof. Directly follows from the choice of the pair and from lemma 1.  Further explanation: notice that when a pair is the rightmost R.M pair, the sequence of pairs to its right constructs a monotone series. (Otherwise the pair that “breaks” the monotony would be the rightmost R.M pair). Minimum Cost Binary Trees Solving the problem Some Interesting Facts about R.M. Pairs

Lemma 3: if p i-1,p i is the rightmost R.M. pair, then h i-1 = h i in some minimal tree. Proof. By lemma 2 it suffices to show that h i-1  h i in some minimal tree. Suppose we have a minimal tree such that h i-1 >h i. In this tree p i-1 is a right child, so we can use the following transformation to get another minimal tree: p i-1 pipi R pipi R Again, |R|  p i-2  p i, so the new tree is necessarily minimal.  Minimum Cost Binary Trees Solving the problem Some Interesting Facts about R.M. Pairs

Minimum Cost Binary Trees Solving the problem... And finally... After we clarified the most important notion for the purpose of this lecture, namely R.M. pairs, and proved some interesting properties related to such, let us finally observe the desired algorithm. Afterwards we shall prove the correctness of our solution.

Minimum Cost Binary Trees Solving the problem The Algorithm Execute the following two steps n-1 times: (1) Locate the rightmost R.M pair of entries. (2) Find the first entry to its right, which is greater than/equals it, and move the pair to the left of this entry.

Minimum Cost Binary Trees Solving the problem Simulating the algorithm We start with the same input We locate the rightmost R.M pair, pair them and move them to their right place The heights yield the tree we have already seen:

Lemma 4: Let p i+k+1 be the first node to the right of the rightmost R.M pair p i-1,p i, s.t p i+k+1 > p i-1 +p i. In some minimal tree T for which h i-1 = h i, either (a) h i+k = h i -1,or (b) h i+k = h i, and p i+k is a right child. Proof. Begin with T the minimal tree of lemma3. By lemma 2 we merely need to show that h i+k  h i -1 in T. Let us suppose h i+k < h i -1. Let p m (m  i+k) be the first node to the right of p i, s.t. h m < h i -1. p m <p i-1 +p i. Minimum Cost Binary Trees Proving the Solution

There are two possibilities: (a) p i-1,p i are siblings in T (b) p i,p i+1 are siblings in T Let p,q denote the two siblings. We can transform T as follows and improve the cost by decreasing p+q-p m  p i-1 +p i -p m >0. This contradicts the hypothesis T was minimal. Hence h i+k  h i -1. pq pmpm... pq pmpm Minimum Cost Binary Trees Proving the Solution

It remains to prove, that when h i+k = h i, p i+k can be made a right child. If p i+k is already a right child, we are done. Otherwise, if k>0, we can transform T: p i-1 pipi... p i+k p i+k+1 p i-1 pipi... p i+k p i+k+1 Minimum Cost Binary Trees Proving the Solution

p i-1 pipi p i+k+1 p i-1 pipi p i+k+1 While if k=0, we can use this transformation: In both cases, our claim holds.  Minimum Cost Binary Trees Proving the Solution

Minimum Cost Binary Trees Proving the solution Lemma 5: For every rearrangement done in an iteration of the algorithm, the cost for the new series cannot exceed the cost for the original series of weights. Proof. Let k be the number of places we move the R.M. pair in a specific iteration. We will exhibit a tree T’ for the new arrangement, whose leaves has the same levels as in T, the minimum tree of lemma 4. This - of- course - will prove the lemma. If k=0, p i-1 and p i are siblings in T, so we may take T’=T.

Minimum Cost Binary Trees Proving the solution Otherwise k>0. First we will transform T, so p i-1 and p i (the current rightmost R.M. pair) are siblings (If they they are not so already): p i-1 pipi p i+1 p i-1 pipi p i+1 Lemma 4 states, that they have the same level Lemma 4 states p i+1 is a right child

Since k>0, the first weight greater than/equal to p i-+ p i, namely p i+k+1, is still to the right of p i. By lemma 4, there are merely two possibilities: h i+k =h i or h i+k =h i -1. In each case all we need to do is “slid” the new node to the right until it passes p i+k.  Minimum Cost Binary Trees Proving the solution p i-1 pipi p i+k... p i-1 pipi p i+k... p i-1 pipi p i+k... p i-1 pipi... p i+k

Minimum Cost Binary Trees Proving the solution Lemma 6: For every rearrangement done in an iteration of the algorithm, the cost for the new series is at least the cost for the original series of weights, and if equality holds, then the two solutions have corresponding levels. Proof. Now - given a minimum tree T’ for the new arrangement - we need to construct a minimum tree T for the original series (while preserving the levels). Again we denote the number of places we move the pair by k. If k=0, T=T’.

Minimum Cost Binary Trees Proving the solution Otherwise k>0. We know that p i+k <p i-1 +p i  p i+k+1, so we can use lemma 1 in T’ with p i+1,...,p i+k,p x, p i+k+1 (p x is the father of the R.M. pair) to obtain: h i+1 ...  h i+k  h x in T’. Hence, either TxTx p i-1 pipi p i+1... TxTx p i-1 pipi p i+1... or In the first case the treatment is simple: we need to move T x to the left.

Minimum Cost Binary Trees Proving the solution The treatment of the second case consists of two phases: (a) Slid T x to the left across all the leaves in its level. (b) Rotate all the nodes two places to the right. TxTx p i-1 pipi p i+1... TxTx p i-1 pipi p i+1... (a)

TxTx p i-1 pipi p i+1 Minimum Cost Binary Trees Proving the solution How do we perform the rotation? A simulation pipi p i-1 p i+1 pipi p i-1 p i+1

Minimum Cost Binary Trees Proving the solution For every level the pair drops downward (possibly 0), some other pair moves up one level. But we have taken a R.M. pair (for which all the pairs with greater indices weight more than it does). This implies that w(T’)=w(T a )  w(T), so the weight of T’ is at least the weight of a minimum cost tree for the original series of weights. If equality holds, the level of the pair is not changed by the rotation, so T is a minimum cost tree for the original series, which preserves the levels of T’. 

Minimum Cost Binary Trees Proving the solution Those two final lemmas allow us to finally state the following: The algorithm we presented is correct. 

Minimum Cost Binary Trees Solving the problem Another way to view this algorithm We start with the same input We locate the rightmost R.M pair, pair them and move them to their right place Which yield the tree we have already seen:

Minimum Cost Binary Trees References This overview was based on: [1]J. H. Kingston. A new proof for the Garcia-Wachs algorithm. J. of Algorithms, 9: ,1988 The Garsia-Wachs algorithm: [2]A. M. Garsia and M. L. Wachs, A new algorithm for minimum cost binary trees. Sicomp, 6(4), , 1977 Huffman’s algorithm: [3]D. Huffman, A method for the construction of minimum- redundancy codes, Proc. IRL, , 1952

Minimum Cost Binary Trees References You might be interested in previous results concerning this problem: The original O(n 3 ) algorithm: [4]E. N. Gilbert and E.F. Moore, Variable length binary encodings, Bell Systems Tech, 38, , An O(n 2 ) algorithm: [5]D. E. Knuth, Optimum binary search trees, Acta Inform. 1, 14-25, The Hu-Tucker algorithm (O(nlogn)): [6]T.C. Hu and C. Tucker, Optimum computer search trees, SIAM J. Appl. Math., , [7]T.C. Hu, A new proof of the T-C algorithm, SIAM J. Appl. Math. 25, 83-94, 1973.