Algorithms CSCI 235, Spring 2019 Lecture 27 Dynamic Programming II

The problem Problem: Find the best way to multiply n matrices (where best means the one that uses the minimum number of scalar multiplications). Practical use: Computer Graphics (Long chains of matrices). Digital Signal processing: applying filters to signals, etc. Formally: Given <A1, A2, A3, ...An> are n matrices, fully parenthesize the product A1A2A3...An in a way to minimize the number of scalar multiplications. Exhaustive Search takes prohibitively long:

Number of Scalar Multiplications Multiplying 2 matrices, A1, A2, where A1 is pxq and A2 is qxr takes pqr scalar multiplications. Example: 2x3 3x1 Number of multiplications = 2x3x1 = 6

1. Characterize structure of optimal solution Notation: Ai .. j = AiAi+1Ai+2 . . . Aj An optimal solution of A1 ..n = (A1A2 ...Ak)(Ak+1Ak+2 ...An), 1<=k<n Cost = cost of computing A1 .. k + cost of computing Ak+1 .. n + cost to multiply the 2 results together. For the solution to be optimal, we must first find the optimal solutions to the subproblems: A1 .. k and Ak+1 .. n (Why must they be optimal solutions?)

Example A1A2A3A4 A1(A2A3A4) (A1A2)(A3A4) (A1A2A3)A4 k=3 k=1 k=2 Note: As we compute the optimal solutions for subproblems, we do the same computation repeatedly (e.g. A2A3). To avoid repeat computations, we will store the values as each one is computed.

2. Recursively define the value of the optimal solution For subproblem, Ai .. j, 1 <= i <= j <=n Compute m[i, j] = minimum number of scalar multiplications to compute Ai .. j Matrix Ai has dimension pi-1 x pi (rows x cols) Example: More notation: pi is number of columns in the ith matrix. A1 A2 A3 3x3 3x1 1x4 p0 p1 p1 p2 p2 p3

Number of multiplications If A1 is p0 x p1, A2 is p1 x p2, then A1A2 is p0 x p2 If A3 is p2xp3 then A1A2A3 is p0 x p3 (A1A2 ... Ak) is p0 x pk M1=(AiAi+1 ... Ak) is pi-1 x pk M2=(Ak+1 ... Aj) is pk x pj Therefore, the number of scalar multiplications to multiply M1M2 is pi-1pkpj

Defining m[i, j] recursively If i = j, no multiplication needed, so m[i, j] = 0 If i < j, m[i, j] = m[i, k] + m[k+1, j] + pi-1pkpj Example: m[2, 6], let k = 4: (A2A3A4)(A5A6) We will work this out in class.

Determining the Best split We must check each value of k to determine which gives the minimum cost: Minimum cost of Ai .. j is: Keep track of minimum cost in table, m[i, j] Keep track of k value that gives the minimum cost in table, s[i, j] = k

3. Compute the value of the optimal solution Step 3. Compute the value of the optimal solution from the bottom up. We start with the smallest subproblems: i = j m[i, i] = 0 Next compute cost of chains of length 2: m[i, i+1] Third, compute cost of chains of length 3 (using values for length 2): m[i, i+2] Total number of subproblems to solve: 1 for each i & j, 1 <=i <=j <=n Combination:

Matrix Chain Order Code Matrix-Chain-Order(p) // p is an array containing pi-1 to pj n = p.length – 1 // Number of matrices in chain Let m[1..n, 1..n] and s[1..n – 1, 2..n] be new tables for i = 1 to n m[i, i] = 0 // Single matrices take 0 multiplications for b = 2 to n // b is length of chain for i = 1 to n – b + 1 // all possible starting indices for length b j = i + b - 1 // Ending index of chain of length b m[i, j] = ∞ // Large value to start to find minimum for k = i to j - 1 // Try all possible splits of this chain q = m[i, k] + m[k + 1, j] + pi-1pkpj // smaller chains are // already computed if q < m[i, j] // If minimum value, then store it m[i, j] = q s[i, j] = k return m and s

Example Find the best parenthesization for multiplying the following chain: A1A2A3A4, where A1 is 3x3, A2 is 3x2, A3 is 2x2 and A4 is 2x4. p = <3, 3, 2, 2, 4> j j m[i, j] s[i, j] 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 i i