Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 CS473 - Algorithms I CS 473 – DP Examples Other Dynamic Programming Problems Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent.

Similar presentations


Presentation on theme: "1 CS473 - Algorithms I CS 473 – DP Examples Other Dynamic Programming Problems Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent."— Presentation transcript:

1 1 CS473 - Algorithms I CS 473 – DP Examples Other Dynamic Programming Problems Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University View in slide-show mode

2 2 CS473 - Algorithms I CS 473 – DP Examples Problem 1 Subset Sum Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University

3 3 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Subset-Sum Problem Given:  a set of integers X = {x 1, x 2, …, x n }, and  an integer B Find:  a subset of X that has maximum sum not exceeding B. Notation: S n,B = {x 1, x 2, …, x n : B} is the subset-sum problem  The integers to choose from: x 1, x 2, …, x n  Desired sum: B

4 4 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Subset-Sum Problem Example: x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 x 9 x 10 x 11 x 12 S 12,99 : {20, 30, 14, 70, 40, 50, 15, 25, 80, 60, 10, 95: 99} Find a subset of X with maximum sum not exceeding 99. An optimal solution: x 1 x 3 x 5 x 8 N opt = {20, 14, 40, 25} with sum = 99

5 5 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Optimal Substructure Property  Consider the solution as a sequence of n decisions: i th decision: whether we pick number x i or not Let N opt be an optimal solution for S n,B Let x k be the highest-indexed number in N opt xkxk N opt (optimal for S n,B ) N ʹ opt = N opt – {x k }

6 6 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Optimal Substructure Property Lemma: N ʹ opt = N opt – {x k } is an optimal solution for the subproblem S k-1, B-xk = {x 1, x 2, …, x k-1 : B-x k } and c(N opt ) = x k + c(N ʹ opt ) where c(N) is the sum of all numbers in subset N xkxk N opt (optimal for S n,B ) N ʹ opt = N opt – {x k } (optimal for S k-1, B-xk )

7 7 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Optimal Substructure Property - Proof Proof: By contradiction, assume that there exists another solution A ʹ for S k-1, B – xk for which: c(A ʹ ) > c(N ʹ opt ) and c(A ʹ ) ≤ B – x k i.e. A ʹ is a better solution than N ʹ opt for S k-1, B-xk Then, we can construct A = A ʹ ∪ {x k } as a solution to S k, B. We have: c(A) = c(A ʹ ) + x k > c(N ʹ opt ) + x k = c(N opt ) Contradiction! N opt was assumed to be optimal for S k,B. Proof complete.

8 8 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Optimal Substructure Property - Example Example: x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 x 9 x 10 x 11 x 12 S 12,99 : {20, 30, 14, 70, 40, 50, 15, 25, 80, 60, 10, 95: 99} x 1 x 3 x 5 x 8 N opt = {20, 14, 40, 25} is optimal for S 12, 99 N ʹ opt = N opt – {x 8 } = {20, 14, 40} is optimal for x 1 x 2 x 3 x 4 x 5 x 6 x 7 the subproblem S 7,74 = {20, 30, 14, 70, 40, 50, 15: 74} and c(N opt ) = 25 + c(N ʹ opt ) = = 99

9 9 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Recursive Definition an Optimal Solution c[i, b]: the value of an optimal solution for S i,b = {x 1, …, x i : b} According to this recurrence, an optimal solution N i,b for S i,b :  either contains x i c(N i,b ) = x i + c(N i-1, b-xi )  or does not contain x i c(N i,b ) = c(N i-1, b )

10 10 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University 1n 1 n i bb-x i c[i, b] i-1 Need to process: c[i, b] after computing: c[i-1, b], c[i-1, b-x i ]

11 11 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University 1n 1 m i bb-x i c[i, b] i-1 for i 1 to m for b 1 to n …. c[i, b] =

12 12 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Computing the Optimal Subset-Sum Value SUBSET-SUM (x, n, B) for b ← 0 to B do c[0, b] ← 0 for i ← 1 to n do c[i, 0] ← 0 for i ← 1 to n do for b ← 1 to B do if x i ≤ b then c[i, b] ← Max{x i + c[i-1, b-x i ], c[i-1, b]} else c[i, b] ← c[i-1, b] return c[n, B]

13 13 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Finding an Optimal Subset SOLUTION-SUBSET-SUM (x, b, B, c) i ← n b ← B N ← ∅ while i > 0 do if c[i, b] = c[i-1, b] then i ← i – 1 else N ← N ∪ {x i } i ← i – 1 b ← b – x i return N

14 14 CS473 - Algorithms I CS 473 – DP Examples Problem 2 Optimal Binary Search Tree Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University

15 15 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Reminder: Binary Search Tree (BST) All keys in the left subtree less than 8 All keys in the right subtree greater than 8 This property holds for all nodes. Image from Wikimedia

16 16 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Binary Search Tree Example Example: English-to-French translation Organize (English, French) word pairs in a BST  Keyword: English word  Satellite data: French word end do then beginelseifwhile We can search for an English word (node key) efficiently, and return the corresponding French word (satellite data).

17 17 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Binary Search Tree Example Suppose we know the frequency of each keyword in texts: begin do else end if then while 5% 40% 8% 4% 10% 10% 23% end do then beginelseifwhile 4% 10%23%8%5% 40%10%

18 18 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Cost of a Binary Search Tree end do then beginelseifwhile 4% 10%23%8%5% 40%10% Example: If we search for keyword “while”, we need to access 3 nodes. So, 23% of the queries will have cost of 3. = 1x x x x x x x0.23 = 2.42

19 19 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Cost of a Binary Search Tree end 4% if 10% while 23% else 8% begin 5% do 40% then 10% A different binary search tree (BST) leads to a different total cost: Total cost = 1x x x x x x x0.04 = 2.18 This is in fact an optimal BST.

20 20 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Optimal Binary Search Tree Problem Given: A collection of n keys K 1 < K 2 < … K n to be stored in a BST. The corresponding p i values for 1 ≤ i ≤ n p i : probability of searching for key K i Find: An optimal BST with minimum total cost: Note: The BST will be static. Only search operations will be performed. No insert, no delete, etc.

21 21 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Cost of a Binary Search Tree Lemma 1: Let T ij be a BST containing keys K i < K i+1 < … < K j. Let T L and T R be the left and right subtrees of T. Then we have: TLTL TRTR Intuition: When we add the root node, the depth of each node in T L and T R increases by 1. So, the cost of node h increases by p h. In addition, the cost of root node r is p r. That’s why, we have the last term at the end of the formula above.

22 22 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Optimal Substructure Property Lemma 2: Optimal substructure property Consider an optimal BST T ij for keys K i < K i+1 < … < K j Let K m be the key at the root of T ij T i,m-1 T m+1,j KmKm Then: T i,m-1 is an optimal BST for subproblem containing keys: K i < … < K m-1 T m+1,j is an optimal BST for subproblem containing keys: K m+1 < … < K j

23 23 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Recursive Formulation Note: We don’t know which root vertex leads to the minimum total cost. So, we need to try each vertex m, and choose the one with minimum total cost. c[i, j]: cost of an optimal BST T ij for the subproblem K i < … < K j where

24 24 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Bottom-up computation Before computing c[i, j], we have to make sure that the values for c[i, r-1] and c[r+1,j] have been computed for all r. How to choose the order in which we process c[i, j] values?

25 25 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University 1n 1 n i ij r j c[i, j] r c[i,r-1] c[r+1,j] c[i,j] must be processed after c[i,r-1] and c[r+1,j]

26 26 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University 1n 1 n i ij j c[i,j] If the entries c[i,j] are computed in the shown order, then c[i,r-1] and c[r+1,j] values are guaranteed to be computed before c[i,j].

27 27 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Computing the Optimal BST Cost COMPUTE-OPTIMAL-BST-COST (p, n) for i ← 1 to n+1 do c[i, i-1] ← 0 PS[1] ← p[1] // PS[i]: prefix_sum(i): Sum of all p[j] values for j ≤ i for i ← 2 to n do PS[i] ← p[i] + PS[i-1] // compute the prefix sum for d ← 0 to n−1 do for i ← 1 to n – d do j ← i + d c[i, j] ← ∞ for r ← i to j do c[i, j] ← min{c[i, j], c[i,r-1] + c[r+1, j] + PS[j] – PS[i-1]} return c[1, n]

28 28 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Note on Prefix Sum  We need P ij values for each i, j (1 ≤ i ≤ n and 1 ≤ j ≤ n), where:  If we compute the summation directly for every (i, j) pair, the total runtime would be Θ(n 3 ).  Instead, we spend O(n) time in preprocessing to compute the prefix sum array PS. Then we can compute each P ij in O(1) time using PS.

29 29 CS 473 – DP Examples Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University Note on Prefix Sum In preprocessing, compute for each i: PS[i]: the sum of p[j] values for 1 ≤ j ≤ i Then, we can compute P ij in O(1) time as follows: P ij = PS[i] – PS[j-1] Example: p: PS: P 27 = PS[7] – PS[1] = 0.53 – 0.05 = 0.48 P 36 = PS[6] – PS[2] = 0.45 – 0.07 = 0.38


Download ppt "1 CS473 - Algorithms I CS 473 – DP Examples Other Dynamic Programming Problems Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent."

Similar presentations


Ads by Google