1. Chapter 15: Dynamic Programming III

2. Subsequence of a String
Let S = s1s2…sm be a string of length m
Any string of the form
si1 si2 … sik
with i1  i2  …  ik is a subsequence of S
E.g., if S = farmers
 fame, arm, mrs, farmers, are some of the subsequences of S

3. Longest Common Subsequence
Let S and T be two strings
If a string is both
a subsequence of S and
a subsequence of T,
it is a common subsequence of S and T
In addition, if it is the longest possible one, it is a longest common subsequence

4. Longest Common Subsequence
E.g.,
S = algorithms
T = logarithms
Then, aim, lots, ohms, grit, are some of the common subsequences of S and T
Longest common subsequences:
lorithms , lgrithms

5. Longest Common Subsequence
Let S = s1s2…sm be a string of length m
Let T = t1t2…tn be a string of length n
Can we quickly find a longest common subsequence (LCS) of S and T ?

6. Optimal Substructure
Let X = x1x2…xk be an LCS of S1,i = s1 s2…si and T1,j = t1t2…tj.
Lemma:
If si = tj, then xk = si = tj, and x1x2…xk-1 must be the LCS of S1,i-1 and T1,j-1
If si  tj, then X must either be
(i) an LCS of S1,i and T1,j-1 , or
(ii) an LCS of S1,i-1 and T1,j

7. Optimal Substructure Let leni,j = length of the LCS of S1,i and T1,j
Lemma: For any i, j  1,
if si = tj, leni,j = leni-1,j-1 + 1
if si  tj, leni,j = max { leni,j-1 , leni-1,j }

8. Compute_L(m, n) runs in O(2m+n) time
Length of LCS
Define a function Compute_L(i,j) as follows:
Compute_L(i, j) /* Finding leni,j */
1. if (i == 0 or j == 0 ) return 0; /* base case */
2. if (si == tj)
return Compute_L(i-1,j-1) + 1;
3. else
return max {Compute_L(i-1,j), Compute_L(i,j-1)};
Compute_L(m, n) runs in O(2m+n) time

9. Overlapping Subproblems
To speed up, we can see that :
To Compute_L(i,j-1) and Compute_L(i-1,j), has a common subproblem:
Compute_L(i-1,j-1)
In fact, in our recursive algorithm, there are many redundant computations !
Question: Can we avoid it ?

10. Bottom-Up Approach
Let us create a 2D table L to store all leni,j values once they are computed
BottomUp_L( ) /* Finding min #operations */
1. For all i and j, set L[i,0] = L[0, j] = 0;
2. for (i = 1,2,…, m)
Compute L[i,j] for all j;
// Based on L[i-1,j-1], L[i-1,j], L[i,j-1]
3. return L[m,n] ;
Running Time = Q(mn)

11. Example Run: After Step 1D O R M I T Y

12. Example Run: After Step 2, i = 1D O R M I T Y 1

13. Example Run: After Step 2, i = 2D O R M I T Y 1 2

14. Example Run: After Step 2, i = 3D O R M I T Y 1 2 3

15. Example Run: After Step 2, i = 4D O R M I T Y 1 2 3

16. Example Run: After Step 2D O R M I T Y 1 2 3 4

17. Extra information to obtain an LCSD O R M I T Y 1 1

18. Extra Info: After Step 2, i = 2D O R M I T Y 1 1 1 2 2

19. Extra Info: After Step 2, i = 3D O R M I T Y 1 1 1 2 2 2 3 3

20. D O R M I T Y Extra Info: After Step 2 1 1 1 2 2 2 3 3 3 41 1 1 2 2 2 3 3 3 4 4 4

21. LCS obtained by tracing from L[m,n]1 1 1 2 2 2 3 3 3 4 4

22. Computing the length of an LCS
LCS_LENGTH(X,Y)
1 m  X.length
2 n  Y.length
3 for i  1 to m
c[i, 0]  0
5 for j  1 to n
6 c[0, j]  0
7 for i  1 to m
for j  1 to n

23. Computing the length of an LCS
if xi == yj
c[i, j]  c[i-1, j-1]+1
b[i, j]  “”
elseif c[i–1, j]  c[i, j-1]
c[i, j]  c[i-1, j]
b[i, j]  “”
else c[i, j]  c[i, j-1]
b[i, j]  “”
17 return c and b

24. Remarks
Again, a slight change in the algorithm allows us to obtain a particular LCS
Also, we can make minor changes to the recursive algorithm and obtain a memoized version (whose running time is O(mn))

25. Writing a Translation Program
In real life, different words may be searched with different frequencies
E.g., apple may be more often than pizza
Also, there may be different frequencies for the unsuccessful searches
E.g., we may unluckily search for a word in the range (hotdog, pizza) more often than in the range (apple, banana)

26. Suppose your friend in Google gives you the probabilities of what a search will be:
= apple
= banana
0.21
 apple
0.01
(apple, banana)
0.10
0.18
= burger
0.05
(burger, hotdog)
0.12
(banana, burger)
= hotdog
= pizza
0.02
(hotdog, pizza)
0.04
= spaghetti
0.07
0.11
(pizza, spaghetti)
 spaghetti

27. This tree has better expected performance
Given these probabilities, we may want words that are searched more frequently to be nearer the root of the search tree
burger
hotdog
apple
banana
pizza
spaghetti
This tree has better expected performance

28. Expected Search Time
To handle unsuccessful searches, we can modify the search tree slightly (by adding dummy leaves), and define the expected search time as follows:
Let k1  k2  …  kn denote the n keys, which correspond to the internal nodes
Let d0  d1  d2  …  dn be dummy keys for ranges of the unsuccessful search
 dummy keys correspond to leaves

29. Search tree of Page 28 after modificationk2 k1 k5 d0 d1 k4 k6 k3 d4 d5 d6 d2 d3
Search tree of Page 28 after modification

30. Search Time Lemma: Based on the modified search tree:
when we search for a word ki,
search time = node-depth(ki)
when we search for a word in range dj,
search time = node-depth(dj)

31. Expected Search Time Let pi = Pr( ki is searched )
Let qj = Pr( word in dj is searched )
So, Si pi + Sj qj = 1
Expected search time
= Si pi node-depth(ki) + Sj qj node-depth(dj)

32. 15.5 Optimal Binary search trees
cost:2.75
optimal!!
cost:2.80

33. Optimal Binary Search Tree
Question:
Given the probabilities pi and qj,
can we construct a binary search tree whose expected search time is minimized?
Such a search tree is called an Optimal Binary Search Tree

34. Optimal Substructure
Let T = optimal BST for the keys ( ki, ki+1, … , kj; di-1, di, … , dj ).
Let L and R be its left and right subtrees.
Lemma: Suppose kr is the root of T. Then,
L must be an optimal BST for the keys (ki , ki+1, … , kr-1; di-1, di, … , dr-1)
R must be an optimal BST for the keys (kr+1, kr+2, … , kj; dr, dr+1, … , dj)

35. Optimal Substructure
Let Ei,j = expected time spent with the keys ( ki, ki+1, …, kj; di-1, di, …, dj ) in optimal BST
Let wi,j = Ss=i to j ps + St=i-1 to j qt
= sum of the probabilities of keys ( ki, ki+1, … , kj; di-1, di, … , dj )

36. Optimal Substructure Lemma: For any j  i,
Ei,j = minr { Ei,r-1 + Er+1,j + wi,j } kr
Contribute wi,j
Contribute Ei,r-1
Contribute Er+1,j

37. Optimal Binary Search Tree
Define a function Compute_E(i,j) as follows:
Compute_E(i, j) /* Finding ei,j */
1. if (i == j+1) return qj; /* Exp time with key dj */
2. min = ;
3. for (r = i, i+1, …, j) {
g = Compute_E(i,r-1) + Compute_E(r+1,j) + wi,j ;
if (g  min) min = g; }
4. return min ;

38. Optimal Binary Search Tree
Question: We want to get Compute_E(1,n)
What is its running time?
Similar to Matrix-Chain Multiplication, the recursive function runs in W(3n) time
In fact, it will examine at most once for all possible binary search tree  Running time = O(C(2n-2,n-1)/n)
Catalan Number

39. Overlapping Subproblems
Here, we can see that :
To Compute_E(i,j) and Compute_E(i,j+1), there are many COMMON subproblems:
Compute_E(i,i+1), …, Compute_E(i,j-1)
So, in our recursive algorithm, there are many redundant computations !
Question: Can we avoid it ?

40. Bottom-Up Approach
Let us create a 2D table E to store all Ei,j values once they are computed
Let us also create a 2D table W to store all wi,j
We first compute all entries in W.
Next, we compute Ei,j for j-i = 0,1,2,…,n-1

41. Bottom-Up Approach BottomUp_E( ) /* Finding min #operations */
1. Fill all entries of W
2. for j = 1, 2, …, n, set E[j+1,j] = qj ;
3. for (length = 0,1,2,…, n-1)
Compute E[i,i+length] for all i;
// From W and E[x,y] with |x-y| < length
4. return E[1,n] ;
Running Time = Q(n3)

42. The table e[i,j], w[i,j], and root[i,j] compute by OPTIMAL-BST on the key distribution.

43. Remarks
Again, a slight change in the algorithm allows us to get the exact structure of the optimal binary search tree
Also, we can make minor changes to the recursive algorithm and obtain a memoized version (whose running time is O(n3))

44. Homework
Use the dynamic programming approach to write a program to find the maximum sum in any contiguous sub-array of a given array of n real values. Show the time complexity of your algorithm. (Due Nov. 23)
Practice at home: exercises , , ,

45. Brainstorm
There is a staircase with n steps. Your friend, Jack, wants to count how many different ways he can walk up this staircase. Because John is quite tall, in one move, he can choose either to walk up 1 step, 2 steps, or 3 steps. Let Fk denote the number of ways John can walk up a staircase with k steps. So, F1 = 1, F2 = 2, F3 = 4, and F4 = 7. Derive a recurrence for Fk, and show that Fn can be computed in O(n) time.

46. Brainstorm
There is a staircase with n stages, and on each stage there is a coupon to your favorite restaurant. Each coupon has an associated value which may be different. You can climb up 1, 2, or 3 stages in a step. Since you are in a hurry, you need to reach the top of the stair in at most L steps, where L ≤ n. When you visit a particular stage, you can collect the coupon on that stage. Find a way to climb the stair within L steps so that you can collect coupons with maximum total value. Your algorithm should run in O(n2) time.

47. Brainstorm
外遇村裡有50對夫妻，丈夫全都有外遇。妻子都知道所有外遇男人，就是不知道自己的先生有外遇，妻子之間彼此也不會互相交換訊息。村子有一個規定，妻子若能證明自己的先生外遇，就必需在當天晚上12:00殺死他。有一天，公認不會說謊的皇后來到外遇村，宣布至少有一位丈夫不忠。結果會發生甚麼事？