1. Dynamic Programming chandakmb@gmail.comchandakmb@gmail.com, hodcs@rknec.eduhodcs@rknec.edu www.mbchandak.com Tutorial &Practice on Longest Common Sub-sequence Optimal Binary Search Tree

2. Characteristics of Dynamic Programming Developed by Richard Bellman in 1950. To provide accurate solution with the help of series of decisions. Define a small part of problem and find out the optimal solution to small part. Enlarge the small part to next version and again find optimal solution. Continue till problem is solved. Find the complete solution by collecting the solution of optimal problems in bottom up manner.

3. Characteristics Types of problems and solution strategies. 1. Problem can be solved in stages. 2. Each problem has number of states 3. The decision at a stage updates the state at the stage into the state for next stage. 4. Given the current state, the optimal decision for the remaining stages is independent of decisions made in previous states. 5. There is recursive relationship between the value of decision at a stage and the value of optimum decisions at previous stages.

4. Characteristics How memory requirement is reduce: How recursion overheads are converted into advantage - By storing the results of previous n-2 computations in computation of n-1 stage and will be used for computation of “n” stage. Not very fast, but very accurate It belongs to smart recursion class, in which recursion results and decisions are used for next level computation. Hence not only results but decisions are also generated. Final collection of results: Bottom Up Approach.

5. Longest Common Subsequence Given two strings of characters, LCS is a method to find a longest subsequence either continuous or non-continuous and is common in both the strings. The subsequence generation starts from comparison of first character of both the given strings. The two strings may or may not be of equal length. Let “A” and “B” be two strings of length “m” and “n” respectively, and let “i” and “j” be two pointers to handle the two strings. Let matrix c[m,n] be the storage matrix to store the results of comparison related with two strings. The C[m,n] matrix will be handled using pointer “i” and “j”.

6. If i=0 and j=0, then c[i,j] = 0 If a[1..i] and b[1..j] is compared and a[i] is not equal to b[j]. Then the comparison will be either In between a[1..i-1] and b[1..j]ora[1..i] and b[1..j-1] Then c[i,j] = max[c[i-1,j], c[I,j-1]] If a[1..i] and b[1..j] is compared and a[i] is equal to b[j]. Then c[i,j] = c[i-1,j-1] + 1

7. Example STRING A = G D V E G T A B = G V C E K S T i/j01234567 GVCEKST 00/H 1G 2D 3V 4E 5G 6T 7A H = Halt Case: String A = No Character String B = All Characters Result = 0 and Halt State

8. Example STRING A = G D V E G T A B = G V C E K S T i/j01234567 GVCEKST 00/H 1G 2D 3V 4E 5G 6T 7A H = Halt Case: String A = All Characters String B = No Character Result = 0 and Halt State

9. Example STRING A = G D V E G T A B = G V C E K S T i/j01234567 GVCEKST 00/H 1G 1/D H = Halt D = Diagonal S = Side Case: String A = G String B = G Result = Match Use diagonal value from the cell and add 1 to diagonal value 0+1 i/j01234567 GVCEKST 00/H 1G 1/D1/S Case: String A = G String B = V Result = No Match Use Side or Upper cell value and select maximum value If both values are same select UPPER cell value 1/S

10. Example STRING A = G D V E G T A B = G V C E K S T i/j01234567 GVCEKST 00/H 1G 1/D 1/S 2D0/H 1/U H = Halt D = Diagonal S = Side Case: String A = D String B = G Result = No Match Since the value in Upper cell is greater than Side cell Use UPPER Cell value

11. Example STRING A = G D V E G T A B = G V C E K S T i/j01234567 GVCEKST 00/H 1G 1/D1/S 2D0/H 1/U H = Halt D = Diagonal S = Side Case: String A = D String B = V Result = No Match Since the value in Upper cell is same as Side cell Use UPPER Cell value 1/U

12. Example STRING A = G D V E G T A B = G V C E K S T i/j01234567 GVCEKST 00/H 1G 1/D1/S 2D0/H1/U 3V0/H1/U2/D2/S 4E0/H1/U2/U 3/D3/S 5G0/H1/D2/U 3/U 6T0/H1/U2/U 3/U 4/D 7A0/H1/U2/U 3/U 4/U

13. Example STRING A = G D V E G T A B = G V C E K S T i/j01234567 GVCEKST 00/H 1G 1/D1/S 2D0/H1/U 3V0/H1/U2/D2/S 4E0/H1/U2/U 3/D3/S 5G0/H1/D2/U 3/U 6T0/H1/U2/U 3/U 4/D 7A0/H1/U2/U 3/U 4/U

14. Example:2 STRING A = X M J Y A U ZB = M Z J A W X U i/j01X2M3J4Y5A6U7Z 00/H 1M0/H1/U1/D1/S 2Z0/H1/U 2/D 3J0/H1/U 2/D2/S 4A0/H1/U 2/U 3/D3/S 5W0/H1/U 2/U 3/U 6X0/H1/D1/U2/U 3/U 7U0/H1/U 2/U 3/U4/D4/S

15. Algorithm: Assumptions: Let the two strings be present in array “a” size “m” and array “b” size “n” handled using pointers “i“ and “j”. The resultant array will be c[m,n] : one instance c[i,j]. The array “c” will be structure: c[i,j].val and c[i,j].dir = “u”, “s”, “d” and “h” Algorithm lcs (a,b: c) { for i = 0 to m do for j = 0 to n do if a[i]=0 or b[j]=0 { c[i,j].val = 0; c[i,j].dir=‘h’ } else { if (a[i] ≠ b[j]) c[i,j].val = max [c[i-1, j].val, c[i,j-1].val] if(c[i-1, j].val >= c[i,j-1].val) c[i,j].dir = ‘u’ else c[i,j].dir=‘s’ else c[i,j].val = c[i-1,j-1].val + 1 c[i,j].dir = ‘d’ }

16. Algorithm: Printing LCS: Assumptions: Let “A” be the given array. Let “C” be the array containing the details of LCS Algorithm print_lcs (a,c,i,j) { if (i=0 or j=0) return; if(c[i,j] = ‘d’ { print_lcs(a,c,i-1,j-1) print(a[i]) } else { if(c[i,j] = ‘u’ print_lcs(a,c,i-1,j) else print_lcs(a,c,i,j-1) }

17. Exercises Question 1: String 1:E X P O N E N T I A L String 2:P O L Y N O M I A L String 1:SUBSEQUENCE String 2:CONSEQUENCES

18. Optimal Binary Search Tree (OBST) Drawback of BST: The first value in the sequence is always used as ROOT node. If the data is present in sorted order, BST will be in the form of linked list. This will increase search time and complexity equation will be not valid. For example if dictionary word are to be stored in BST, it will be in the form of linked list: Apple, Bat, Cat, Dog, Elephant, Fish, Gun, Horse, Ink, Jug, King, Lion Since all the start characters are Greater than “A”, all the values will be on the Right side of the Apple.

19. Example of word list

20. Optimal Binary Search Tree Need: Mechanism to decide out of the available values, which value should be used as ROOT, so that the tree is balanced and COST of searching is minimum. Requirement: For selection, the probability of data in the domain text is required. There are two values: Successful probability value and Unsuccessful probability value. Application: OBST is applied to generate the tree for the set of values (keys) with defined successful and unsuccessful search probabilities.

21. Example: Creation of matrices: “w” = probability matrix “e” = evolution matrix “r” = root matrix Dimensions: n = Total number of keys given in the search space Matrix “w” = [1..n+1, 0..n] Matrix “e” = [1..n+1, 0..n] For example number of keys are K1, K2, K3, K4 then n=4 and matrix “w” and “e” will have dimensions as: [1..5, 0..4]

22. Optimal Binary Search Tree i01234 pi0.120.100.090.20 qi0.100.080.050.110.15 sum0.100.200.150.200.35

23. W01234 10.100.300.450.651.0 2#0.080.230.430.78 3##0.050.250.60 4###0.110.46 5####0.15 E01234 10.100.48/10.91/11.54/22.63/3 2#0.080.36/20.90/31.83/4 3##0.050.41/31.16/4 4###0.110.72/4 5####0.15 i01234 pi0.120.100.090.20 qi0.100.080.050.110.15 sum0.100.200.150.200.35

24. Example 2: OBST i012345 pi0.050.200.050.100.15 qi0.10 0.05 0.10 sum0.100.150.250.100.150.25

25. Solution: Example 2: Animation W012345 10.10 2 30.05 4 5 60.10 e012345 1 2 30.05 4 5 60.10 i012345 pi0.050.200.050.100.15 qi0.10 0.05 0.10 sum0.100.150.250.100.150.25

26. Empty matrix structure and formula w012345 1 20.10 30.050.15 40.050.20 50.050.30 60.10 e012345 1 2 30.05 4 5 60.10 i012345 pi0.050.200.050.100.15 qi0.10 0.05 0.10 sum0.10 0.150.25 0.15 0.10 0.10+0.15= 0.25 0.25 0.10+0.25= 0.35 0.35

27. Empty matrix structure and formula w012345 10.25 20.100.35 30.050.150.30 40.050.200.45 50.050.30 60.10 e012345 1 2 30.05 4 5 60.10 i012345 pi0.050.200.050.100.15 qi0.10 0.05 0.10 sum0.100.15 0.25 0.10 0.25 0.25+0.25=0.50 0.50 0.10 0.35+0.10=0.45 0.45

28. Empty matrix structure and formula W012345 10.250.600.751.00 20.100.350.600.85 30.050.150.300.55 40.050.200.45 50.050.30 60.10 e012345 1 2 30.05 4 5 60.10 i012345 pi0.050.200.050.100.15 qi0.10 0.05 0.10 sum0.100.150.250.100.150.25 0.100.50 0.45

29. Empty matrix structure and formula e012245 10.250.600.751.00 20.100.350.600.85 30.050.150.300.55 40.050.200.45 50.050.30 60.10 e012345 1 0.45/1 20.10 30.05 4 5 60.10 i012345 pi0.050.200.050.100.15 qi0.10 0.05 0.10 sum0.100.150.250.100.150.25 0.100.50 0.45 e012345 10.10 0.45/1 20.10 0.50/2 3 0.050.25/3 4 0.050.30/4 5 0.050.45/5 60.10 e[1,1] = e[1,0]+e[2,1]+w[1,1]  0.10+0.10+0.25  0.45 e[2,2] = e[2,1]+2[3,2]+w[2,2]  0.10+0.05+0.35  0.50

30. Empty matrix structure and formula e012345 10.250.600.751.00 20.100.350.600.85 30.050.150.300.55 40.050.200.45 50.050.30 60.10 e012345 1 0.45/1 1.00/ 2 20.10 0.50/2 3 0.050.25/3 4 0.050.30/4 5 0.050.45/5 60.10 0.50 0.45 e012345 10.10 0.45/1 1.00/ 2 2 0.80/2 3 0.05 4 0.30/4 5 0.050.45/5 60.10 e[1,2] = Two options for root between values 1 and 2 are r=1 and r=2 e[1,2] will be minimum of e[1,0]+e[2,2]+w[2,2]  0.10+0.50+0.50 =1.10 [r=1] e[1,1]+e[3,2]+w[2,2]  0.45+0.05+0.50 =1.00 [r=2] 0.10 0.25/3 First 0.80 0.50 0.05 0.45 Second 1.00 Final 0.80/2

31. Empty matrix structure and formula e012345 10.250.600.751.00 20.100.350.600.85 30.050.150.300.55 40.050.200.45 50.050.30 60.10 0.50 0.45 e012345 10.10 0.45/1 1.00/ 2 2 0.80/2 3 0.050.25/30.60/4 4 0.050.30/40.85/5 5 0.050.45/5 60.10 0.50 0.45

32. Empty matrix structure and formula e012345 10.250.600.751.00 20.100.350.600.85 30.050.150.300.55 40.050.200.45 50.050.30 60.10 0.50 0.45 e012345 10.10 0.45/1 1.00/ 2 1.30/ 2 2 0.80/2 3 0.050.25/30.60/4 4 0.050.30/40.85/5 5 0.050.45/5 60.10 0.50 0.45 e012345 10.10 0.45/1 1.00/ 2 1.30/ 2 20.100.50 0.80/2 1.30/ 2 3 0.050.25/30.60/4 4 0.050.30/40.85/5 5 0.050.45/5 60.10 012345 1 0.45/1 1.00/ 2 1.30/ 2 20.100.50 0.80/2 1.30/ 2 3 0.050.25/30.60/41.25/4,5 4 0.050.30/40.85/5 5 0.050.45/5 60.10

33. Empty matrix structure and formula e012345 10.250.600.751.00 20.100.350.600.85 30.050.150.300.55 40.050.200.45 50.050.30 60.10 0.50 0.45 e012345 10.10 0.45/1 1.00/ 2 1.30/ 2 1.80/ 2 20.100.50 0.80/2 1.30/ 2 3 0.050.25/30.60/41.25/4,5 4 0.050.30/40.85/5 5 0.050.45/5 60.10 e012345 1 0.45/1 1.00/ 2 1.30/ 2 1.80/ 2 20.100.50 0.80/2 1.30/ 2 2.25/ 5 3 0.050.25/30.60/41.25/4,5 4 0.050.30/40.85/5 5 0.050.45/5 60.10

34. Empty matrix structure and formula 012345 10.250.600.751.00 20.100.350.600.85 30.050.150.300.55 40.050.200.45 50.050.30 60.10 0.50 0.45 012345 10.10 0.45/1 1.00/21.30/21.80/22.70/2 20.100.50 0.80/2 1.30/22.25/5 3 0.050.25/30.60/41.25/4,5 4 0.050.30/40.85/5 5 0.050.45/5 60.10 k2 K3,K4,K5 K1 Refer the cell 35 from the Matrix and Find root Information Take next root as 4 or 5 Consider : 4 Cost of Tree And main root

35. Empty matrix structure and formula 012345 10.250.600.751.00 20.100.350.600.85 30.050.150.300.55 40.050.200.45 50.050.30 60.10 0.50 0.45 012345 10.10 0.45/1 1.00/21.30/21.80/22.70/2 20.100.50 0.80/2 1.30/22.25/5 3 0.050.25/30.60/41.25/4,5 4 0.050.30/40.85/5 5 0.050.45/5 60.10 k2 K3,K4,K5 K1 k3 k1 k3k5

36. Empty matrix structure and formula 012345 10.10 0.45/ 1 1.00 /2 1.30 /2 1.80 /2 2.70 /2 20.100.50 0.80/ 2 1.30 /2 2.25 /5 3 0.050.25/ 3 0.60/ 4 1.25/ 4,5 4 0.050.30/ 4 0.85/ 5 5 0.050.45/ 5 60.10 k2 K3,K4,K5 K1 k3 k1 k3k5 i012345 pi0.050.200.050.100.15 qi0.10 0.05 0.10 sum0.100.150.250.100.150.25 do d1 d2 d3 d4 d5 Verification: 1x0.20 = 0.20 2x(0.05+0.05) = 0.20 3x(0.10+0.10) + 3x(0.05+0.15)=1.20 4x(0.05+0.05+0.05+0.10) = 1.00 TOTAL = 2.70 L1 L2 L3 L4 pi qi