1. Design and Analysis of Algorithms Lecture Dynamic programming Alexander Kolesnikov 17.10.205 DEPARTMENT OF COMPUTER SCIENCE UNIVERSITY OF JOENSUU JOENSUU, FINLAND

2. List of sample problems Shortest path in trellis graph; Shortest path in trellis graph; Optimal allocation of constrained resource; Optimal allocation of constrained resource; Optimal sequence partition (k-link shortest path). Optimal sequence partition (k-link shortest path). to be continued...

3. Shortest path in trellis graph

4. Stagecoach problem A traveler wishes to minimize the length of a journey A J from town A to J.

5. Greedy algorithm The length of the route A-B-F-I-J: 2+4+3+4=13. Can we find shorter route?

6. Exhaustive search: try all Route A-D-F-I-J: 3+1+3+4=11 The total number of routes to be tested: 3  3  2  1=18 Can we avoid exhaustive search?

7. Shortest path construction: 1st stage (B) ?

8. Shortest path construction: 1st stage S(A,B)=2 S(A,C)=4 S(A,D)=3

9. Shortest path construction: 2nd stage (E) 1. (A..B)-E: 2+7=9 2. (A..C)-E: 4+3=7 3. (A..D)-E: 3+4=7 ---------------------- (A..C)-E: 7 ?

10. Shortest path construction: 2nd stage (E) 1. (A..B)-E: 2+7=9 2. (A..C)-E: 4+3=7 *) 3. (A..D)-E: 3+4=7 ---------------------- (A..C)-E: 7

11. Shortest path construction: 2nd stage (F) 1. (A..B)-F: 2+4=6 2. (A..C)-F: 4+2=6 3. (A..D)-F: 3+1=4 *) ---------------------- (A..C)-F: 4 ?

12. Shortest path construction: 2nd stage (F) 1. (A..B)-F: 2+4=6 2. (A..C)-F: 4+2=6 3. (A..D)-F: 3+1=4 *) ---------------------- (A..D)-F: 4

13. Shortest path construction: 2nd stage (G) 1. (A..B)-G: 2+6=8 *) 2. (A..C)-G: 4+6=10 3. (A..D)-G: 3+5=8 ---------------------- (A..B)-G: 8 ?

14. Shortest path construction: 2nd stage (G) 1. (A..B)-G: 2+6=8 *) 2. (A..C)-G: 4+6=10 3. (A..D)-G: 3+5=8 ---------------------- (A..B)-G: 8

15. Shortest path construction: 3rd stage (H) 1. (A..E)-H: 7+1=8 *) 2. (A..F)-H: 4+6=10 3. (A..G)-H: 5+3=8 ---------------------- (A..E)-H: 5

16. Shortest path construction: 3rd stage (H) 1. (A..E)-H: 7+1=8 *) 2. (A..F)-H: 4+6=10 3. (A..G)-H: 5+3=8 ---------------------- (A..E)-H: 5

17. Shortest path construction: 3rd stage (I) 1. (A..E)-I: 7+4=11 2. (A..F)-I: 4+3=7 *) 3. (A..G)-I: 5+3=8 ---------------------- (A..F)-I: 7

18. Shortest path construction: 3rd stage (I) 1. (A..E)-I: 7+4=11 2. (A..F)-I: 4+3=7 *) 3. (A..G)-I: 5+3=8 ---------------------- (A..F)-I: 7

19. Shortest path construction: 4th stage (J) 1. (A..H)-J: 8+3=11 *) 2. (A..I) -J: 7+4=11 ---------------------- (A..H)-J: 11

20. Shortest path construction: 4th stage (J) 1. (A..H)-J: 8+3=11 *) 2. (A..I) -J: 7+4=11 ---------------------- (A..H)-J: 11

21. Backtrack the shortest path 11 8 7 8 4 7 3 4 2 0

22. The shortest path 11 8 7 8 4 7 3 4 2 0 Route A-C-E-H-J: 4+3+1+3=11

23. Trellis graph 1 2 3 4 K-1 K

24. Trellis graph Distance (weight) from point i 1 at stage (j  1) to point i 2 at stage j : The total value of cost function:

25. Principle of optimality of Bellman An optimal path has the property that whatever the initial conditions and control variables (choices) over some initial period, the control (or decision variables) chosen over the remaining period must be optimal for the remaining problem, with the state resulting from the early decisions taken to be the initial condition.

26. Dynamic programming Initialization: Recursive eqution: Cost function:

27. Complexity Exhaustive search: O(n K ) Dynamic programming algorithm: O(Kn 2 ) where K is the number of stages, n is the number of points in a stage

28. Optimal allocation of constrained resource

29. Problem formulation N units of a resource; This resource must be distributed among K activities; Functions f k (x) - profit for allocated resource; Allocate N units of resource to K activities with given return functions so that the total profit is maximal: subject to:

30. Dynamic programming formulation Initialization: Recursive equation: Optimal value function:

31. Allocate 3 mln euros into four projects Profit f k (x), K=3, N=3.

32. Trellis graph

33. Solution #1: 2; f 1 (2)=8 #2: 1; f 2 (1)=5 #3: 0; f 3 (0)=0 #4: 0; f 4 (0)=0 ---------------------- N=3; G 4 (3)=13 *)

34. Search in the state space K N0 0 n k j G k (n) G k-1 (j) G K (N) Start state f k (n-j ) G k (n) = max{G k (0) + f k (n), G k (1) + f k (n-1),... G k (j-1)+ f k (j),... G k (n) + f k (0)} k-1 * A k (n)=j opt 1

35. Optimal partition of data sequence

36. Problem formulation Given a sequence of data X={x 1, x 2, …,x N } Do partition of the sequence X into to K groups with given cost functions f(x i,x j ) so that the total value of the cost function is minimal:

37. Partition into groups: Example Data: x 0 =  < x 1 <... < x j <... < x N Partition indices: i 0 = 0 < i 1 <... < i j <... < i M =N. Groups:... ]( ] ( #1 #2 #3 (x 0 =  ) x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 x 9 x 10 x 11 x 12 x 13 x 14 x N (i 0 =0) i 1 =4 i 2 =10 i K =N=15 K=3

38. Problem formulation Recursive equation: Cost function: Initialization:

39. Search in the state space  K N1 0 b n k j State space  G k (n) G k-1 (j) G K (N) Start state f(j,n ) G k (n) = min{G k (k) + f(k, n], G k (k) + f(k+1,n],... G k (j-1) + f(j, n]* ),... G k (n-1) + f(n, n]} k-1 * A k (n)=j opt

40. Scheme of the DP algorithm // Initialization G 1 (n) FOR n = 1 TO N DO G 1 (n)= f(1,n] // Minimum search FOR k = 2 TO K DO FOR n = k TO N DO d min   FOR j= k-1 TO n-1 DO G k-1 (j) c  G k-1 (j) + f(j,n] IF(c < c min ) c min  c; j min  j ENDIF ENDFOR G k (n) G k (n)  d min A k (n) A k (n)  j min ENDFOR Complexity: O(KN 2 )

41. Backtrack in the state space  K N1 0 b nj State space  A K (N) Start state S(M+1)= N FOR m = K+1 TO 2 DO S(m  1) = A(S(m), m)) G K (N) P = G K (N) S(M+1)= N FOR m = K+1 TO 2 DO S(m  1) = A(S(m), m)) G K (N) P = G K (N) N=22, K=8: S={22,18,14,12,9,6,4,3,1} (x 0,x 3 ], (x 3,x 4 ], (x 4,x 6 ], (x 6,x 9 ], (x 9,x 12 ], (x 12,x 14 ], (x 14,x 18 ], (x 18,x 22 ]