1. Chapter 10 Dynamic Programming

2. 2 Agenda for This Week Dynamic Programming –Definition –Recursive Nature of Computations in DP –Forward and Backward Recursion –Basic Elements of DP –Applications of DP

3. 3 Dynamic Programming An approach to problem solving that decomposes a large problem into a number of smaller problems for finding optimal solution Used for problems that are in multistage in nature No standard mathematical formulation

4. 4 Stagecoach Problem Used to illustrate the features of DP Concerns a mythical fortune seeker to go to west to join the gold rush in California during mid-19 th century A prudent man who was quite concerned about his safety Could not go through some territories Starting and final destination points are fixed Had considerable choice as to which states to travel

5. 5 Cost Estimates 567 2106- 3986 4647 234 1435 89 559 648 775 3 2 7 6 5 9 8 4 1 4 3 5 6 8 9 6 7 4 6 5 4 9 5 8 7 6 4 86 94

6. 6 Solving Problem Select routes that minimizes total cost One possible approach is to use trail and error method DP starts with a small portion of the original problem and finds the optimal solution Optimizes the nest stage using the solution from preceding one Continues until all stages are solved First decompose it into stages as delineated by the vertical dashed lines 3 2 7 6 5 9 8 10 4 1 4 3 5 6 8 9 6 7 4 6 5 4 9 5 8 7 6 4

7. 7 Recursive Nature of Computations in DP Start with the smaller problem where he has completed his journey and has only one to go Computations are done recursively Solution of one subproblem is used as an input for next subproblem 3 2 7 6 5 9 8 10 4 1 4 3 5 6 8 9 6 7 4 6 5 4 9 5 8 7 6 4

8. 8 DP Concept Stages are formed by decomposing the original problem into a number of subproblems Shortest path context –Input— any node or state –Decision Problem— which arc? –Decision criteria—shortest distance –Output– any node or state Decision Problem Decision Criteria Input Output

9. 9 DP Notation X n =input to stage n (output from stage n+1) d n =decision variable at stage n (state variable) Stage transformation function Return function—return of a stage dNdN XNXN Stage N X N-1 XnXn X n-1 X0X0 X1X1 d1d1 dndn Stage nStage 1 r N (x N,d N )r n (x n,d n )r 1 (x 1,d 1 )

10. 10 Stage one First stage begins with nodes that are exactly one arc away from final destination (CA) Nodes 8 and 9 are the input nodes or states for stage 1 Nodes 10 is the output nodes for stage 1 3 2 7 6 5 9 8 10 4 1 4 3 5 6 8 9 6 7 4 6 5 4 9 5 8 7 6 4

11. 11 Decomposing the Problem-Stage two Second stage begins with all nodes that are exactly two arcs away from destination and ends with all nodes that are exactly one arc away Nodes 5, 6, and 7 are the input nodes for stage 2 Nodes 8 and 9 are the output nodes for stage 2 3 2 7 6 5 9 8 10 4 1 4 3 5 6 8 9 6 7 4 6 5 4 9 5 8 7 6 4

12. 12 Decomposing the Problem-Stage three Third stage begins with all nodes that are exactly three arcs away from destination and ends with all nodes that are exactly two arcs away Nodes 2, 3, and 4 are the input nodes for stage 3 Nodes 5, 6 and 7 are the output nodes for stage 3 3 2 7 6 5 9 8 10 4 1 4 3 5 6 8 9 6 7 4 6 5 4 9 5 8 7 6 4

13. 13 Decomposing the Problem-Stage Four Forth stage begins with all nodes that are exactly four arcs away from destination and ends with all nodes that are exactly three arcs away Nodes 1 the input node for stage 4 Nodes 2, 3 and 4 are the output nodes for stage 3 3 2 7 6 5 9 8 10 4 1 4 3 5 6 8 9 6 7 4 6 5 4 9 5 8 7 6 4 Stage 1 Stage 4

14. 14 Solving the Problem-Stage one 10 9 8 6 4 6 4 InputArcOutputShortest to 10 8989 8-10 9-10 6464 10

15. 15 Solving by DP Notation-Stage 1 X 1 =input to stage 1 (output from stage 2) d 1 =decision variable at stage 1 (state variable) Stage transformation function Return function—return of a stage f 1 (x 1 )= r 1 (x 1,d 1 ) d1x1d1x1 8989 10 6464 d1*d1* dndn XnXn Stage n X n-1 r n (x n,d n ) 10 9 8 6 4 6 4

16. 16 Solving the Problem- Stage two 10 9 8 4 6 4 InputArcOutputShortest 6 5 6 7 567567 5-8 6-8 7-9 11 10 9 5 9 4 8 7 5 889889 11 10 9 Min [(5+6),(9+4)]=11 Min [(4+6),(8+4)]=10 Min [(7+6),(5+4)]= 9

17. 17 Solving by DP Notation-Stage 2 r 2 (x 2,d 2 )+ f 1 (x 1 ) d2x2d2x2 567567 89 11 10 13 12 9 d2*d2* 11 10 9 f 2 (x 2 ) 889889 10 9 8 4 6 4 6 5 6 7 5 9 4 8 7 5 11 10 9

18. 18 Solving the Problem- Stage Three 10 9 8 4 6 4 InputArcOutputShortest 6 5 6 7 234234 2-6 3-7 4-6 16 15 14 5 9 4 8 7 5 676676 11 10 9 2 3 4 6 9 8 6 6 4 7 16 15 14 Min [(10+11),(6+10)]=16 Min [(9+11),(8+10), (6+9)]=15 Min [(11+6),(10+4),(7+9)]= 14

19. 19 Solving by DP Notation-Stage 3 r 3 (x 3,d 3 )+ f 2 (x 2 ) d3x3d3x3 234234 56 21 20 17 16 18 14 d3*d3* 16 15 14 f 3 (x 3 ) 676676 7 - 15 16 10 9 8 4 6 4 6 5 6 7 5 9 4 8 7 5 11 10 9 2 3 4 6 9 8 6 6 4 7 16 15 14

20. 20 Solving the Problem- Stage Four 10 9 8 4 6 4 InputArcOutputShortest 6 5 6 7 118 5 9 4 8 7 5 3 11 10 9 2 3 4 6 9 8 6 6 4 7 16 15 14 1 4 3 5 1-3 18 Min [(4+16),(3+15), (5+14)]=18

21. 21 Solving by DP Notation-Stage 4 r 4 (x 4,d 4 )+ f 3 (x 3 ) d4x4d4x4 1 23 2018 d3*d3* f 4 (x 4 ) 3 4 19 10 9 8 4 6 4 6 5 6 7 5 9 4 8 7 5 11 10 9 2 3 4 6 9 8 6 6 4 7 16 15 14 1 4 3 5 18

22. 22 Best Decision at Each stage Stage 4 3 2 1 Arc 1-3 3-7 7-9 9-10 3 2 7 6 5 9 8 10 4 1 4 3 5 6 8 9 6 7 4 6 5 4 9 5 8 7 6 4

23. 23 Dynamic EOQ Models Dynamic EOQ models are different, because –Inventory level is reviewed periodically over a number finite periods –Demand per period, though deterministic, is dynamic (MRP situation) No shortage is allowed and setup cost is incurred each time a production lot is started

24. 24 A Production and Inventory Control Problem Making decision on a production quantity for each of periods so that the demand can be met at a minimum cost –N=no. of periods (stages) –D n =demand during stage n –X n = amount of inventory on hand at beginning of stage n –d n = production quantity for stage n –P n =production capacity in stage n –W n =storage capacity at the end of stage –C n =production cost per unit in stage –H n =holding cost per unit of ending inventory for stage n

25. 25 Example Capacity Cost/unit Month Demand Production Storage ProductionHolding 123123 233233 323323 232232 175 150 200 30 40 Beginning inventory for month 1is 1 units

26. 26 DP Formulation Month 1Month 2Month 3 D3=2 P3=3 W3=2 d3=? P2=2 D2=3 W2=3 d2=? D1=3 P1=3 W1=2 d1=? X3=1X1 X2 X0X0 r 3 (x 3,d 3 )r 2 (x 2,d 2 )r 1 (x 1,d 1 ) Note: Ending Inventory= Beginning Inventory +Production -Demand

27. 27 Knapsack Problem N items can be put into a knapsack Each time has a certain weight and a value Determine how many units of each item to place in the knapsack to maximize the total value Constraint is placed on the maximum weight Similar to a manger who must make a biweekly selection of jobs from each of four categories to process during a 2-week period

28. 28 Job Data for the Manufacturing Operation Select certain jobs during the next 2 weeks such that all the jobs selected can be processed within 10 working days and the total value of the jobs is maximized Job Category No. of Jobs to be processed Estimated Completion time per job Value rating per job 1412 2338 32411 42720

29. 29 Formulation At stage 1, we must decide how many jobs from category 1 to process At stage 2, we must decide how many jobs from category 2 to process, etc. Value rating is a subjective score (scale 1 to 20) x n =No. of days of processing time remaining at the beginning of stage n (state variable) d n =No. of jobs processed from category n (decision variable at stage n)