Chapter 10 Dynamic Programming

2 Agenda for This Week Dynamic Programming –Definition –Recursive Nature of Computations in DP –Forward and Backward Recursion –Basic Elements of DP –Applications of DP

3 Dynamic Programming An approach to problem solving that decomposes a large problem into a number of smaller problems for finding optimal solution Used for problems that are in multistage in nature No standard mathematical formulation

4 Stagecoach Problem Used to illustrate the features of DP Concerns a mythical fortune seeker to go to west to join the gold rush in California during mid-19 th century A prudent man who was quite concerned about his safety Could not go through some territories Starting and final destination points are fixed Had considerable choice as to which states to travel

5 Cost Estimates

6 Solving Problem Select routes that minimizes total cost One possible approach is to use trail and error method DP starts with a small portion of the original problem and finds the optimal solution Optimizes the nest stage using the solution from preceding one Continues until all stages are solved First decompose it into stages as delineated by the vertical dashed lines

7 Recursive Nature of Computations in DP Start with the smaller problem where he has completed his journey and has only one to go Computations are done recursively Solution of one subproblem is used as an input for next subproblem

8 DP Concept Stages are formed by decomposing the original problem into a number of subproblems Shortest path context –Input— any node or state –Decision Problem— which arc? –Decision criteria—shortest distance –Output– any node or state Decision Problem Decision Criteria Input Output

9 DP Notation X n =input to stage n (output from stage n+1) d n =decision variable at stage n (state variable) Stage transformation function Return function—return of a stage dNdN XNXN Stage N X N-1 XnXn X n-1 X0X0 X1X1 d1d1 dndn Stage nStage 1 r N (x N,d N )r n (x n,d n )r 1 (x 1,d 1 )

10 Stage one First stage begins with nodes that are exactly one arc away from final destination (CA) Nodes 8 and 9 are the input nodes or states for stage 1 Nodes 10 is the output nodes for stage

11 Decomposing the Problem-Stage two Second stage begins with all nodes that are exactly two arcs away from destination and ends with all nodes that are exactly one arc away Nodes 5, 6, and 7 are the input nodes for stage 2 Nodes 8 and 9 are the output nodes for stage

12 Decomposing the Problem-Stage three Third stage begins with all nodes that are exactly three arcs away from destination and ends with all nodes that are exactly two arcs away Nodes 2, 3, and 4 are the input nodes for stage 3 Nodes 5, 6 and 7 are the output nodes for stage

13 Decomposing the Problem-Stage Four Forth stage begins with all nodes that are exactly four arcs away from destination and ends with all nodes that are exactly three arcs away Nodes 1 the input node for stage 4 Nodes 2, 3 and 4 are the output nodes for stage Stage 1 Stage 4

14 Solving the Problem-Stage one InputArcOutputShortest to

15 Solving by DP Notation-Stage 1 X 1 =input to stage 1 (output from stage 2) d 1 =decision variable at stage 1 (state variable) Stage transformation function Return function—return of a stage f 1 (x 1 )= r 1 (x 1,d 1 ) d1x1d1x d1*d1* dndn XnXn Stage n X n-1 r n (x n,d n )

16 Solving the Problem- Stage two InputArcOutputShortest Min [(5+6),(9+4)]=11 Min [(4+6),(8+4)]=10 Min [(7+6),(5+4)]= 9

17 Solving by DP Notation-Stage 2 r 2 (x 2,d 2 )+ f 1 (x 1 ) d2x2d2x d2*d2* f 2 (x 2 )

18 Solving the Problem- Stage Three InputArcOutputShortest Min [(10+11),(6+10)]=16 Min [(9+11),(8+10), (6+9)]=15 Min [(11+6),(10+4),(7+9)]= 14

19 Solving by DP Notation-Stage 3 r 3 (x 3,d 3 )+ f 2 (x 2 ) d3x3d3x d3*d3* f 3 (x 3 )

20 Solving the Problem- Stage Four InputArcOutputShortest Min [(4+16),(3+15), (5+14)]=18

21 Solving by DP Notation-Stage 4 r 4 (x 4,d 4 )+ f 3 (x 3 ) d4x4d4x d3*d3* f 4 (x 4 )

22 Best Decision at Each stage Stage Arc

23 Dynamic EOQ Models Dynamic EOQ models are different, because –Inventory level is reviewed periodically over a number finite periods –Demand per period, though deterministic, is dynamic (MRP situation) No shortage is allowed and setup cost is incurred each time a production lot is started

24 A Production and Inventory Control Problem Making decision on a production quantity for each of periods so that the demand can be met at a minimum cost –N=no. of periods (stages) –D n =demand during stage n –X n = amount of inventory on hand at beginning of stage n –d n = production quantity for stage n –P n =production capacity in stage n –W n =storage capacity at the end of stage –C n =production cost per unit in stage –H n =holding cost per unit of ending inventory for stage n

25 Example Capacity Cost/unit Month Demand Production Storage ProductionHolding Beginning inventory for month 1is 1 units

26 DP Formulation Month 1Month 2Month 3 D3=2 P3=3 W3=2 d3=? P2=2 D2=3 W2=3 d2=? D1=3 P1=3 W1=2 d1=? X3=1X1 X2 X0X0 r 3 (x 3,d 3 )r 2 (x 2,d 2 )r 1 (x 1,d 1 ) Note: Ending Inventory= Beginning Inventory +Production -Demand

27 Knapsack Problem N items can be put into a knapsack Each time has a certain weight and a value Determine how many units of each item to place in the knapsack to maximize the total value Constraint is placed on the maximum weight Similar to a manger who must make a biweekly selection of jobs from each of four categories to process during a 2-week period

28 Job Data for the Manufacturing Operation Select certain jobs during the next 2 weeks such that all the jobs selected can be processed within 10 working days and the total value of the jobs is maximized Job Category No. of Jobs to be processed Estimated Completion time per job Value rating per job

29 Formulation At stage 1, we must decide how many jobs from category 1 to process At stage 2, we must decide how many jobs from category 2 to process, etc. Value rating is a subjective score (scale 1 to 20) x n =No. of days of processing time remaining at the beginning of stage n (state variable) d n =No. of jobs processed from category n (decision variable at stage n)