Download presentation

Presentation is loading. Please wait.

Published byRoxana Easterbrook Modified over 2 years ago

1
Dynamic Programming Lecture 13 (5/21/2014)

2
- A Forest Thinning Example - 1 260 2 650 3 535 4 410 5 850 6 750 7 650 8 600 9 500 10 400 11 260 0 50 100 0 150 200 850 750 650 600 500 400 0 0 100 175 150 75 Stage 1 Stage 2 Stage 3 Age 10 Age 20 Age 30 Age 10 Source: Dykstra’s Mathematical Programming for Natural Resource Management (1984)

3
Dynamic Programming Cont. Stages, states and actions (decisions) Backward and forward recursion

4
Solving Dynamic Programs Recursive relation at stage i (Bellman Equation):

5
Dynamic Programming Structural requirements of DP –The problem can be divided into stages –Each stage has a finite set of associated states (discrete state DP) –The impact of a policy decision to transform a state in a given stage to another state in the subsequent stage is deterministic –Principle of optimality: given the current state of the system, the optimal policy for the remaining stages is independent of any prior policy adopted

6
Examples of DP The Floyd-Warshall Algorithm (used in the Bucket formulation of ARM):

7
Examples of DP cont. Minimizing the risk of losing an endangered species (non-linear DP): Source: Buongiorno and Gilles’ (2003) Decision Methods for Forest Resource Management $2M $1M $0 $1M $2M Stage 1Stage 2Stage 3 $2M $0 $1M $0 $2M $0 $1M $0 $2M $1M $0

8
Minimizing the risk of losing an endangered species (DP example) Stages (t=1,2,3) represent $ allocation decisions for Project 1, 2 and 3; States (i=0,1,2) represent the budgets available at each stage t; Decisions (j=0,1,2) represent the budgets available at stage t+1; denotes the probability of failure of Project t if decision j is made (i.e., i-j is spent on Project t); and is the smallest probability of total failure from stage t onward, starting in state i and making the best decision j*. Then, the recursive relation can be stated as:

9
Markov Chains (based on Buongiorno and Gilles 2003) Transition probability matrix P Vector p t converges to a vector of steady-state probabilities p*. p* is independent of p 0 !

10
Markov Chains cont. Berger-Parker Landscape Index Mean Residence Times Mean Recurrence Times

11
Markov Chains cont. Forest dynamics (expected revenues or biodiversity with vs. w/o management:

12
Markov Chains cont. Present value of expected returns

13
Markov Decision Processes

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google