Dynamic Programming

A Simple Example

Capital as a State Variable

The Value Function

The Functional Equation

Computing the Value Function

The standard approach for computing value functions is known as “iteration on the value function” 1.To proceed, first specify functional forms and parameter values for the primitives of the problem: utility functions, production functions, the discount factor, etc. 2.Then set up a grid of possible values of the state variable: in cases where the state variable is continuous (such as the capital accumulation problem) this results in an approximate solution rather than an exact one, but this is usually good enough 3.The next step is to choose an initial value function; this sounds difficult but in practice it usually doesn’t matter too much what you start with: setting V(k) = 0 for every k on the grid is a typical choice

Iteration on the Value Function

Mathematical Foundations We would like our model to yield a unique solution for V(.) (a unique function, not a point) This means no matter which initial value function we choose at the start of the iterations, the final value function is always the same Similarly, we would like g(.) to be single-valued (a function instead of a correspondence) Then, given an initial state, the model would yield a unique prediction for the entire path of capital accumulation Whether the model yields unique solutions for the value function and policy function depends on the primitives of the problem: utility functions, production functions, etc. Stokey and Lucas describe general sufficient conditions for a model to yield a unique value function and a unique policy function (these conditions are sufficient, not necessary)

General Notation

Assumptions

Key Results

The Content of the Assumptions

The Assumptions