1. Ecocomic Modelling: PG1 Markov Chain and Its Use in Economic Modelling Markov process Transition matrix Convergence Likelihood function Expected values and Policy Decision

2. Ecocomic Modelling: PG2 A stochastic processhas the Markov process if for all and all t A Markov process is characterised by three elements:

3. Ecocomic Modelling: PG3 A typical Transition matrix

4. Ecocomic Modelling: PG4 Chapman-Kolmogorov Equations

5. Ecocomic Modelling: PG5 Likelihood Function for a Markov Chain Two uses of likelihood function to study alternative histories of a Markov Chain to estimate the parameter

6. Ecocomic Modelling: PG6 Convergence of Markov Process with Finite States Reference: Stokey and Lucas (page 321) A Markov Process Converges when each element of the of the transition matrix approaches to a limit like this. Process is stationary in this example.

7. Ecocomic Modelling: PG7 Recurrent or absorbing State or Transient State in a Markov Chain S1 is the recurrent state whenever the process leaves, re-enters in it and stays there forever. It is transient when it does not return to S1 when it leaves it. Here S1 is the recurrent state whenever the process leaves, re-enters in it. S2 and S3 are transient.

8. Ecocomic Modelling: PG8 Converging and Non-converging Sequences Even Odd

9. Ecocomic Modelling: PG9 One Example of Markov Chain Stochastic life cycle optimisation model (preliminary version of Bhattarai and Perroni) Probability of recurrent state Prob of Transient state If transient High incomeLow income Probability of being in Ambiguous state

10. Ecocomic Modelling: PG10 Impact of Risk Aversion and Ambiguity in Expected Wealth with Markov Process

11. Ecocomic Modelling: PG11 Markov Decision problem (refer Ross (187)).

12. Ecocomic Modelling: PG12 Markov perfect equilibrium is the pair of value functions and a pair of policy functions for i=1,2 that satisfies the above Bellman equation. Equilibrium is computed by backward induction and he optimising behaviours of firms by iterating forward for all conceivable future states. Use of Markov Chain in analysis of Duopoly Sargent and Ljungqvist (133)

13. Ecocomic Modelling: PG13 Other Application of Markov Process Regime -Switch analysis in economic time series (Hamilton pp. 677-699; Harvey (285)) Industry investment under uncertainty (SL chap 10) Stochastic dynamic programming (SL chapter 8,9) Weak and strong convergence analysis (SLChap 11-13) Arrow Securities (Ljungqvist and Sargent Chapter 7). Life cycle consumption and saving: An example Precautionary saving

14. Ecocomic Modelling: PG14 References:

15. Ecocomic Modelling: PG15 Markov Chain Example in GAMS

16. Ecocomic Modelling: PG16 Markov Chain Example in GAMS

17. Ecocomic Modelling: PG17 Markov Chain Example in GAMS: Model Equations variables u(t,z) c(t,z) v(t,z) w(t,z) obj; equations defu(t,z) defc(t,z) defwl(t,z) defwn(t,z) defwt(t,z) dobj;

18. Ecocomic Modelling: PG18 Calculation of Weight Among Various States