Download presentation

Presentation is loading. Please wait.

Published byZoe Ballard Modified over 2 years ago

1
Random Processes Markov Chains Professor Ke-Sheng Cheng Department of Bioenvironmental Systems Engineering

2
In other words, the probability of any particular future behavior of the process, when its current state is known exactly, is not altered by additional knowledge concerning its past behavior.

3
It is customary to label the state space of the Markov chain by the nonnegative integers {0, 1, 2, …} and to use X n =i to represent the process being in state i at time (or stage) n.

4
The notation emphasizes that in general the transition probabilities are functions not only of the initial and final states, but also of the time of transition as well.

5

6

7
The (i+1)st row of P is the probability distribution of the values of X n+1 under the condition that X n =i. If the number of states is finite, then P is a finite square matrix whose order (the number of rows) is equal to the number of states.

8

9

10

11
It is also evident that for all time points n, m and all states i 0, …, i n, j 1, …, j m.

12

13

14

15

16

17
Transition probability matrices of a Markov chain

18

19
Chapman- Kolmogorov equations

20

21

22

23

24

25

26

27

28

29

30

31

32

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google