Markov chains

Probability distributions

Exercise 1.Use the Matlab function nchoosek(n,k) to implement a generic function BinomialPMF(k,n,p) for calculating the Binomial PMF with k successes in n trials with probability p. 2.Use the barplot function to plot the pmf values (k=0..10) for Bin(10,0.5) as below:

Markov chains

Example: A very simple weather model (modified from Wikipedia page Examples of Markov chains) The probabilities of weather conditions, modeled as either sunny=0 or rainy=1, given the weather on the preceding day, can be represented by a transition matrix

Weather model example The matrix P represents the weather model in which a sunny day is 90% likely to be followed by another sunny day, and a rainy day is 50% likely to be followed by another rainy day. The columns can be labelled sunny and rainy respectively, and the rows can be labelled in the same order. P ij is the probability that, if a given day is of type i, it will be followed by a day of type j. Note that the rows of P add up to 1: sum(P)

Weather model example

Exercise If the the weather on the first day has 50% probability of being sunny or rainy, then what is the probability of the ninth day being sunny? What about the 30 th day?

Weather model example

Eigenvectors / Eigenvalues

We can calculate eigenvalues and eigenvectors in Matlab using the built-in function eig. The default behavior is for right eigenvalues/eigenvectors, but left eigenvalues/eigenvectors are easily obtained by transposing the matrix. We will be using the form (from Matlab help for eig): [V,D] = eig(A) produces matrices of eigenvalues (D) and eigenvectors (V) of matrix A, so that A*V = V*D. Matrix D is the canonical form of A a diagonal matrix with As eigenvalues on the main diagonal. Matrix V is the modal matrix - its columns are the eigenvectors of A. Note that Matlab always returns eigenvectors with norm 1.

Exercise 1.Use the eig function to find out, in the long term, what is the percentage of sunny days in the weather example.