1. Markov Chains & Randomized algorithms BY HAITHAM FALLATAH FOR COMP4804

2. Outline  History Of Markov chains  The Markov property  Markov Chains Example  3 MC models and applications  Analysis of a randomized algorithm using MC

3. History of the Markov chain  Andrey Markov was a Russian mathematician  Lived in 19 th and early 20 th century  He was bad at school in everything but math  Student of Chebyshev  was motivated to work on a model that was later known as the MC because of 2 reasons.  Show that Chebyshev’s approach to extending the Law of Large numbers to sums of dependent random variables could be improved.  He had an animosity with another Russian mathematician called Nekrasov who claimed that only independent events can converge on predictable distributions

4. The Markov property  Markov disproved Nekrasov’s claim by coming up with a model that has states and transitions.  Pick a ball at random, if Blue, pick a ball from Bin 1 otherwise from Bin 2  Probability will converge on a predictable distribution 1:1 Red:Blue 2:1 Red:Blue 1/3 2/3 1/2 Bin 1Bin 2 1/2

5. Markov Chain Example SunnyCloudyRainySnowy Sunny0.70.300 Cloudy0.2 0.40.2 Rainy00.30.50.2 Snowy00.20.10.7 SunnyCloudyRainySnowy 0.1610.2420.260.335 Transition Matrix P 100 matrix for gen(100) SunnyCloudyRainySnowy 1000 P 0 matrix for initial state

6. Markov Models & applications  Discreet time Markov chain (DTMC)  It is the notion that there is a chain of steps between states where movement from a state to the next, depends only on the current state.  Pr(X n+1 = x | X 1 = x 1, X 2 = x 2, ….., X n = x n ) = Pr(X n+1 = x | X n = x n )  This process is used with simulations and analysing random algorithms

7. Markov Models & applications  Continues Time Markov chain (CTMC)  Similar to the DTMC model except that steps have no meaning in continuous time so the Monrovian property must hold for all future times instead of just for one step. This model is best suited for modeling biological and physical systems because of their continues time nature.  Used in construction of phylogenies (evolutionary trees)

8. Markov Models & applications  Hidden Markov chain model  It is a model in which the “real Markov chain” is hidden and instead, another set of states is observed and is dependent on the states of the hidden chain. This is one of the most powerful Markov models because it can fill in the blanks in problems where not all the information is given.  This model is used in pattern, speech, and writing recognition Z1Z1 Z2Z2 Z3Z3 ZnZn X1X1 X2X2 X3X3 XnXn

9. Analysis a randomized algorithm  One example of a random algorithm that can be analysed by the Markov chain model, is a randomized algorithm that solves the 2- SAT problem.  The 2-SAT problem consists of n Boolean variables and m clauses that are in a 2-conjunctive normal form. This problem is solved when all clauses have a value of true and thus the overall result of the statement would be true. This problem is considered in P (polynomial) as many algorithms can solve it in polynomial time.

10. The Problem  We are given an algorithm that takes O(n 2 ) steps to find a satisfiable assignment. Our job is to prove independently using the Markov chain that this claim is true.  The algorithm runs as follows (pseudo code):  Repeat the following 2mn 2 times  Select a clause that’s not satisfied (ie. False) at random  Select one of the variables in the clause at random and flip its value  If the formula is satisfied, return true  Otherwise, return that this formula is unsatisfiable.

11. Example of Algorithm Execution X 1 = True X 2 = False X 3 = False X 4 = False X 1 = False X 2 = False X 3 = False X 4 = False X 1 = True X 2 = False X 3 = True X 4 = False X 1 = True X 2 = False X 3 = False X 4 = False

12. Analysis

13. j-1 jj+1n 0 1/2

14. Analysis

15.  Thus:

16. Analysis  The System of equations we have is:  Using Induction we can get:

17. Analysis  Finally, we can substitute and get:

18. References  Probability and Computing (book) by Michael mitzenmacher and Eli Upfal (Main)  https://www.coursera.org/ https://www.coursera.org/  http://en.wikipedia.org/wiki/Markov_chain http://en.wikipedia.org/wiki/Markov_chain  Markov Models and Hidden Markov Models: A Brief Tutorial  Discrete Mathematics & Mathematical Reasoning by Kousha Etessami  http://www.nimbios.org/tutorials/TT_stochastic_modeling_talks/NIMBioST ut_LJSA_2.pdf  https://www.youtube.com/watch?v=TPRoLreU9lA https://www.youtube.com/watch?v=TPRoLreU9lA  https://www.youtube.com/watch?v=7KGdE2AK_MQ https://www.youtube.com/watch?v=7KGdE2AK_MQ  Khan Academy