Download presentation

Presentation is loading. Please wait.

Published byLouise Gardner Modified over 4 years ago

1
Randomized Algorithms Kyomin Jung KAIST Applied Algorithm Lab Jan 12, WSAC 2010 1

2
Randomness in Algorithms A Probabilistic Turing Machine is a TM given with a (binary) random “coin”. In the computation process of the PTM, PTM can toss a coin to decide its decision. Ex) Computer games, random screen savor… Note: in computer programming, the random number generating function (ex, rand() in C) takes an initial random seed, and generates a “deterministic” sequence of numbers. Hence they are not “truly” random. 2

3
Why randomness can be helpful? A Simple example Suppose we want to check whether an integer set has an even number or not. Even when A has n/2 many even numbers, if we run a Deterministic Algorithm, it may check n/2+1 many elements in the worst case. A Randomized Algorithm: At each time, choose an elements (to check) at random. Smooths the “worst case input distribution” into “randomness of the algorithm” 3

4
Property of “independent” random trials Law of Large Number If we repeat independent random samplings many times according to a fixed distribution D, the “the average becomes close to the expectation” (ex: dice rolling) Ex) Erdos-Renyi Random Graph G(n,p). Voting poll … 4

5
Suppose we have a coin with probability of heads is p. Let X be a random variable where X=1 if the coin flip gives heads and X=0 otherwise. E[X] = 1*p + 0*(1-p) = p After flipping a coin w times we can estimate the heads prob by average of x i. The Chernoff Bound tells us that this estimate converges exponentially fast to the true mean p. Chernoff Bound 5

6
Las Vegas vs Monte Carlo A Las Vegas algorithm is a randomized algorithm that always gives correct results Ex: randomized quick sort A Monte Carlo algorithm is one whose running time is deterministic, but whose output may be correct only with a certain probability. Class BPP (Bounded-error, Probabilistic, Polynomial time) problem which has a Monte Carlo algorithm to solve it 6

7
Randomized complexity classes Model: probabilistic Turing Machine BPP L BPP if there is a p.p.t. TM M: x L Pr y [M(x,y) accepts] ≥ 2/3 x L Pr y [M(x,y) rejects] ≥ 2/3 “p.p.t” = probabilistic polynomial time y : (a sequence of) random coin tosses 7

8
Randomized complexity classes RP (Random Polynomial-time) L RP if there is a p.p.t. TM M: x L Pr y [M(x,y) accepts] ≥ ½ x L Pr y [M(x,y) rejects] = 1 coRP (complement of Random Polynomial-time) L coRP if there is a p.p.t. TM M: x L Pr y [M(x,y) accepts] = 1 x L Pr y [M(x,y) rejects] ≥ ½ 8

9
Error reduction for BPP given L, and p.p.t. TM M: x L Pr y [M(x,y) accepts] ≥ ½ + ε x L Pr y [M(x,y) rejects] ≥ ½ + ε new p.p.t. TM M’: simulate M k/ ε 2 times, each time with independent coin flips accept if majority of simulations accept otherwise reject 9

10
Error reduction for BPP X i random variable indicating “correct” outcome in i-th simulation (out of m = k/ ε 2 ) Pr[X i = 1] ≥ ½ + ε Pr[X i = 0] ≤ ½ - ε E[X i ] ≥ ½+ ε X = Σ i X i μ = E[X] ≥ (½ + ε )m By Chernoff Bound: Pr[X ≤ m/2] ≤ 2 - ( ε 2 m) 10

11
Derandomization Question: is BPP=P? Goal: try to simulate BPP in polynomial time use Pseudo-Random Generator (PRG): Good if t=O(log m) Blum-Micali-Yao PRG seed length t = m δ seedoutput string G t bitsm bits 11

12
Example of randomized algorithm : Minimum Cut Problem A B C D Note: Size of the min cut must is no larger than the smallest node degree in graph 12

13
Application: Internet Minimum Cut 13

14
Randomized Algorithm (by David Karger) While |V| > 2: Pick a random edge (x, y) from E Contract the edge: Keep multi-edges, remove self-loops Combine nodes The two remaining nodes represent reasonable choices for the minimum cut sets 14

15
Analysis Suppose C is a minimum cut (set of edges that disconnects G) When we contract edge e: Unlikely that e C So, C is likely to be preserved What is the probability a randomly chosen edge is in C? 15

16
Random Edge in C? |C| must be degree of every node in G How many edges in G: |E| = sum of all node degrees / 2 n |C| / 2 Probability a random edge is in C 2/n 16

17
Iteration How many iterations? Probability for first iteration: Prob(e 1 C) 1 – 2/n Probability for second iteration: Prob(e 2 C | e 1 C) 1 – 2/(n-1)... Probability for last iteration: Prob(e n-2 C |…) 1 – 2/(n-(n-2-1)) 1 – 2/3 n - 2 17

18
Probability of finding C? Probability of not finding C = 1 – 2/(n(n-1)) 18

19
Probability of finding C? Probability of not finding C on one trial: 1 – 2/(n(n-1)) 1 – 2/n 2 Probability of not finding C on k trials: [1 – 2/n 2 ] k If k = cn 2, Prob failure (1/e) c Recall: lim (1 – 1/x) x = 1/e x 19

20
Example of randomized algorithm : Random Sampling What is a random sampling? Given a probability distribution, pick a point according to. e.g. Monte Carlo method for integration Choose numbers uniformly at random from the integration domain, and sum up the value of f at those points 20

21
How to use Random Sampling? Volume computation in Euclidean space. 21

22
Hit and Run Hit and Run algorithm is used to sample from a convex set in an n-dimensional Eucliden space. Assume that we can evaluate its ratios at given points It converges in time. 22

23
Example of randomized algorithm : Markov Chain Monte Carlo Let be a probability density function on S={1,2,..n}. f( ‧ ) is any function on S and we want to estimate Construct P ={P ij }, the transition matrix of an irreducible Markov chain with states S, where and π is its unique stationary distribution. 23

24
Markov Chain Monte Carlo Run this Markov chain for times t=1,…,N and calculate the Monte Carlo sum then 24

25
Applied Algorithm Lab ( http://aa.kaist.ac.kr/ ) Research Projects Markov Random Field and Image segmentation Network consensus algorithm design Community detection in complex networks Routing/scheduling algorithm analysis in wirel ess networks Motion surveillance in video images Character recognition in images SAT solver design based on formula structures Decentralized control algorithms for multi-agent rob ot system Financial data mining Yongsub LimBoyoung Kim ProfessorGraduate Students Kyomin JungNam-ju Kwak Members

Similar presentations

© 2020 SlidePlayer.com Inc.

All rights reserved.

To make this website work, we log user data and share it with processors. To use this website, you must agree to our Privacy Policy, including cookie policy.

Ads by Google