1. Random-Number Generation

2. 2 Properties of Random Numbers Random Number, R i, must be independently drawn from a uniform distribution with pdf: Two important statistical properties:  Uniformity  Independence. Figure: pdf for random numbers U(0,1)

3. 3 Generation of Pseudo-Random Numbers “Pseudo”, because generating numbers using a known method removes the potential for true randomness. Goal: To produce a sequence of numbers in [0,1] that simulates, or imitates, the ideal properties of random numbers (RN). Important considerations in RN routines:  Fast  Portable to different computers  Have sufficiently long cycle  Replicable  Closely approximate the ideal statistical properties of uniformity and independence.

4. 4 Linear Congruential Method [Techniques] To produce a sequence of integers, X 1, X 2, … between 0 and m-1 by following a recursive relationship: The selection of the values for a, c, m, and X 0 drastically affects the statistical properties and the cycle length. The random integers are being generated [0,m-1], and to convert the integers to random numbers: The multiplier The increment The modulus

5. 5 Examples [LCM] Use X 0 = 27, a = 17, c = 43, and m = 100. The X i and R i values are: X 1 = (17*27+43) mod 100 = 502 mod 100 = 2,R 1 = 0.02; X 2 = (17*2+43) mod 100 = 77, R 2 = 0.77; X 3 = (17*77+43) mod 100 = 52, R 3 = 0.52; X 4 = (17* 52 +43) mod 100 = 27, R 3 = 0.27 “Numerical Recipes in C” advocates the generator a = 1664525, c = 1013904223, and m = 2 32 Classical LCG’s can be found on http://random.mat.sbg.ac.at

6. 6 Characteristics of a Good Generator [LCM] Maximum Density  Such that the values assumed by R i, i = 1,2,…, leave no large gaps on [0,1]  Problem: Instead of continuous, each R i is discrete  Solution: a very large integer for modulus m Approximation appears to be of little consequence Maximum Period  To achieve maximum density and avoid cycling.  Achieve by: proper choice of a, c, m, and X 0. Most digital computers use a binary representation of numbers  Speed and efficiency are aided by a modulus, m, to be (or close to) a power of 2.

7. 7 A Good LCG Example X=2456356; %seed value for i=1:10000, X=mod(1664525*X+1013904223,2^32); U(i)=X/2^32; end edges=0:0.05:1; M=histc(U,edges); bar(M); hold; figure; hold; for i=1:5000, plot(U(2*i-1),U(2*i)); end

8. 8 Randu (from IBM, early 1960’s) X=1; %seed value for i=1:10000, X=mod(65539*X+57,2^31); U(i)=X/2^31; end edges=0:0.05:1; M=histc(U,edges); bar(M); hold; figure; hold; for i=1:3333, plot3(U(3*i-2),U(3*i-1),U(3*i)); end Marsaglia Effect (1968)

9. 9 Random-Numbers Streams [Techniques] The seed for a linear congruential random-number generator:  Is the integer value X 0 that initializes the random-number sequence.  Any value in the sequence can be used to “seed” the generator. A random-number stream:  Refers to a starting seed taken from the sequence X 0, X 1, …, X P.  If the streams are b values apart, then stream i could defined by starting seed:  Older generators: b = 10 5 ; Newer generators: b = 10 37. A single random-number generator with k streams can act like k distinct virtual random-number generators To compare two or more alternative systems.  Advantageous to dedicate portions of the pseudo-random number sequence to the same purpose in each of the simulated systems.

10. Random-Variate Generation

11. 11 Purpose & Overview Develop understanding of generating samples from a specified distribution as input to a simulation model. Illustrate some widely-used techniques for generating random variates.  Inverse-transform technique  Convolution technique  Acceptance-rejection technique  A special technique for normal distribution

12. 12 Inverse-transform Technique The concept: For cdf function: r = F(x)  Generate R sample from uniform (0,1)  Find X sample: X = F -1 (R) r1r1 x1x1 r = F(x)

13. 13 Exponential Distribution [Inverse-transform] Exponential Distribution:  Exponential cdf:  To generate X 1, X 2, X 3,… generate R 1, R 2, R 3,… r = F(x) = 1 – e - x for x  0 X i = F -1 (R i ) = -(1/  ln(1-R i ) Figure: Inverse-transform technique for exp( = 1)

14. 14 Example: Generate 200 variates X i with distribution exp( = 1) Matlab Code for i=1:200, expnum(i)=-log(rand(1)); end Exponential Distribution [Inverse-transform] R and (1 – R) have U(0,1) distribution

15. 15 Uniform Distribution:  Uniform cdf:  To generate X 1, X 2, X 3,…, generate R 1, R 2, R 3,… Uniform Distribution [Inverse-transform]

16. 16 Example: Generate 500 variates X i with distribution Uniform (3,8) Matlab Code for i=1:500, uninum(i)=3+5*rand(1); end Uniform Distribution [Inverse-transform]

17. 17 Discrete Distribution [Inverse-transform] All discrete distributions can be generated by the Inverse-transform technique. p1p1 p 1 + p 2 p 1 + p 2 + p 3 a b c F(x)F(x) R1R1 General Form

18. 18 Discrete Distribution [Inverse-transform] Example: Suppose the number of shipments, x, on the loading dock of IHW company is either 0, 1, or 2  Data - Probability distribution:  Method - Given R, the generation scheme becomes: Consider R 1 = 0.73: F(x i-1 ) < R <= F(x i ) F(x 0 ) < 0.73 <= F(x 1 ) Hence, x 1 = 1

19. 19 Convolution Technique Use for X = Y 1 + Y 2 + … + Y n Example of application  Erlang distribution Generate samples for Y 1, Y 2, …, Y n and then add these samples to get a sample of X.

20. 20 Example: Generate 500 variates X i with distribution Erlang-3 (mean: k/  Matlab Code for i=1:500, erlnum(i)=-1/6*(log(rand(1))+log(rand(1))+log(rand(1))); end Erlang Distribution [Convolution]

21. 21 Useful particularly when inverse cdf does not exist in closed form a.k.a. thinning Steps to generate X with pdf f(x)  Step 0: Identify a majorizing function g(x) and a pdf h(x) satisfying  Step 1: Generate Y with pdf h(x)  Step 2: Generate U ~ Uniform(0,1) independent of Y  Step 3: Acceptance-Rejection technique Generate Y,U Condition Output X=Y yes no Efficiency parameter is c

22. 22 Triangular Distribution [Acceptance-Rejection] Generate Y~U(0,0.5) and U~U(0,1) Output X = Y yes no

23. 23 Triangular Distribution [Acceptance-Rejection] Matlab Code: (for exactly 1000 samples) i=0; while i<1000, Y=0.5*rand(1); U=rand(1); if Y 0.25 & U<=2-4*Y i=i+1; X(i)=Y; end

24. – N can be interpreted as number of arrivals from a Poisson arrival process during one unit of time – Then, the time between the arrivals in the process are exponentially distributed with rate  Poisson Distribution [Acceptance-Rejection]

25. Step 1. Set n = 0, and P = 1 Step 2. Generate a random number R n+1 and let P = P. R n+1 Step 3. If P < e - , then accept N = n. Otherwise, reject current n, increase n by one, and return to step 2 How many random numbers will be used on the average to generate one Poisson variate? Poisson Distribution [Acceptance-Rejection]

26. 26 Approach for normal(0,1):  Consider two standard normal random variables, Z 1 and Z 2, plotted as a point in the plane:  B 2 = Z 2 1 + Z 2 2 ~ chi-square distribution with 2 degrees of freedom = Exp( = 1/2). Hence,  The radius B and angle  are mutually independent. Normal Distribution [Special Technique] In polar coordinates: Z 1 = B cos  Z 2 = B sin   Uniform(0,2 

27. 27 Normal Distribution [Special Technique] Approach for normal( ,   ):  Generate Z i ~ N(0,1) Approach for lognormal( ,   ):  Generate X ~ N( ,   ) Y i = e X i X i =  +  Z i

28. 28 Normal Distribution [Special Technique] Generate 1000 samples of Normal(7,4) Matlab Code for i=1:500, R1=rand(1); R2=rand(1); Z(2*i-1)=sqrt(-2*log(R1))*cos(2*pi*R2); Z(2*i)=sqrt(-2*log(R1))*sin(2*pi*R2); end for i=1:1000, Z(i)=7+2*Z(i); end