1. Random Number Generators

2. Why do we need random variables? random components in simulation → need for a method which generates numbers that are random examples –interarrival times –service times –demand sizes 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 2

3. terminology Random Numbers –stochastic variable that meets certain conditions –numbers randomly drawn from some known distribution –It is impossible to “generate” them with the help of a digital computer. Pseudo-random Numbers –numbers which seem to be randomly drawn from some known distribution –may be generated with the help of a digital computer 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 3

4. techniques for generating random variables –ten-sided die (each throw generates a decimal) –throwing a coin n times get a binary number between 0 and 2^n-1 –other physical devices –observe substances undergoing atomic decay e.g., Caesium-137, Krypton-85,... 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 4

5. techniques for generating pseudo-random variables numbers are generated using a recursive formula –r i is a function of r i-1, r i-2, … –relation is deterministic – no random numbers –if the mathematical relation not known and well chosen it is practically impossible to predict the next number In most cases we want the generated random numbers to simulate a uniform distribution over (0, 1), that is a U(0, 1)-distribution –there’re many simple techniques to transform uniform (U(a, b)) samples into samples from other well-known distributions 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 5 1 01 01

6. terminology seed –random number generators are usually initialized with a starting number which is called the “seed” –different seeds generates different streams of pseudorandom numbers –the same seed results in the same stream cycle length –length of the stream of pseudorandom numbers without repetition 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 6

7. methods for generating pseudo-random variables all methods usually generate uniformly distributed pseudorandom numbers in the interval [0,1] –midsquare method (outdated) –congruential method (popular) –shift register (Tausworthe generator) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 7

8. midsquare method arithmetic generator –proposed by von Neumann and Metropolis in 1940s algorithm –start with an initial number Z 0 with m digits (seed) [m even] –square it and get a number with 2m digits (add a zero at the beginning if the number has only 2m-1 digits) –obtain Z 1 by taking the middle m digits –to be in the interval [0,1) divide the Z i by 10 m –etc…. 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 8

9. midsquare method: example (Z 0 = 7182, m = 4) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 9 i01234567…i01234567… ZiZi 7182 5811 7677 9363 6657 3156 9603 2176 … UiUi 0.7182 0.5811 0.7677 0.9363 0.6657 0.3156 0.9603 0.2176 … Zi2Zi2 51581124 33767721 58936329 87665769 44315649 9960336 92217609 4734976 …

10. midsquare method (cont.) advantages –fairly simple disadvantage –tends to degenerate fairly rapidly to zero example: try Z 0 = 1009 –not random as predictable 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 10

11. Linear Congruential Generator (LCG) LCGs –first proposed by Lehmer (1951). –produce pseudorandom numbers such that each single number determines its successor by means of a linear function followed by a modular reduction –to be in the interval [0,1) divide the Z i by m –Z 0 seedamultiplier (integer) –cincrement (integer)m modulus (integer) Variations are possible –combinations of previous numbers instead of using only the last value 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 11

12. LCG: example 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 12 i01234567…i01234567… ZiZi 7 6 1 8 11 10 5 12 … UiUi 0.4375 0.375 0.0625 0.5 0.6875 0.625 0.3125 0.75 … a = 5 c = 3 m = 16 Z 0 = 7 Z 1 = (5*Z 0 + 3) mod 16 = 38 mod 16 = 6 Z 2 = (5*Z 1 + 3) mod 16 = 33 mod 16 = 1 Z 3 = (5*Z 2 + 3) mod 16 = 8 mod 16 = 8 …

13. properties of LCGs parameters need to be chosen carefully –nonnegative –0 < ma < m c < mZ 0 < m disadvantages –not random –if a and m are properly chosen, the U i s will “look like” they are randomly and uniformly distributed between 0 and 1. –can only take rational values 0, 1/m, 2/m, … (m-1)/m –looping 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 13

14. LCG (cont.) looping –whenever Z i takes on a value it has had previously, exactly the same sequence of values is generated, and this cycle repeats itself endlessly. –length of sequence = period of generator –period can be at most m if so: LCG has full period –LCG has full period iff the only positive integer that divides both m and c is 1 if q is a prime number that divides m then q divides a-1 if 4 divides m, then 4 divides a-1 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 14

15. Linear Feedback Shift Register Generators (LFSR) developed by Tausworthe (1965) –related to cryptographic methods –operate directly on bits to form random numbers linear combination of the last q bits –c i constants (equal to 0 or 1, c q = 1) –in most applications only two of the c j coefficients are nonzero –execution can be sped up modulo 2 equivalent to exclusive-or 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 15

16. LFSR (cont.) form a sequence of binary integers W 1, W 2, … –string together l consecutive bits –consider them as numbers in base 2 transform them into U i s maximum period2 q -1 –if l is relatively prime to 2 q -1 LFSR has full period 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 16

17. LFSR : example r = 3q = 5b 1 = b 2 = b 3 = b 4 = b 5 = 1l = 4 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 17 I123456789101112131415 1718 bibi 111110001101110101 i123 W i (base 2) 111110001101 W i (base 10) 15813 Ui 0.93750.50.8125

18. Testing RNGs Empirical Tests 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 18

19. random number generators methods presented so far… –completely deterministic –U i s appear as if they were U(0,1) test their actual quality –how well do the generated U i s resemble values of true IID U(0,1) different kinds of tests –empirical tests based on actual U i s produced by RNG –theoretical tests 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 19

20. Chi-Square test (with all parameters known) checks whether the U i s appear to be IID U(0,1) –divide [0,1] into k subintervals of equal length (k > 100) –generate n random variables: U 1, U 2,.. U n –calculate f j (number of U i s that fall into jth subinterval) –calculate test statistic –for large n χ 2 will have an approximate chi-square distribution with k-1 df under the null hypothesis that the U i s are IID U(0,1) –reject null hypothesis at level ® if 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 20

21. Generating Random Variates

22. General Approach –Inverse Transformation –Composition –Convolution Generating Continuous Random Variates –Uniform U(a,b) –Exponential –m-Erlang –Gamma, Weibull –Normal –etc… 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 22

23. Inverse Transformation generate continuous random variate X –distribution function F (continuous, strictly increasing) 0 < F(x) < 1 if x 1 < x 2 then 0 < F(x 1 ) · F(x 2 ) < 1 –inverse of F: F -1 algorithm –generate U ~ U(0,1) –return X = F -1 (U) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 23

24. Inverse Transformation (cont.) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 24 x F(x) 0 1 U1U1 X1X1 X2X2 U2U2 returned value X has desired distribution F

25. Inverse Transformation (cont.) create X according to exponential distribution with mean ¯ in order to find F -1 set 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 25 take natural logarithm (base e)

26. Inverse Transformation for discrete variates distribution function probability mass function algorithm –generate U ~ U(0,1) –determine smallest possible integer i such that U · F(x i ) –return X = x i 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 26

27. Composition applies then the distribution function F can be expressed as a convex combination of other distribution functions F 1, F 2,.. –we hope to be able to sample from the F j ’s more easily than from the original F –if X has a density it can be written as algorithm –generate positive random integer J such that P(J = j) = p j –Return X with distribution function F J 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 27

28. Composition (example) double-exponential distribution (laplace distribution) –can be rewritten as –where I A (x) is the indicator function of set A f(x) is a convex combination of –f 1 (x) = e x I (- 1, 0) and f 2 (x) = e -x I [0, 1 ) –p 1 = p 2 = 0.5 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 28

29. Convolution desired random variable X can be expressed as sum of other random variables that are IID and can be generated more readily then X directly algorithm –generate Y 1, Y 2, … Y m IID each with distribution function G –return X = Y 1 + Y 2 +  + Y m example: m-Erlang random variable X with mean ¯ –sum of m IID exponential random variables with common mean ¯ /m 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 29

30. Generate Continuous Variates Uniform distribution X(a,b) X = a + (b-a)U Exponential (mean ¯ ) X = - ¯ ln (1-U)orX = - ¯ ln U m-Erlang 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 30