1. CPSC 531: RN Generation1 CPSC 531:Random-Number Generation Instructor: Anirban Mahanti Office: ICT 745 Email: mahanti@cpsc.ucalgary.ca Class Location: TRB 101 Lectures: TR 15:30 – 16:45 hours Class web page: http://pages.cpsc.ucalgary.ca/~mahanti/teaching/F05/CPSC531 http://pages.cpsc.ucalgary.ca/~mahanti/teaching/F05/CPSC531 Notes from “Discrete-event System Simulation” by Banks, Carson, Nelson, and Nicol, Prentice Hall, 2005. Slides derived from the textbook’s website.

2. CPSC 531: RN Generation2 Properties of Random Numbers r Two important statistical properties: m Uniformity m Independence. r Random Number, R i, must be independently drawn from a uniform distribution with pdf: Figure: pdf for random numbers

3. CPSC 531: RN Generation3 Generation of Pseudo-Random Numbers r “Pseudo”, because generating numbers using a known method removes the potential for true randomness. r Goal: To produce a sequence of numbers in [0,1] that simulates, or imitates, the ideal properties of random numbers (RN). r Important considerations in RN routines: m Fast m Portable to different computers m Have sufficiently long cycle m Replicable m Closely approximate the ideal statistical properties of uniformity and independence.

4. CPSC 531: RN Generation4 Techniques for Generating Random Numbers r Linear Congruential Method (LCM). r Combined Linear Congruential Generators (CLCG). r Random-Number Streams.

5. CPSC 531: RN Generation5 Linear Congruential Method r To produce a sequence of integers, X 1, X 2, … between 0 and m-1 by following a recursive relationship: r The selection of the values for a, c, m, and X 0 drastically affects the statistical properties and the cycle length. r The random integers are being generated [0,m-1], and to convert the integers to random numbers: The multiplier The increment The modulus

6. CPSC 531: RN Generation6 LCM Example r Use X 0 = 27, a = 17, c = 43, and m = 100. r The X i and R i values are: X 1 = (17*27+43) mod 100 = 502 mod 100 = 2,R 1 = 0.02; X 2 = (17*2+32) mod 100 = 77, R 2 = 0.77; X 3 = (17*77+32) mod 100 = 52, R 3 = 0.52; …

7. CPSC 531: RN Generation7 Characteristics of a Good Generator r Maximum Density m Such that he values assumed by R i, i = 1,2,…, leave no large gaps on [0,1] m Problem: Instead of continuous, each R i is discrete m Solution: a very large integer for modulus m Approximation appears to be of little consequence r Maximum Period m To achieve maximum density and avoid cycling. m Achieve by: proper choice of a, c, m, and X 0. r Most digital computers use a binary representation of numbers m Speed and efficiency are aided by a modulus, m, to be (or close to) a power of 2.

8. CPSC 531: RN Generation8 Combined Linear Congruential Generators r Reason: Longer period generator is needed because of the increasing complexity of stimulated systems. r Approach: Combine two or more multiplicative congruential generators. r Let X i,1, X i,2, …, X i,k, be the i th output from k different multiplicative congruential generators. m The j th generator: Has prime modulus m j and multiplier a j and period is m j-1 Produces integers X i,j is approx ~ Uniform on integers in [1, m-1] W i,j = X i,j -1 is approx ~ Uniform on integers in [1, m-2]

9. CPSC 531: RN Generation9 Combined Linear Congruential Generators m Suggested form: The maximum possible period is: The coefficient: Performs the subtraction X i,1-1

10. CPSC 531: RN Generation10 Combined Linear Congruential Generators r Example: For 32-bit computers, L’Ecuyer [1988] suggests combining k = 2 generators with m 1 = 2,147,483,563, a 1 = 40,014, m 2 = 2,147,483,399 and a 2 = 20,692. The algorithm becomes: Step 1: Select seeds X 1,0 in the range [1, 2,147,483,562] for the 1 st generator X 2,0 in the range [1, 2,147,483,398] for the 2 nd generator. Step 2: For each individual generator, X 1,j+1 = 40,014 X 1,j mod 2,147,483,563 X 2,j+1 = 40,692 X 1,j mod 2,147,483,399. Step 3: X j+1 = (X 1,j+1 - X 2,j+1 ) mod 2,147,483,562. Step 4:Return Step 5: Set j = j+1, go back to step 2. m Combined generator has period: (m 1 – 1)(m 2 – 1)/2 ~ 2 x 10 18

11. CPSC 531: RN Generation11 Random-Numbers Streams r The seed for a linear congruential random-number generator: m Is the integer value X 0 that initializes the random-number sequence. m Any value in the sequence can be used to “seed” the generator. r A random-number stream: m Refers to a starting seed taken from the sequence X 0, X 1, …, X P. m If the streams are b values apart, then stream i could defined by starting seed: m Older generators: b = 10 5 ; Newer generators: b = 10 37. r A single random-number generator with k streams can act like k distinct virtual random-number generators r To compare two or more alternative systems. m Advantageous to dedicate portions of the pseudo-random number sequence to the same purpose in each of the simulated systems.

12. CPSC 531: RN Generation12 Tests for Random Numbers r Two categories: m Testing for uniformity: H 0 : R i ~ U[0,1] H 1 : R i ~ U[0,1] Failure to reject the null hypothesis, H 0, means that evidence of non-uniformity has not been detected. m Testing for independence: H 0 : R i ~ independently H 1 : R i ~ independently Failure to reject the null hypothesis, H 0, means that evidence of dependence has not been detected.  Level of significance  the probability of rejecting H 0 when it is true:  = P(reject H 0 |H 0 is true) / /

13. CPSC 531: RN Generation13 Tests for Random Numbers r When to use these tests: m If a well-known simulation languages or random-number generators is used, it is probably unnecessary to test m If the generator is not explicitly known or documented, e.g., spreadsheet programs, symbolic/numerical calculators, tests should be applied to many sample numbers. r Types of tests: m Theoretical tests: evaluate the choices of m, a, and c without actually generating any numbers m Empirical tests: applied to actual sequences of numbers produced. Our emphasis.

14. CPSC 531: RN Generation14 Frequency Tests r Test of uniformity r Two different methods (already discussed …) m Kolmogorov-Smirnov test (look at this again) m Chi-square test

15. CPSC 531: RN Generation15 Kolmogorov-Smirnov Test r Example: Suppose 5 generated numbers are 0.44, 0.81, 0.14, 0.05, 0.93. Step 1: Step 2: Step 3: K = max(K +, K - ) = 0.26 Step 4: For  = 0.05, K  α = 0.565 > K Hence, H 0 is not rejected. Arrange R (j) from smallest to largest K+K+ K-K- R (j) 0.050.140.440.810.93 j/N0.200.400.600.801.00 j/N – R (j) 0.150.260.16-0.07 R (j) – (j-1)/N0.05-0.040.210.13

16. CPSC 531: RN Generation16 Tests for Autocorrelation r Testing the autocorrelation between every m numbers (m is a.k.a. the lag), starting with the i th number  The autocorrelation  im between numbers: R i, R i+m, R i+2m, R i+(M+1)m m M is the largest integer such that r Hypothesis: r If the values are uncorrelated:  For large values of M, the distribution of the estimator of  im, denoted is approximately normal.

17. CPSC 531: RN Generation17 Tests for Autocorrelation r Test statistics is: m Z 0 is distributed normally with mean = 0 and variance = 1, and:  If  im > 0, the subsequence has positive autocorrelation m High random numbers tend to be followed by high ones, and vice versa.  If  im < 0, the subsequence has negative autocorrelation m Low random numbers tend to be followed by high ones, and vice versa.

18. CPSC 531: RN Generation18 Autocorrelation Example r Test whether the 3 rd, 8 th, 13 th, and so on, for the following output on P. 265.  Hence,  = 0.05, i = 3, m = 5, N = 30, and M = 4 m From Table A.3, z 0.025 = 1.96. Hence, the hypothesis is not rejected.

19. CPSC 531: RN Generation19 Autocorrelation test: Shortcomings r The test is not very sensitive for small values of M, particularly when the numbers being tests are on the low side. r Problem when “fishing” for autocorrelation by performing numerous tests:  If  = 0.05, there is a probability of 0.05 of rejecting a true hypothesis. m If 10 independence sequences are examined, The probability of finding no significant autocorrelation, by chance alone, is 0.95 10 = 0.60. Hence, the probability of detecting significant autocorrelation when it does not exist = 40%

20. CPSC 531: RN Generation20 Summary r We learnt: m Generation of random numbers m Testing for uniformity and independence r Caution: m Even with generators that have been used for years, some of which still in used, are found to be inadequate. m This chapter provides only the basic m Also, even if generated numbers pass all the tests, some underlying pattern might have gone undetected.