1.  How does the computer generate observations from various distributions specified after input analysis?  There are two main components to the generation.

Slides:



Advertisements
Similar presentations
RSLAB-NTU Lab for Remote Sensing Hydrology and Spatial Modeling 1 An Introduction to R Pseudo Random Number Generation (PRNG) Prof. Ke-Sheng Cheng Dept.
Advertisements

Mathematics of Cryptography Part II: Algebraic Structures
Generating Random Numbers
Random Number Generation. Random Number Generators Without random numbers, we cannot do Stochastic Simulation Most computer languages have a subroutine,
Session 2: Secret key cryptography – stream ciphers – part 2.
Random Numbers. Two Types of Random Numbers 1.True random numbers: True random numbers are generated in non- deterministic ways. They are not predictable.
Random Number Generators. Why do we need random variables? random components in simulation → need for a method which generates numbers that are random.
Session 2 Symmetric ciphers 1. Stream cipher definition Recall the Vernam cipher: Plaintext Ciphertext (Running) key
Pseudorandom Number Generators
Random Number Generation
Distinguishing Features of Simulation Time (CLK)  DYNAMIC focused on this aspect during the modeling section of the course Pseudorandom variables (RND)
Properties of Random Numbers
Session 2: Secret key cryptography – stream ciphers – part 1.
1 Chapter 7 Generating and Processing Random Signals 第一組 電機四 B 蔡馭理 資工四 B 林宜鴻.
APPENDIX D RANDOM NUMBER GENERATION
Random-Number Generation. 2 Properties of Random Numbers Random Number, R i, must be independently drawn from a uniform distribution with pdf: Two important.
Random Number Generation Fall 2013
Fall 2011 CSC 446/546 Part 6: Random Number Generation.
Random Number Generation Pseudo-random number Generating Discrete R.V. Generating Continuous R.V.
The Rational Zero Theorem
ETM 607 – Random Number and Random Variates
The Fundamental Theorem of Algebra And Zeros of Polynomials
Chapter 9 Mathematics of Cryptography Part III: Primes and Related Congruence Equations Copyright © The McGraw-Hill Companies, Inc. Permission required.

Slide 5-1 Copyright © 2005 Pearson Education, Inc. SEVENTH EDITION and EXPANDED SEVENTH EDITION.
MATH 224 – Discrete Mathematics
The Rational Zero Theorem The Rational Zero Theorem gives a list of possible rational zeros of a polynomial function. Equivalently, the theorem gives all.
Bell Ringer 1. What is the Rational Root Theorem (search your notebook…Unit 2). 2. What is the Fundamental Theorem of Algebra (search your notebook…Unit.
Random-Number Generation Andy Wang CIS Computer Systems Performance Analysis.
CPE 619 Random-Number Generation Aleksandar Milenković The LaCASA Laboratory Electrical and Computer Engineering Department The University of Alabama in.
CS433 Modeling and Simulation Lecture 15 Random Number Generator Dr. Anis Koubâa 24 May 2009 Al-Imam Mohammad Ibn Saud Islamic University College Computer.
© 2003, Carla Ellis Simulation Techniques Overview Simulation environments emulation exec- driven sim trace- driven sim stochastic sim Workload parameters.
CPSC 531: RN Generation1 CPSC 531:Random-Number Generation Instructor: Anirban Mahanti Office: ICT Class Location:
Chapter 7 Random-Number Generation
Module 1: Statistical Issues in Micro simulation Paul Sousa.
Basic Concepts in Number Theory Background for Random Number Generation 1.For any pair of integers n and m, m  0, there exists a unique pair of integers.
Modeling and Simulation Random Number Generators
Cyclic Codes for Error Detection W. W. Peterson and D. T. Brown by Maheshwar R Geereddy.
Copyright © 2013, 2009, 2005 Pearson Education, Inc. 1 3 Polynomial and Rational Functions Copyright © 2013, 2009, 2005 Pearson Education, Inc.
Session 1 Stream ciphers 1.
APPENDIX D R ANDOM N UMBER G ENERATION Organization of chapter in ISSO* – General description and linear congruential generators Criteria for “good” random.
Linear Feedback Shift Register. 2 Linear Feedback Shift Registers (LFSRs) These are n-bit counters exhibiting pseudo-random behavior. Built from simple.
Copyright © 2009 Pearson Education, Inc. Chapter 5 Section 1 - Slide 1 Chapter 1 Number Theory and the Real Number System.
Random Number Generator. Random number Random number: Uniform distribution on [0,1] Random device: dice, coin -> cannot generate the equal sequence.
9/22/15UB Fall 2015 CSE565: S. Upadhyaya Lec 7.1 CSE565: Computer Security Lecture 7 Number Theory Concepts Shambhu Upadhyaya Computer Science & Eng. University.
1 © 2010 Pearson Education, Inc. All rights reserved © 2010 Pearson Education, Inc. All rights reserved Chapter 3 Polynomial and Rational Functions.
Pseudo-random-number generators. Random numbers: the numbers generated by a stochastic process They are indispensible in any simulations based on radom.
9.1 Primes and Related Congruence Equations 23 Sep 2013.
R ANDOM N UMBER G ENERATORS Modeling and Simulation CS
0 Simulation Modeling and Analysis: Input Analysis 7 Random Numbers Ref: Law & Kelton, Chapter 7.
Chapter 3 Generating Uniform Random Variables. In any kind of simulation, we need data, or we have to produce them. Especially in Monte Marco simulation.
MONTE CARLO METHOD DISCRETE SIMULATION RANDOM NUMBER GENERATION Chapter 3 : Random Number Generation.
Slide Copyright © 2009 Pearson Education, Inc. Slide Copyright © 2009 Pearson Education, Inc. Chapter 1 Number Theory and the Real Number System.
Random Numbers All stochastic simulations need to “generate” IID U(0,1) “random numbers” Other random variates coming from other distribution can be generated.
UNIFORM RANDOM NUMBER GENERATION Chapter 7 (first half) The goal is to generate a sequence of The goal is to generate a sequence of Uniformly distributed.
3.3 Dividing Polynomials.
Trigonometric Identities
CSE565: Computer Security Lecture 7 Number Theory Concepts
Random Numbers All stochastic simulations need to “generate” IID U(0,1) “random numbers” Other random variates coming from other distribution can be generated.
Generating Random Numbers
Random Number Generators
Random-Number Generation
Chapter 7 Random Number Generation
Chapter 7 Random-Number Generation
Properties of Random Numbers
Random Number and Random Variate Generation
Computer Simulation Techniques Generating Pseudo-Random Numbers
Random Number Generation
Presentation transcript:

1

 How does the computer generate observations from various distributions specified after input analysis?  There are two main components to the generation of observations from probability distributions. 1.Random number generation. 2.Random variate generation. 2

 Random number generation – The generation of U(0,1) random variates (observations from Uniform (0,1) distribution). ◦ This serves as the foundation for the generation of observations from other distributions, which is called random variate generation.  Random Number Generator is the term used to describe the procedure and parameters used to generate the U(0,1) observations. 3

 Since the “stream” of random numbers generated is reproducible, random number generation procedures are also referred to as pseudo random number generators.  The stream or sequence of numbers produced by a generator should pass statistical tests for randomness. ◦ An outside observer should not be able to tell the difference (statistically) between a stream of pseudo random numbers and an actual random number stream. 4

 A pseudorandom process appears random, but isn’t  Pseudorandom sequences exhibit statistical randomness ◦ but generated by a deterministic process  Pseudorandom sequences are easier to produce than a genuine random sequences  Pseudorandom sequences can reproduce exactly the same numbers ◦ useful for testing and fixing software. 5

 Random number generators typically compute the next number in the sequence from the previous number  The first number in a sequence is called the seed ◦ to get a new sequence, supply a new seed (current machine time is useful) ◦ to repeat a sequence, repeat the seed 6

7 l Desirable Attributes: ◦ Uniformity ◦ Independence ◦ Efficiency ◦ Replicability ◦ Long Cycle Length

8 Each random number R t is an independent sample drawn from a continuous uniform distribution between 0 and 1  1, 0  x  1 pdf: f(x) =   0, otherwise

9 x f(x) 0 1 PDF:

 One early method – the midsquare method (von Neumann and Metropolis 1940) ◦ Start with a four digit positive integer Z 0. ◦ Square Z 0 to get an integer with up to eight digits (append zeros if less than eight). ◦ Take the middle four digits as the next four digit integer Z 1. ◦ Place a decimal point to the left of Z 1 to form the first “U(0,1)” observation. ◦ Repeat 10

11 MidSquare Example: X 0 = 7182 (seed) = ==> R 1 = = (5811) 2 = ==> R 2 = etc.

12 Note: Cannot choose a seed that guarantees that the sequence will not degenerate and will have a long period. Also, zeros, once they appear, are carried in subsequent numbers. Ex1: X 0 = 5197 (seed) = ==> R 1 = = ==> R 2 = Ex2: X 0 = 4500 (seed) = ==> R 1 = = ==> R 2 =

 The prior method does not work well. ◦ Degenerates to zero.  What are good methods? ◦ Linear Congruential Generators (LCGs). ◦ Composite generators. ◦ Tausworthe generators. 13

 Linear Congruential Generators (LCGs).  A LCG generates a sequence of integers Z 1, Z 2, Z 3,… using the following recursive formula,  mod m is short for modulo m or the remainder when divided by m. 14

 Since the mod m operation is used, all Z i ’s will be between 0 and m-1.  To get the “U(0,1)” random observations each Z i generated is divided by m.  So are the U i ’s really U(0,1) random observations ? 15

 Let m=63, a=22, c=4 and Z 0 =19. ◦ Generate the first five “U(0,1)” observations. 16

17

 What will happen after the 63 rd number is generated?  m, a, and c are the parameters of the random number generator. ◦ There can be an infinite number of different implementations of a LCG.  The values used for m, a, and c determine whether the generator is good or bad. 18

 The example LCG demonstrates cycling in the prior table. ◦ Since m=63, it can generate at most 63 numbers before it repeats the same sequence.  This small random number generator has full period since it generates all possible (m=63) numbers before cycling. ◦ A long period (full if possible) is desirable since more observations can be generated before cycling. ◦ No “gaps”. 19

 The example generator has full period but bad statistical properties (next slide).  A good random number generator will have values for m, a, and c such that full or close to full period is obtained, as well as good statistical properties. ◦ Crystal Ball  m = 2 31 – 1  a =  c = 0  Period = 2 31 – 2 20

 Theorem (Hull and Dobell 1962) ◦ The LCG Z i = (aZ i-1 + c) mod m has full period if and only if the following three conditions hold. 1.The only positive integer that exactly divides both m and c is 1. 2.If q is a prime number that divides m, then q divides a-1. 3.If 4 divides m, then 4 divides a-1.  The parameters of the LCG dictate the period length of the LCG as well as other properties of the numbers generated. 21

 A generator that has the maximum possible period is called a full-period generator.  Lower autocorrelations between successive numbers are preferable.  Both generators have the same full period, but the first one has a correlation of 0.25 between x n-1 and x n, whereas the second one has a negligible correlation of less than

 Types of LCGs ◦ When c = 0, the LCG is called a multiplicative generator. ◦ When c ≠ 0, the LCG is called a mixed generator.  Most LCGs implemented are multiplicative ◦ Can’t have full period.  How is m selected. ◦ A large period is desired → m=2 31 (based on a 32 bit word size).  With m=2 31 it has been proven that the period can be at most 2 29 (25% of the values are cycled and gaps may be present). 23

 Multiplicative LCG: c=0  Two types: 24

25 Example: Using the multiplicative congruential method, find the period of the generator for a = 13, m = 2 6, and X 0 = 1, 2, 3, and 4. The solution is given in next slide. When the seed is 1 and 3, the sequence has period 16. However, a period of length eight is achieved when the seed is 2 and a period of length four occurs when the seed is 4.

26 Period Determination Using Various seeds iX i X i X i X i

 Maximum possible period 2 k-2  Period achieved if multiplier a is of the form 8i± 3, and the initial seed is an odd integer  One-fourth the maximum possible may not be too small  Low order bits of random numbers obtained using multiplicative LCG's with m=2 k have a cyclic pattern. 27

28

29

 When the modulus m is a prime number and a >1, the maximum period is m-1, no matter whether c=0 or not.  The maximum period m-1 is obtained if and only if ‘a’ is a primitive element modulo m.  If m is prime then ‘a’ is a primitive element modulo m (or primitive root of m) if and only if a n mod m ≠ 1 for n=1, 2, 3, …,m-2.  Recommended. (Prime moduli are best in terms of sequence randomness.) 30

Example:  Starting with a seed of x 0 =1: 1, 3, 9, 27, 19, 26, 16, 17, 20, 29, 25, 13, 8, 24, 10, 30, 28, 22, 4, 12, 5, 15, 14, 11, 2, 6, 18, 23, 7, 21, 1, … The period is 30 ⇒ 3 is a primitive root of 31  With a multiplier of a = 5: 1, 5, 25, 1,… The period is only 3 ⇒ 5 is not a primitive root of 31  Primitive roots of 31= 3, …????????. 31

 LCGs are a special case of the form Z i = g(Z i-1, Z i-2,...) (mod m), U i = Z i /m, for some function g  Examples:  g(Z i-1 ) = aZ i-1 + c LCG  g(Z i-1, Z i-2,..., Z i-q ) = a 1 Z i-1 + a 2 Z i a q Z i-q multiple recursive generator  g(Z i-1 ) = a'Z 2 i-1 + aZ i-1 + c quadratic CG  g(Z i-1, Z i-2 ) = Z i-1 + Z i-2 Fibonacci (bad) 32

Composite Generators Combine two (or more) individual generators in some way.  Differencing LCGs ◦ Z 1i and Z 2i from LCGs with different moduli ◦ Let Z i = (Z 1i – Z 2i ) (mod m); U i = Z i / m ◦ Very good statistical properties ◦ Very portable (micros, different languages)  Wichmann/Hill ◦ Use three LCGs to get U 1i, U 2i, and U 3i sequences ◦ Let U i = fractional part of U 1i + U 2i + U 3i ◦ Long period, good statistics, portability 33

 Originated in cryptography  Can achieve very long periods  Theoretical appeal: for properly chosen parameters, can prove that over a cycle, ◦ mean  1/2 (as for true U(0,1)) ◦ Variance  1/12 (as for true U(0,1)) ◦ Autocorrelation  0 (as for true IID sequence)  Define a sequence of binary digits B 1,B 2,..., by where c j = 0 or 1. 34

 Looks a bit like a generalization of LCG’s.  Let D = delay operator such that Db(n)=b(n+1)  or  Since in mod 2 arithmetic subtraction is equivalent to addition, the preceding equation is equivalent to 35

 The polynomial on the left-hand side of this equation is called a characteristic polynomial and is traditionally written using x in place of D  The period of a Tausworthe generator depends upon the characteristic polynomial. In particular, the period is the smallest positive integer n for which x n - 1 is divisible by the characteristic polynomial. The maximum possible period with a polynomial of order q is 2 q - 1. The polynomials that give this period are called primitive polynomials. 36

 Example: Consider the following polynomial: x 7 + x Using the D operator in place of x, we get or or using the XOR operator or Substituting n-7 for n, we get 37

Starting with b 0 = b 1 =... = b 6 = 1, we get the following bit sequence: 38

 The complete sequence is:  Period = 127 or bits ⇒ The polynomial x 7 +x 3 +1 is a primitive polynomial. 39

 A Tausworthe sequence can be easily generated in hardware using Linear-Feedback Shift Registers (LFSRs).  For example, the polynomial x 5 + x results in the generator b n = b n-2  b n-5. This can be implemented using the LFSR shown in the Figure presented next slide.  The circuit consists of six registers, each holding one bit. On every clock cycle, each register’s content is shifted out, and the new content is determined by the input to the register. 40

 Linear Feedback Shift Register: x 5 +x 3 +1 ⇒ b n = b n-2 ⊕ b n-5 This can be easily implemented using shift registers: 41

Generating U(0,1):  Divide the sequence into successive groups of s bits and use the first l bits of each group as a binary fraction: x n = 0.b sn b sn+1 b sn+2 b sn+3...b sn+l-1  Here, s is a constant greater than or equal to l and is relatively prime to 2 q -1.  s ≥ l ⇒ x n and x j for n ≠ j have no bits in common.  Relative prime-ness guarantees a full period 2 q -1 for x n. 42

Example: b n = b n-4 ⊕ b n-7  The period =127  l=8, s=8: x 0 = = x 1 = = x 2 = = x 3 = = x 4 = = x 5 = = …. 43

List of Primitive Trinomials x 2 + x + 1 x 3 + x + 1 x 4 + x + 1 x 5 + x x 6 + x + 1 x 7 + x + 1 x 7 + x x 9 + x x 10 + x x 11 + x x 15 + x + 1 x 15 + x x 15 + x x 17 + x x 17 + x x 17 + x x 18 + x x 20 + x x 21 + x x 22 + x + 1 x 23 + x x 23 + x x 25 + x x 25 + x x 28 + x x 28 + x x 28 + x x 29 + x x 31 + x x 31 + x x 31 + x x 31 + x If x q + x r + 1 is listed, then x q + x q-r +1 is also primitive. 44

Homework:  Generate random numbers using the primitive polynomial x 5 +x (use l=4)  Generate the same sequence using LFSR. 45