Even More Random Number Generators Using Genetic Programming Joe Barker

Topics Genetic Programming Random Numbers Previous Efforts Design & Implementation Results Conclusion Future Bibliography

Genetic Programming Evolve programs for solutions, instead of solutions Difficulty of representation Higher level than standard EA compounds standard problems

Genetic Programming Gene Expression Programming Encodes information in a similar way to genes(operation) to DNA(string) Mutation & Crossover obvious string operations Care required to avoid gibberish (2)

Genetic Programming Gene Expression Programming Example (3) AG-CC-GT-TA-CC 2+1*3

Genetic Programming Expression Trees Encodes operations in a natural tree structure –Internal nodes are operations –Leaf-nodes are variables or constants Mutation & Crossover follow from the structure Layout of the tree avoids non-sensical results (4)

Genetic Programming Expression Trees Example (5)

Random Numbers Why? –Evolutionary Algorithms –Monte-Carlo Simulations –Software Regression Testing –Game Playing

Random Numbers What? –“Random” is difficult to define –Even statistical definitions necessarily describe what we would consider random –Uniform is Uniformly distributed but not what we would consider random –Tests exist to try and cover the important aspects of random (2)

Random Numbers Tests –Chi-Squared test for closeness of fit (3)

Random Numbers Tests –Frequency or Equidistribution test Break number space into a small number of blocks Use the counts for these blocks in Chi-Squared test for Uniform distribution –Gap test Break number space into to classes(Normally upper and lower parts) Count length of runs of class 2 between class 1 Use a Chi-Squared test with the following distribution: (4)

Random Numbers Tests –Entropy Arrange the numbers as a bitstring and count occurrences with certain lengths – – Use percent occurrences in the following formula: (5)

Random Numbers How? –Computers are deterministic, so we must approximate –Several classes of pseudo-random number generators(PRNGs) (6)

Random Numbers PRNGs –Linear congruential randomizers Some of the earliest known (7) Common choices –Park-Miller: a=7^5 b=0 c=2^31-1 –URN08/RANDU: a=65539 b=0 c=2^31

Random Numbers PRNGs –Shift register randomizers SR[a,b,c] A common choice is SR[3,28,31] (8)

Random Numbers PRNGs –Shuffling randomizer Uses two other PRNGs The first PRNG re/fills a list of numbers The second PRNG selects number from the list –Inversive –Mersenne Twister (9)

Previous Efforts This project is based largely on the work by John R. Koza –Used expression trees as individuals –The tree was executed on numbers 1..16K to obtain a random sequence –Bit entropy (lengths 1..7) –Non-Terminals=+,-,*,/,% –Terminals=J,0,1,2,3

Design & Implementation Individuals –Expression Tree –Non-Terminals=+,-,*,/,%  =XOR Each non-terminal is equally likely in a random tree –Terminals=J,0,1,2,3 2^i = Power of 2 (i=1-31, uniform) Each terminal is equally likely in a random tree –Output range is 0..2^32-1 –Aged some number of steps before mature

Design & Implementation Evaluation – Fitness –Bit entropy (lengths 5,6,7,8) –Frequency tests (512 blocks) –Gap test for “runs above the mean” (Up to 10) –The alpha value calculated from the above two tests was adjusted by the formula: –All three values were normalized to a maximum of 1 and summed –These are then averaged over the life of the individual (2)

Design & Implementation Selection –2 mature parents selected uniformly –There is a small chance of both crossover and mutation, but most likely only one Crossover –Subtrees selected from each parent and swapped Mutation –Replaced subtree with equal or smaller random tree (3)

Design & Implementation Crossover - Mutation (4)

Design & Implementation Competition –Replaces the bottom, by fitness, two mature population members Other –No termination, runs indefinitely –HUP signal causes population to be dumped to file (5)

Results Population size=100 Mature Age=30 Initial Maximum Tree Depth=10 Crossover only chance=0.8 Both chance=0.15

Results After 10 hours on 5 machines, the best candidate was: (2)

Results Performance - Entropy Test PRNGAvg. EntropyStd. Deviation Stage E-07 Stage R E-05 glibc rand() E-05 Ideal (3)

Results Performance - Frequency Test PRNGChi-Sq. StatisticChi-Sq. Percentile Stage % Stage % R % glibc rand() % Ideal % (4)

Results Performance - Gap Test PRNGChi-Sq. StatisticChi-Sq. Percentile Stage % Stage % R % glibc rand() % Ideal % (5)

Results Performance - Speed Test (random nums/sec) PRNGSpeed Stage Compiled Stage E6 R E7 glibc rand()6.2893E6 (6)

Conclusion It appears that employing EAs in this manner has promise I hesitate to recommend using Stage 2 as a production randomizer as of now, but it does bear more investigation

Future Work Add prime numbers to the available terminals Add more tests, such as periodicity, to the fitness function Some type of runtime compilation instead of interpreting the expression trees

Bibliography Koza, John R., Evolving a Computer Program to Generate Random Numbers Using the Genetic Programming Paradigm, Proceedings of the Fourth International Conference on Genetic Algorithms, Morgan Kaufmann Publishers, Inc., pages 37-44, Knuth, D. E., The Art of Computer Programming, Volume 2, Second Edition, Addison-Wesley, pages 9-114, Reading, MA, Koza, John R., Genetically Breeding Populations of Computer Programs to Solve Problems in Artificial Intelligence, Proceedings of the Second International Conference on Tools for AI. Washington, November, 1990, IEEE Computer Society Press, Los Alamitos, CA Kinnear, Kenneth E. Jr., Evolving a sort: Lessons in genetic programming. Proceedings of the 1993 International Conference on Neural Networks, volume 2. IEEE Press, Kinnear, Kenneth E. Jr., Generality and Difficulty in Genetic Programming: Evolving a Sort, Proceedings of the Fifth International Conference on Genetic Algorithms, Morgan Kaufmann Publishers, pages , Inc.,