Genetic Learning from Experience Sushil J. Louis Evolutionary Computing Systems LAB Department of Computer Science University of Nevada, Reno

2 Outline MotivationMotivation Case Injected Genetic AlgoRithmsCase Injected Genetic AlgoRithms Test problem: Designing 2-bit adderTest problem: Designing 2-bit adder RepresentationRepresentation ResultsResults Conclusions and future workConclusions and future work

3 Motivation Deployed systems are expected to confront and solve many problems over their lifetimeDeployed systems are expected to confront and solve many problems over their lifetime How can we increase genetic algorithm performance with experience?How can we increase genetic algorithm performance with experience? –Provide GA with a memory  Case Base

4 Case-Based Reasoning When confronted by a new problem, adapt similar (already solved) problem’s solution to solve new problemWhen confronted by a new problem, adapt similar (already solved) problem’s solution to solve new problem CBR  Associative Memory + AdaptationCBR  Associative Memory + Adaptation CBR: Indexing and adaptation are domain dependentCBR: Indexing and adaptation are domain dependent

5 Case Injected Genetic AlgoRithm Combine genetic search with case- based memoryCombine genetic search with case- based memory Case-base provides memoryCase-base provides memory Genetic algorithm provides adaptationGenetic algorithm provides adaptation Genetic algorithm generates casesGenetic algorithm generates cases –Any member of the GA’s population is a case

6 Human Problem 1Problem 2 CIGAR Problem 3

7 CIGAR acquires knowledge during problem solving Periodically save members of the GA’s population to the case- basePeriodically save members of the GA’s population to the case- base –A member of the population is a candidate solution to the problem Periodically inject appropriate cases into the GA’s population replacing lowest-fitness membersPeriodically inject appropriate cases into the GA’s population replacing lowest-fitness members Genetic Algorithm Case- base CBR module Preprocessor Candidate solutions Cases Save best individual Inject closest to the best

8 How does CIGAR operate ? Which cases do we inject?Which cases do we inject? Hamming Distance –Inject cases that are closest to the current best member of the population. Genetic algorithms usually use binary encodings. For these encodings, our distance metric is therefore Hamming Distance – the number of differing bits. GA theory points to other injection strategies –Probabilistic version: The probability of injection of a case in the case base is inversely proportional to distance from the current best member of the population relative to the distances of other cases. How often should we inject cases?How often should we inject cases? –Takeover time – number of generations needed for an individual to take over the population. P(Case i ) = (l – d i )/∑(l – d j ) Sum over all cases Chromosome length Hamming distance from best member

9 Test Problems Designing 4-input 3-output combinational logic circuitsDesigning 4-input 3-output combinational logic circuits 2 bit adder is one of these2 bit adder is one of these Evaluation functionEvaluation function –Feed all possible inputs and count the number of correct outputs –Maximum fitness = 16 x 3 = 48

10 Representation A two dimensional circuit is represented as a one dimensional chromosome

11 Generating 50 problems I1I2o carry Output String for 1-bit adder Output String for a boolean function that is similar to 1-bit adder Change output string Truth table for 1-bit adder We generate fifty similar problems by randomly selecting and flipping 0 – 10 bits of the output string of the 2-bit adder

12 Setup 50, 4-bit input, 3bit output combinational logic design problems50, 4-bit input, 3bit output combinational logic design problems Randomly select and flip between 0 – 10 bits in adder output to define logic function (the design problem)Randomly select and flip between 0 – 10 bits in adder output to define logic function (the design problem) Compare performanceCompare performance –Quality of final design solution (correct output) –Time to this final solution (in generations)

13 Parameters Population size: 100Population size: 100 No of generations: 150No of generations: 150 CHC (elitist) selectionCHC (elitist) selection Scaling factor: 1.05Scaling factor: 1.05 Prob. Crossover: 0.95Prob. Crossover: 0.95 Prob. Mutation: 0.05Prob. Mutation: 0.05 Store best individual every generationStore best individual every generation Inject every 8 generationsInject every 8 generations Inject 10 cases (10%)Inject 10 cases (10%) Multiple injection strategiesMultiple injection strategies Averages over 10 runs

14 Expected behavior Number of problems attempted Learning system/CIGAR No learning Quality Number of problems attempted Learning system/CIGAR No learning Time Expected behavior of a learning system QUALITY TIME

15 Actual Behavior Quality RIGA = Randomly Initialized GA

16 Actual Behavior Time to best solution RIGA = Randomly Initialized GA

17 Different Injection strategies Closest to bestClosest to best Probabilistic closest to bestProbabilistic closest to best Furthest from worstFurthest from worst Probabilistic furthest from worstProbabilistic furthest from worst

18 Quality and Time Injection strategy does not make significant difference

19 Less similar set of 50 problems Less difference in performance Flip 8 – 16 output bits

20 Summary CIGAR is a promising approach to learning from experienceCIGAR is a promising approach to learning from experience Hamming distance is similarity metricHamming distance is similarity metric Need to save intermediate solutionsNeed to save intermediate solutions Quality increases with experienceQuality increases with experience Time to solution decreases with experienceTime to solution decreases with experience

21 Future Work Investigate injection strategiesInvestigate injection strategies Similarity metrics for order-based encodingsSimilarity metrics for order-based encodings Does case-injection add useful diversity?Does case-injection add useful diversity? Is performance increase because of useful diversity or domain knowledge or both?Is performance increase because of useful diversity or domain knowledge or both?