Representing Hypothesis Operators Fitness Function Genetic Programming Genetic Algorithms Introduction Representing Hypothesis Operators Fitness Function Genetic Programming Models of Evolution and Learning What is machine learning?

Population concentrates in a small region of the search space. Crowding A problem in genetic algorithms where one individual is much more fit than others. The population then begins to concentrate around this individual and variations of it. Population diversity is then reduced. What is machine learning? F(X) X Population concentrates in a small region of the search space.

Fitness sharing (similar individuals see their fitness value reduced). A Solution to Crowding One solution is to change the selection function, using for example rank selection. In the case where an individual is better fit than others, the ranking would not show the magnitude of the difference. Other solutions: Fitness sharing (similar individuals see their fitness value reduced). b. Spatially distribute individuals and only allow nearby individuals to recombine. c. Restrict certain kinds of individuals to recombine. What is machine learning?

To do this let’s begin with the concept of a schema. The Schema Theory Can we characterize mathematically the evolution of a population over time? To do this let’s begin with the concept of a schema. A schema is a pattern over a sequence of bits represented using 0, 1, or * for don’t care. Example: 0*10 is the set of all sequences with 010 in the 1st, 3rd, and 4rd positions. What is machine learning?

The theory tells us what to expect of m(s,t+1). The Schema Theory Schema theory tells us how each schema in the population evolves through time. Let m(s,t) be the number of instances representing schema s at time or step t. The theory tells us what to expect of m(s,t+1). We start by remembering what is the probability of selecting a hypothesis h: P(hi) = Fitness (hi) / Σj Fitness (hj) What is machine learning?

The probability can be expressed similarly as The Schema Theory The probability can be expressed similarly as P(hi) = f (hi) / n f’ (ti) Where f(hi) is the fitness of hi (for short) and f’(t) is the mean of the fitness value for all hypotheses in the population at time t. Now let us define the mean of the fitness value of the members of schema s in the following way: u(s,t) = Σ f(h) / m(s,t) Where the sum goes through only those hypotheses in schema s. What is machine learning?

E[ m(s, t+1) ] = ( u(s,t) / f’(t) ) m(s,t) The Schema Theory Then the expected value of the number of instances in schema s at time t+1 is: E[ m(s, t+1) ] = ( u(s,t) / f’(t) ) m(s,t) which means the value is proportional to the average fitness of those individuals in schema s at time t and inversely proportional to the average fitness of all individuals at time t. The theory provides a more general formulation based on the effect of crossover and mutation. What is machine learning?

Genetic programming is a form of evolutionary computation in which the individuals in the population are computer programs. Programs are normally represented by trees. A program is executed by parsing the tree. Example: + F = sin(x) + sqrt( x^2 + y) What is machine learning? Sin sqrt x + ^ y x 2

To apply genetic programming one has to define the functions Vocabulary To apply genetic programming one has to define the functions that can be applied: Example: sin, cos, sqrt, +, -, etc. A genetic programming algorithm explores the space of combinations of these functions. The fitness of an individual is determined by executing the program on a set of training data. What is machine learning?

The crossover operator is performed by replacing subtrees of one program with subtrees of another program. + + Sin ^ Sin sqrt 2 x + x y What is machine learning? x + ^ y x 2

Cross Over: Result + + Sin ^ Sin sqrt 2 x ^ x y x + + y x 2 What is machine learning? x + + y x 2

Learning an algorithm for stacking blocks: An Example Learning an algorithm for stacking blocks: N E S What is machine learning? R V U L A I Learn an algorithm to stack the blocks so that it reads “UNIVERSAL”

The only valid operations are An Example The only valid operations are move a block from the stack to the surface of the table move a block from the surface of table to the stack The primitive functions are CS (current stack) returns the name of the block on top of the stack. TB (top current block) returns the name of the topmost block where all blocks and itself are in correct order. c. NN (next necessary) name of the block needed above TB to read the word “UNIVERSAL”. What is machine learning?

Additional functions are An Example Additional functions are (MS x) move block x to stack if x is on table. (MT x) move block x to table if x is on the stack. (EQ x y) returns true if x = y. (NOT x) returns the complement of x. DO( x y) do x until expression y is true What is machine learning?

What is the fitness value? An Example What is the fitness value? In this experiment the author provided 166 different examples having each different initial block configurations. The fitness of a program is how many problem were correctly solved by the program. Initial population = 100 programs After 10 generations the genetic programming strategy found the following program: (EQ (DU (MT CS)(NOT CS)) (DU (MS NN)(NOT NN)) ) which solved all 166 programs correctly. What is machine learning?

Designing filter circuits. Another Example Designing filter circuits. Programs are evolved to transform a fixed circuit into a final circuit design. Primitive functions: Insert circuit components Delete circuit components Fitness: measure how well the design meets the specifications Population size = 640,000 What is machine learning?

Models of Evolution and Learning What is the relationship between what one individual learns during his lifetime compared to what evolution learns as the population evolves? Lamarckian Evolution: Lamarck (late 19th century) hypothesized that the experience of an individual during its lifetime affects the characteristics of its offspring. But evidence points to the contrary: the genetic makeup of an individual is unaffected by the experience of its parents. What is machine learning?

The Baldwin effect relies on the following: After J. M. Baldwin (1986). The Baldwin effect relies on the following: If the environment is continually changing, those individuals capable of learning and adapting quickly to the environment will have greater advantage compared to other individuals. Those individuals who can learn and adapt quickly will have less dependence on the genetic code and will help to achieve more rapid evolutionary adaptation. What is machine learning?

Parallel Genetic Agorithms It seems very natural to produce a parallel version of genetic algorithms. Some approaches divide the population into groups called demes. Each deme resides on a computational node, and a standard search is carried out on each node. There is communication and combination among demes but these are rare compared to those occurring within demes. What is machine learning?

Genetic algorithm do a parallel, hill climbing search looking Summary Genetic algorithm do a parallel, hill climbing search looking to optimize a fitness function. The search resembles the mechanism of biological evolution. Genetic algorithms have many application outside machine learning. In machine learning the goal is to evolve a population of hypotheses to find the hypothesis with highest accuracy. Genetic programming is similar to genetic algorithms but what we evolve is computer programs. What is machine learning?