1 Introduction to Complex Systems: How to think like nature Russ Abbott Sr. Engr. Spec. Rotn to CCAE  The Aerospace Corporation. All Rights Reserved. Evolutionary processes: genetic algorithms and genetic programming

2 Complex systems: How to think like nature Unintended consequences. Emergence: what’s right and what’s wrong with reductionism. Design: levels of abstraction/platforms, and entities as nature’s memes. Evolutionary processes, genetic algorithms, genetic programming. Wisdom of crowds: hierarchy, autonomy, commons, C2. Externalizing thought Feasibility ranges (vs. engineering margins) Simulations: floor and noticing emergence. Computer Science vs. Engineering

3 Peppered moths: evolution in action The (very simple) model: at each time tick –A moth’s probability of survival—not being eaten by predators (not shown)— depends on: How close its color (1-9) is to the background color (0-8). The “Selection” slider, which controls the impact of the environment. -The higher the slider, the more important the environment. –Moths both reproduce and die (of old age). During reproduction, they may mutate—have offspring of a different color. Originally, the vast majority of peppered moths in Manchester, England had light coloration—which camouflaged them from predators since they blended into the light-colored trees. With the industrial revolution: –Pollution blackened the trees. –Light-colored moths died off. –Dark-colored moths flourished. With improved environmental standards, light-colored peppered moths have again become common.

4 Try it out File > Models Library > Biology > Evolution > Peppered Moths Click Open

5 The nature of evolution Moth (and their colors) are rivals, not competitors. They are rivals with respect to their ability –to survive –to acquire resources from the environment. Nature is not “red in tooth and claw.” –The moths and their colors don’t compete with each other directly. There are no moth-on-moth battles. Nor do the dark moths attempt to convince the light moths that it’s better to be dark — or vice versa. –Coke and Pepsi are rivals for consumer dollars. They do not attempt to kill each other’s CEOs or to sabotage each other’s delivery trucks. Moth coloring confers survival value (fitness)—which depends on the environment. Hence Darwin’s “natural selection,” i.e., environmental selection. The environment selects the winners. It’s more like a race than a boxing match. Moth (and their colors) are rivals, not competitors. They are rivals with respect to their ability –to survive –to acquire resources from the environment. Nature is not “red in tooth and claw.” –The moths and their colors don’t compete with each other directly. There are no moth-on-moth battles. Nor do the dark moths attempt to convince the light moths that it’s better to be dark — or vice versa. –Coke and Pepsi are rivals for consumer dollars. They do not attempt to kill each other’s CEOs or to sabotage each other’s delivery trucks. Moth coloring confers survival value (fitness)—which depends on the environment. Hence Darwin’s “natural selection,” i.e., environmental selection. The environment selects the winners. It’s more like a race than a boxing match.

6 Application to engineering problems: The Traveling Salesman Problem (TSP) Connect the cities with a path that: Starts and ends at the same city. Includes all cities. Includes no city twice. In this case the problem is easy to solve by inspection. In general, it’s computationally explosive since there are (n-1)! possible tours. B B A A D D E E C C The obvious tour will include the sequence: ACED-54 (or its reverse). The question is where to put B: ABCED- 55, ACBED-57, or ACEBD-56?

7 An exchange (or reverse or mutation) solves this problem. ACBED-57 → ABCED-55 Genetic algorithm approach Create a population of random tours. AEBCD-59, ACBED-57, ADCBE-59, ACDEB-71, … In this case there are only 4! = 24 possible tours. Could examine them all. Usually that’s not possible. Repeat until good enough or no improvement. But beware local optima. Select one or two tours as parents. −Ensure that better tours are more likely to be selected. Generate offspring using genetic operators to replace poorer elements. −Exchange two cities: ACDEB-71 → ACBED-57 −Reverse a subtour: ACBED-57 → AEBCD-59 −(Re)combine two tours: AEBCD-59 & ACBED-57 → AEDCB-71. Possibly mutate the result: ADCBE-59 → ACBDE-70 B B A A D D E E C C

8 Try it out: TSP.jar After starting a run, double click in the display area to add a city or on a city to remove it. –New cities are added to the tour next to their nearest neighbor. Stop and restart for new random cities. –The number of new cities will be the same as the number of old cities. The differences between the current best and its predecessor are shown by link color. –New links are shown in green. –Removed links are in dashed magenta. No “geographical” heuristics are used. Just the structural ones shown on the previous slide.

9 Genetic algorithms: parameter setting/tuning The number of variables is constant. –Both the TSP and the peppered moths examples illustrate genetic algorithms. Peppered moths: one parameter (color) to set. TSP: N variables. As a parameter setting problem think of each tour as consisting of N variables, each of which may contain any city number. The additional constraint is that no city may repeat. Often there are hundreds of variables (or more) or the search space is large and difficult to search for some other reason. There is no algorithmic way to find values that optimize (maximize/minimize) an objective function. Terrile et. al. (JPL), “Evolutionary Computation applied to the Tuning of MEMS gyroscopes,” GECCO, Abstract: We propose a tuning method for MEMS gyroscopes based on evolutionary computation to efficiently increase the sensitivity of MEMS gyroscopes through tuning and, furthermore, to find the optimally tuned configuration for this state of increased sensitivity. The tuning method was tested for the second generation JPL/Boeing Post-resonator MEMS gyroscope using the measurement of the frequency response of the MEMS device in open-loop operation.

10 Genetic Programming: design and analysis The number of variables (and the structure of the possible solution) is not fixed. Originally attempted (not very successfully) to be applied to generating computer programs. But applied successfully to other design and analysis problems. –Circuit design –Lens design Bongard and Lipson (Cornel), “Automated reverse engineering of nonlinear dynamical systems,” PNAS, Abstract: Complex nonlinear dynamics arise in many fields of science and engineering, but uncovering the underlying differential equations directly from observations poses a challenging task. The ability to symbolically model complex networked systems is key to understanding them, an open problem in many disciplines. Here we introduce for the first time a method that can automatically generate symbolic equations for a nonlinear coupled dynamical system directly from time series data. This method is applicable to any system that can be described using sets of ordinary nonlinear differential equations, and assumes that the (possibly noisy) time series of all variables are observable. … “Symbolic regression”

11 The Human-competitive awards: “Humies” Each year at the Genetic and Evolutionary Computing Conference (GECCO), prizes are awarded to systems that perform at human-competitive levels—including the previous two slides. –See An automatically created result is considered “human-competitive” if it satisfies at least one of the eight criteria below. A.The result was patented as an invention in the past, is an improvement over a patented invention, or would qualify today as a patentable new invention. B.The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer-reviewed scientific journal. C.The result is equal to or better than a result that was placed into a database or archive of results maintained by an internationally recognized panel of scientific experts. D.The result is publishable in its own right as a new scientific result — independent of the fact that the result was mechanically created. E.The result is equal to or better than the most recent human-created solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions. F.The result is equal to or better than a result that was considered an achievement in its field at the time it was first discovered. G.The result solves a problem of indisputable difficulty in its field. H.The result holds its own or wins a regulated competition involving human contestants (in the form of either live human players or human-written computer programs).

12 Tom Lang and Laura Speckman: Genetic Algorithm for Constellation Optimization (GACO) Finds optimal constellation orbits using a genetic algorithm under multiple design constraints and with multiple sensor types. For low number of sats, GA arrangement is significantly better than Walker