1. 1 Principles of Complex Systems How to think like nature Russ Abbott Does nature really think?

2. 2 Complex systems overview Part 1. Introduction and motivation. Overview – unintended consequences, mechanism, function, and purpose; levels of abstraction, emergence, introduction to NetLogo. Emergence, levels of abstraction, and the reductionist blind spot. Modeling; thought externalization; how engineers and computer scientists think. Part 2. Evolution and evolutionary computing. Innovation – exploration and exploitation. Platforms – distributed control and systems of systems. Groups – how nature builds systems; the wisdom of crowds. Summary/conclusions – remember this if nothing else. Lots of echoes and repeated themes from one section to another.

3. 3 Principles of Complex Systems: How to think like nature What “complex systems” means, and why you should care. Russ Abbott

4. 4 But if I had to define what a “complex system” is … –A collection of autonomous elements that interact both with each other and with their environment and that exhibits ensemble, macro behaviors that none of the elements exhibit individually. But if I had to define what a “complex system” is … –A collection of autonomous elements that interact both with each other and with their environment and that exhibits ensemble, macro behaviors that none of the elements exhibit individually. What we will be talking about. “Complex systems” refers to an anti-reductionist way of thinking that developed in the 1980s in Biology, Computer Science, Economics, Physics, and other fields. (The term complexity is also used this way.) –It is not intended to refer to a particular category of systems, which are presumably distinguished from other systems that aren’t “complex.” Santa Fe Institute was founded in 1984.

5. 5 A satellite in a geostationary orbit: one of the simplest possible “complex systems” But nothing is tying it down. No cable is holding it in place. Wikipedia commons period of the orbit = period of the earth’s rotation Typical of complex system mechanisms. Multiple independent or quasi-independent processes — which are not directly connected causally (agents) — interact within an environment to produce a result. Typical of complex system mechanisms. Multiple independent or quasi-independent processes — which are not directly connected causally (agents) — interact within an environment to produce a result. Fixed with respect to the earth as a reference frame. An “emergent” property What is the environment? 3D space

6. 6 Complex systems terms Emergence. A level of abstraction—which can be described independently of its implementation. Multi-scalar. Applicable to systems that are understood on multiple levels simultaneously, especially when a lower level implements some functionality at a higher level.

7. 7 Why should you care? Because it’s a hot (still warm?) topic. Most corporate executives have heard the term. Some of them think it’s important. –The Command and Control Research Program (CCRP) in the Pentagon (Dave Alberts) is successfully promoting this style of thinking within the DoD. –Complex systems thinking is a generalization of and the foundation for net- centric thinking—and the way the world has changed as a result of the web. You should understand what they are talking about –So that you can explain it to them. Because it offers a powerful new way to think about how systems work. Because large systems—and especially systems of systems (another important buzz-word)—tend to be complex in the ways we will discuss. Because the ideas are interesting, important, and good for you. Anne-Marie is keeping it hot here.

8. 8 This complex system involves many independently operating elements (the train lines, let’s assume) that together enable one to get from any station to any other station without a massive number of point-to-point connections. Simon Patterson is fascinated by the information which orders our lives. He humorously dislocates and subverts sources of information such as maps, diagrams and constellation charts; one of his best known works is The Great Bear, in which he replaced the names of stations on the London Underground map with names of philosophers, film stars, explorers, saints and other celebrities. By transforming authoritative data with his own associations he challenges existing rationales.

9. 9 Introduction to Complex Systems: How to think like nature Unintended consequences; mechanism, function, and purpose Russ Abbott This segment introduces some basic concepts.

10. 10 A fable Once upon a time, a state in India had too many snakes. To solve this problem the government instituted an incentive- based program to encourage its citizens to kill snakes. It created the No Snake Left Alive program. –Anyone who brings a dead snake into a field office of the Dead Snake Control Authority (DSCA) will be paid a generous Dead Snake Bounty (DSB). Once upon a time, a state in India had too many snakes. To solve this problem the government instituted an incentive- based program to encourage its citizens to kill snakes. It created the No Snake Left Alive program. –Anyone who brings a dead snake into a field office of the Dead Snake Control Authority (DSCA) will be paid a generous Dead Snake Bounty (DSB).

11. 11 The DSCA mechanism Catch, kill, and submit a dead snake. DSCA Receive money. Dead snake verifier Receive dead snake certificate. Submit certificate to DSCA.

12. 12 A fable (continued) A year later the DSB budget was exhausted. DSCA had paid for a great many dead snakes. But there was no noticeable reduction in the number of snakes plaguing the good citizens of the state. What went wrong? A year later the DSB budget was exhausted. DSCA had paid for a great many dead snakes. But there was no noticeable reduction in the number of snakes plaguing the good citizens of the state. What went wrong?

13. 13 The DSCA mechanism Catch, kill, and submit a dead snake. DSCA Receive money. Dead snake verifier Receive dead snake certificate. Submit certificate to DSCA. What would you do if this mechanism were available in your world? Start a snake farm.

14. 14 Moral: unintended consequences A mechanism is installed in an environment. The mechanism is used/exploited in ways … –which may not be that for which it was originally intended. This is especially important when the mechanism is a source of energy … –which is fundamental. The fundamental relationships in complex systems are among entities, their environments, and energy.

15. 15 Dicrocoelium dendriticum * D. dendriticum spends its adult life inside the liver of its host. After mating, the eggs are excreted in the feces. The first intermediate host, the terrestrial snail (Cionella lubrica in the United States), eats the feces, and becomes infected by the larval parasites. … The snail tries to defend itself by walling the parasites off in cysts, which it then excretes and leaves behind in the grass. The second intermediate host, an ant (Formica fusca in the United States) swallows a cyst loaded with hundreds of juvenile lancet flukes. The parasites enter the gut and then drift through its body. Some move to a cluster of nerve cells where they take control of the ant's actions. Every evening the infested ant climbs to the top of a blade of grass until a grazing animal comes along and eats the grass—and the ant and the fluke. The fluke grows to adulthood and lives out its life inside the animal—where it reproduces, and the cycle continues. * Text and image from Wikipedia.org. See also, Shelby Martin, “The Petri Dish: The journeys of the brainwashing parasite,” The Stanford Daily, April 20, 2007. http://daily.stanford.edu/article/2007/4/20/thePetriDishTheJourneysOfTheBrainwashingParasite

16. 16 Toxoplasma gondii * The life cycle of T. gondii has two phases. –The sexual part of the life cycle (coccidia like) takes place only in members of the Felidae family (domestic and wild cats). –The asexual part of the life cycle can take place in any warm-blooded animal. T. gondii infections have the ability to change the behavior of rats and mice, making them drawn to rather than fearful of the scent of cats. –This effect is advantageous to the parasite, which will be able to sexually reproduce if its host is eaten by a cat. –The infection is almost surgical in its precision, as it does not impact a rat's other fears such as the fear of open spaces or of unfamiliar smelling food. * Text and image from Wikipedia.org. See also, Charles Q. Choi, “Bizarre Human Brain Parasite Precisely Alters Fear,” Live Science, April 2, 2007. http://www.livescience.com/animals/070402_cat_urine.html

17. 17 Spinochordodes tellinii * The nematomorph hairworm Spinochordodes tellinii is a parasitic worm whose larvae develop in Orthopteran insects. When it is ready to leave the host, the parasite causes the host to jump into water, where it drowns, but which returns the parasite to the medium where it grows to adulthood. * Text and image from Wikipedia.org. See also, James Owen, “Suicide Grasshoppers Brainwashed by Parasite Worms,” National Geographic News, September 1, 2005. http://news.nationalgeographic.com/news/2005/09/0901_050901_wormparasite.html

18. 18 Each tumble reorients the cell and sets it off in a new direction. Cells that are moving up the gradient of an attractant tumble less frequently than cells wandering in a homogeneous medium or moving away from the source. In consequence, cells take longer runs toward the source and shorter ones away. Locomotion in E. coli E. coli movements consist of short straight runs, each lasting a second or less, punctuated by briefer episodes of random tumbling. Harold, Franklyn M. (2001) The Way of the Cell: Molecules, Organisms, and the Order of Life, Oxford University Press.The Way of the Cell Exploitation Exploration Gain benefit Microcosm, Carl Zimmer

19. 19 Mechanism, function, and purpose Mechanism: The physical processes within an entity. –The chemical reactions built into E.coli that result in its flagella movements. –The internal bureaucratic DSCA mechanism. –The chemical mechanisms stimulated by the parasites. Function: The effect of a mechanism on the environment and on the relationship between an entity and its environment. –E. coli moves about. In particular, it moves up nutrient gradients. –Snakes are killed. –The actions of the hosts. Purpose: The (presumably positive) consequence for the entity of the change in its environment or its relationship with its environment. (But Nature is not teleological.) –E. coli is better able to feed, which is necessary for its survival. –Snake farming is encouraged? –Survival and reproduction. Compare to Measures of Performance, Effectiveness, and Utility Wikipedia Commons Socrates

20. 20 Most of the design steps require significant conceptualization abilities. In both cases, the world will be changed by the addition of the new functionality. The purpose is more likely to be achieved by nature since it’s only a purpose if it succeeds. In both cases, the world will be changed by the addition of the new functionality. The purpose is more likely to be achieved by nature since it’s only a purpose if it succeeds. Teleology: building “purpose” E.g., E. coli locomotion to food Evolve a new mechanism Experience the resulting functionality If the functionality enhances survival, keep the mechanism “ Purpose ” has been created (and by definition achieved) implicitly E.g., E. coli locomotion to food Evolve a new mechanism Experience the resulting functionality If the functionality enhances survival, keep the mechanism “ Purpose ” has been created (and by definition achieved) implicitly E.g., Reduce snake population Identify a purpose (need) Imagine how a function can achieve that purpose Design and develop a mechanism to perform that function Deploy the mechanism and hope the purpose is achieved E.g., Reduce snake population Identify a purpose (need) Imagine how a function can achieve that purpose Design and develop a mechanism to perform that function Deploy the mechanism and hope the purpose is achieved DesignedNature

21. 21 MechanismFunctionPurpose Nature Design

22. 22 NetLogo: let’s try it File > Models Library > Biology > Ants Click Open

23. 23 Two levels of emergence No individual chemical reaction—or line of code—is responsible for making the ants follow the rules that describe their behavior. –A first level of emergence are the behaviors produced by the chemical reactions. No individual rule and no individual ant is responsible for the ant colony gathering food. –A second level of emergence is the phenomenon of the ants gathering food. Colony results Ant behaviors Ant chemistry Each layer is a level of abstraction

24. 24 Two levels of emergence No individual chemical reaction inside the ants is responsible for making them follow the rules that describe their behavior. That the internal chemical reactions together do is an example of emergence. No individual rule and no individual ant is responsible for the ant colony gathering food. That the ants together bring about that result is a second level of emergence. Colony results Ant behaviors Ant chemistry Presentation Session Transport Network Physical WWW (HTML) — browsers + servers Applications, e.g., email, IM, Wikipedia Each layer is a level of abstraction Notice the similarity to layered communication protocols

25. 25 Principles of Complex Systems: How to think like nature Emergence: what’s right and what’s wrong with reductionism Presumptuous? Russ Abbott

26. 26 How macroscopic behavior arises from microscopic behavior. Emergent entities (properties or substances) ‘arise’ out of more fundamental entities and yet are ‘novel’ or ‘irreducible’ with respect to them. Stanford Encyclopedia of Philosophy http://plato.stanford.edu/entries/properties-emergent/ Emergence: the holy grail of complex systems The ‘scare’ quotes identify problematic areas. Plato Emergence: Contemporary Readings in Philosophy and Science Mark A. Bedau and Paul Humphreys (Eds.), MIT Press, April 2008.

27. 27 Why is there anything except physics? The reductionist challenge If a higher level explanation can be related to physical processes, it becomes redundant since the explanatory work can be done by physics. — Maurice Schouten and Huib Looren de Jong, The Matter of the Mind, 2007 Why is there anything except physics? — Fodor, 1998 Well, I admit that I don’t know why. I don’t even know how to think about why. I expect to figure out why there is anything except physics the day before I figure out why there is anything at all.

28. 28 Phenomena that arise from and depend on some more basic phenomena yet are simultaneously autonomous from that base. When we finally understand what emergence truly is [we will know] whether there are any genuine examples of emergence. How should emergence be defined? … irreducibility, unpredictability, conceptual novelty, ontological novelty, supervenience? In what ways are emergent phenomena autonomous from their emergent bases? … irreducible to their bases, inexplicable from them, unpredictable from them, supervenient on them, multiply realizable in them? Does emergence necessarily involve novel causal powers, especially powers that produce “downward causation?” Emergence … is simultaneously palpable and confusing. The very idea of emergence seems opaque, and perhaps even incoherent. Paul HumphreysMark Bedau Emergence 2008

29. 29 Cosma Shalizi http://cscs.umich.edu/~crshalizi/reviews/holland-on-emergence/ Someplace … where quantum field theory meets general relativity and atoms and void merge into one another, we may take “the rules of the game” to be given. But the rest of the observable, exploitable order in the universe benzene molecules, PV = nRT, snowflakes, cyclonic storms, kittens, cats, young love, middle-aged remorse, financial euphoria accompanied with acute gullibility, prevaricating candidates for public office, tapeworms, jet-lag, and unfolding cherry blossoms Where do all these regularities come from? Call this emergence if you like. It’s a fine-sounding word, and brings to mind southwestern creation myths in an oddly apt way.

30. 30 Living matter, while not eluding the ‘laws of physics’ … is likely to involve ‘other laws,’ [which] will form just as integral a part of [its] science. [Starting with the basic laws of physics] it ought to be possible to arrive at … the theory of every natural process, including life, by means of pure deduction. All of nature is the way it is … because of simple universal laws, to which all other scientific laws may in some sense be reduced. There are no principles of chemistry that simply stand on their own, without needing to be explained reductively from the properties of electrons and atomic nuclei, and … there are no principles of psychology that are free-standing. [Starting with the basic laws of physics] it ought to be possible to arrive at … the theory of every natural process, including life, by means of pure deduction. — Einstein Living matter, while not eluding the ‘laws of physics’ … is likely to involve ‘other laws,’ [which] will form just as integral a part of [its] science. — Schrödinger All of nature is the way it is … because of simple universal laws, to which all other scientific laws may in some sense be reduced. There are no principles of chemistry that simply stand on their own, without needing to be explained reductively from the properties of electrons and atomic nuclei, and … there are no principles of psychology that are free-standing. — Weinberg The ability to reduce everything to simple fundamental laws [does not imply] the ability to start from those laws and reconstruct the universe. — The ability to reduce everything to simple fundamental laws [does not imply] the ability to start from those laws and reconstruct the universe. — Anderson

31. 31 Are there autonomous higher level laws of nature? Fodor cites Gresham’s law. The fundamental dilemma of science How can that be if everything can be reduced to the fundamental laws of physics? The functionalist claim The reductionist position It can all be explained in terms of levels of abstraction. My answer Emergence

32. 32 Game of Life Gliders A 2-dimensional cellular automaton. The Game of Life rules determine everything that happens on the grid. A dead cell with exactly three live neighbors becomes alive. A live cell with either two or three live neighbors stays alive. In all other cases, a cell dies or remains dead. The “glider” pattern Nothing really moves. Just cells going on and off.

33. 33 The Game of Life Click Open File > Models Library > Computer Science > Cellular Automata > LifeLife

34. 34 Gliders are causally powerless. –A glider does not change how the rules operate or which cells will be switched on and off. A glider doesn’t “ go to a cell and turn it on. ” –A Game of Life run will proceed in exactly the same way whether one notices the gliders or not. A very reductionist stance. But … –One can write down equations that characterize glider motion and predict whether—and if so when—a glider will “turn on” a particular cell. –What is the status of those equations? Are they higher level laws? Gliders Like shadows, they don’t “do” anything. The rules are the only “forces!” Good GoL website

35. 35 Amazing as they are, gliders are also trivial. –Once we know how to build a glider, it’s simple to make as many of them as we want. Can build a library of Game of Life patterns and their interaction APIs. By suitably arranging these patterns, one can simulate a Turing Machine. Paul Rendell. http://rendell.server.org.uk/gol/tmdetails.htm Game of Life as a Programming Platform A second level of emergence. Emergence is not particularly mysterious. What does it mean to compute with shadows?

36. 36 Downward causation The unsolvability of the TM halting problem entails the unsolvability of the GoL halting problem. –How strange! We can conclude something about the GoL because we know something about Turing Machines. –Yet Turing Machines are just shadows in the GoL world. –And the theory of computation is not derivable from GoL rules. Downward causation entailment “Reduce” GoL unsolvability to TM unsolvability by constructing a TM within the GoL. Paul Davies, “The physics of downward causation” in Philip Clayton (Claremont Graduate University), Paul Davies (Macquarie/NSW/Arizona State University), The re-emergence of emergence, 2006

37. 37 A GoL Turing machine … … is an entity. –Like a glider, it is recognizable; it has reduced entropy; it persists and has coherence—even though it is nothing but patterns created by cells going on and off. … obeys laws from the theory of computability. … is a GoL phenomenon that obeys laws that are independent of the GoL rules while at the same time being completely determined by the GoL rules. Reductionism holds. Everything that happens on a GoL grid is a result of the application of the GoL rules and nothing else. Computability theory is independent of the GoL rules. Just as Schrödinger said.said Living matter, while not eluding the ‘laws of physics’ … is likely to involve ‘other laws,’ [which] will form just as integral a part of [its] science. — Schrödinger

38. 38 Level of abstraction: causally reducible yet ontologically real A collection of entities and relationships that can be described independently of their implementation. A Turing machine; biological entities; every computer application, e.g., PowerPoint. When implemented, a level of abstraction is causally reducible to its implementation. You can look at the implementation to see how it works. Its independent description makes it ontologically real. How it behaves depends on its description at its level of abstraction, which is independent of its implementation. The description can’t be reduced away to the implementation without losing information. If the level of abstraction is about nature, reducing it away is bad science.

39. 39 Supervenience A set of predicates H (for Higher-level) about a world supervenes on a set of predicates L (for Lower-level) if it is never the case that two states of affairs of that world will assign the same configuration of truth values to the elements of L but different configurations of truth values to the elements of H. –In other words, L fixes H. –Or, no change in H without a change in L. Think of L as statements in physics and H as statements in a Higher-level (“special”) science. Or, think of L as statements in a computer program and H as the specification of the program’s functionality. Developed originally in philosophy of mind in an attempt to link mind and brain.

40. 40 Supervenience example H: {An odd number of bits is on., The bits that are on are the start of the Fibonacci sequence., The bits that are on represent the value 10., …} H supervenes over L1. The truth value of a statement in H depends on the truth values of the statements in L1. But not over L2. The H statement “ An odd number of bits is on.” can be either true or false (by varying bit 2) without changing the truth values in L2—since L2 ignores bit 2. The world in two different states L1: {Bit 0 is on., Bit 1 is on., Bit 2 is on. Bit 3 is on., Bit 4 is on.} t, t, t, t, t t, t, f, t, t L2: {Bit 0 is on., Bit 1 is on., Bit 3 is on., Bit 4 is on.} t, t, t, t

41. 41 Evolution as a level of abstraction Darwin and Wallace’s theory of evolution by natural selection is expressed in terms of –entities –their properties –how suitable the properties of the entities are for the environment –populations –reproduction –etc. These concepts are a level of abstraction. –The theory of evolution is about entities at that level of abstraction. Let’s assume that it’s (theoretically) possible to trace how any state of the world—including the biological organisms in it—came about by tracking elementary particles Even so, it is not possible to express the theory of evolution in terms of elementary particles. Reducing everything to the level of physics, i.e., naïve reductionism, results in a blind spot regarding higher level entities and the laws that govern them. Darwin and Wallace’s theory of evolution by natural selection is expressed in terms of –entities –their properties –how suitable the properties of the entities are for the environment –populations –reproduction –etc. These concepts are a level of abstraction. –The theory of evolution is about entities at that level of abstraction. Let’s assume that it’s (theoretically) possible to trace how any state of the world—including the biological organisms in it—came about by tracking elementary particles Even so, it is not possible to express the theory of evolution in terms of elementary particles. Reducing everything to the level of physics, i.e., naïve reductionism, results in a blind spot regarding higher level entities and the laws that govern them.

42. 42 How are levels of abstraction built? By adding persistent constraints to what exists. –Constraints “break symmetry” by ruling out possible future states. –Should be able to relate this to symmetry breaking more generally. Easy in software. –Software constrains a computer to operate in a certain way. –Software (or a pattern set on a Game of Life grid) “breaks the symmetry” of possible sequences of future states. How does nature build levels of abstraction? Two ways. –Energy wells produce static entities. Atoms, molecules, solar systems, … –Activity patterns use imported energy to produce dynamic entities. The constraint is imposed by the homeostatic processes that the dynamic entity employs to maintain its structure. Biological entities, social entities, hurricanes. A constrained system operates differently (has additional laws— the constraints) from one that isn’t constrained. Isn’t this just common sense? A TM GoL acts differently from a random configuration.

43. 43 Not surprising A constrained system is likely to obey special rules How can you use two tablespoons of water to break a window? Russ Abbott 4. Hurl the “water stone” at the window. 2. Freeze the water, thereby constraining its molecules into a rigid lattice structure. 3. Remove the frozen water from the tray. 1. Spoon the water into an ice cube tray.

44. 44

45. 45 Not surprising A constrained system is likely to obey special rules So if we constrain the GoL to act like a TM, it shouldn’t be surprising that it is governed by TM laws. How can you use two tablespoons of water to break a window? Russ Abbott 4. Hurl the “water stone” at the window. 2. Freeze the water, thereby constraining its molecules into a rigid lattice structure. 3. Remove the frozen water from the tray. A phase transition often signals the imposition or removal of a constraint. 1. Spoon the water into an ice cube tray. Frozen water implements a solid. It can be used like a solid, and it obeys the laws of solids. (That’s because it is a solid— which is an abstraction.) Is this a trivial observation? Is it just common sense?

46. 46 Categories of entities Naturally occurringHuman designed Energy Status Static. At an energy equilibrium; in an “energy well.” Supervenience is useful. Atoms, molecules, solar systems, … Homeostatic mechanisms: lowest energy state. Tables, boats, houses, cars, ships, … Homeostatic mechanisms: few; generally dependent to “maintenance” processes, Dynamic. Must import energy (and usually other resources) to persist. Supervenience is not useful. Hurricanes(!), biological organisms, biological groups, … Homeostatic mechanisms: specialized for individual cases. Social groups such as governments, corporations, clubs, the ship of Theseus(!), … Homeostatic mechanisms: specialized for individual cases, ranging from force to incentives. Subsidized. Energy is not relevant since it is provided “for free” within a “laboratory” which has built-in support for entities. Ideas, concepts, “memes,” … The elements of a conceptual system. (This paper is not about consciousness. This category just fits here.) Homeostatic mechanisms: don’t understand how memory works. The “first class” values—such as objects, classes, class instances, etc.—within a computational system. Homeostatic mechanisms: generally not required since no natural degradation.

47. 47 Does nature use levels of abstraction? Given the imposition of some (random) constraints, what entities result? Two possibilities. –There are none, or they don’t persist. Back to nature’s drawing board. –They persist and by their interaction create a new level of abstraction. –Nature then asks: what can I build on top of that? (Think James Burke’s Connections.) Software developers do the same thing. It’s all very bottom-up—and in nature’s case random. Each new entity or level of abstraction creates a range of possible laws/mechanisms that didn’t exist before. These could not have been “deduced” from lower levels—except through exhaustive enumeration—any more than a new piece of software can be “deduced” from the programming language in which it is written.

48. 48 Principle of ontological emergence. Extant levels of abstraction are those whose implementations have materialized and whose environments enable their persistence. In some sense it is possible to “deduce” the theory of every natural process and reconstruct the universe, but the reconstruction will involve random constructions. Recall Einstein vs. Anderson [Starting with the basic laws of physics] it ought to be possible to arrive at … the theory of every natural process, including life, by means of pure deduction. — Einstein The ability to reduce everything to simple fundamental laws [does not imply] the ability to start from those laws and reconstruct the universe. — Anderson

49. 49 Practical corollary: feasibility ranges Creating or breaking a level of abstraction frequently corresponds to a phase transition. Physical levels of abstraction are implemented only within feasibility ranges. When the feasibility range is exceeded a phase transition generally occurs. Require contractors to identify the feasibility range within which the implementation will succeed and describe the steps taken to ensure that those feasibility ranges are honored—and what happens if they are not. (Think O-rings.)