1. 1 Principles of Complex Systems How to think like nature Russ Abbott Does nature really think?

2. 2 Complex systems overview Part 1. Introduction and motivation. Overview – unintended consequences, mechanism, function, and purpose; levels of abstraction, emergence, introduction to NetLogo. Emergence, levels of abstraction, and the reductionist blind spot. Modeling; thought externalization; how engineers and computer scientists think. Part 2. Evolution and evolutionary computing. Innovation – exploration and exploitation. Platforms – distributed control and systems of systems. Groups – how nature builds systems; the wisdom of crowds. Summary/conclusions – remember this if nothing else. Lots of echoes and repeated themes from one section to another.

3. 3 Principles of Complex Systems: How to think like nature What “complex systems” means, and why you should care. Russ Abbott

4. 4 But if I had to define what a “complex system” is … –A collection of autonomous elements that interact both with each other and with their environment and that exhibits aggregate, ensemble, macro behaviors that none of the elements exhibit individually. But if I had to define what a “complex system” is … –A collection of autonomous elements that interact both with each other and with their environment and that exhibits aggregate, ensemble, macro behaviors that none of the elements exhibit individually. What we will be talking about. “Complex systems” refers to an anti-reductionist way of thinking that developed in the 1980s in Biology, Computer Science, Economics, Physics, and other fields. (The term complexity is also used this way.) –It is not intended to refer to a particular category of systems, which are presumably distinguished from other systems that aren’t “complex.” Isn’t that true of all systems? You are in the business of building “complex systems.” System: a construct or collection of different elements that together produce results not obtainable by the elements alone. — Eberhardt Rechtin Systems Architecting of Organizations: Why Eagles Can't Swim, CRC, 1999. System: a construct or collection of different elements that together produce results not obtainable by the elements alone. — Eberhardt Rechtin Systems Architecting of Organizations: Why Eagles Can't Swim, CRC, 1999. Santa Fe Institute was founded in 1984.

5. 5 A satellite in a geostationary orbit: one of the simplest possible “complex systems” But nothing is tying it down. No cable is holding it in place. Wikipedia commons period of the orbit = period of the earth’s rotation Typical of complex system mechanisms. Multiple independent or quasi-independent processes — which are not directly connected causally (agents) — interact within an environment to produce a result. Typical of complex system mechanisms. Multiple independent or quasi-independent processes — which are not directly connected causally (agents) — interact within an environment to produce a result. Fixed with respect to the earth as a reference frame. An “emergent” property What is the environment? 3D space

6. 6 Complex systems terms Emergence. A level of abstraction that can be described independently of its implementation. Multi-scalar. Applicable to systems that are understood on multiple levels simultaneously, especially when a lower level implements some functionality at a higher level.

7. 7 Why should you care? Because it’s a hot (still warm?) topic. Most corporate executives have heard the term. Some of them think it’s important. –Rumsfeld’s inspiration for transformation in the military grew out of this way of thinking. The field is not new. It’s at least 2 decades old. –The Command and Control Research Program (CCRP) in the Pentagon (Dave Alberts) is successfully promoting this style of thinking within the DoD. –Complex systems thinking is a generalization of and the foundation for net- centric thinking—and the way the world has changed as a result of the web. You should understand what they are talking about –So that you can explain it to them. Because it offers a powerful new way to think about how systems work. Because large systems—and especially systems of systems (another important buzz-word)—tend to be complex in the ways we will discuss. Because the ideas are interesting, important, and good for you. Anne-Marie is keeping it hot here.

8. 8 What is a System of Systems? Small stovepipes to large stovepipes – NO Loosely coupled and tightly integrated – YES 8 Platforms Level of abstraction Functional decomposition

9. 9 Planning Complex Endeavors (April 2007) by David S. Alberts and Richard E. Hayes The Command and Control Research Program (CCRP) has the mission of improving DoD’s understanding of the national security implications of the Information Age. John G. Grimes, Assistant Secretary of Defense (NII) & Chief Information Officer Dr. Linton Wells, II, Principal Deputy Assistant Secretary of Defense (NII) Dr. David S. Alberts, Special Assistant to the ASD(NII) & Director of Research Alberts’ term for what a complex system does.

10. 10 From the forward by John G. Grimes As this latest book from the CCRP explains, we can no longer be content with building an “enterprise-wide” network that stops at the edges of our forces, nor with a set of information sources and channels that are purely military in nature. We need to be able to work with a large and diverse set of entities and information sources. We also need to develop new approaches to planning that are better suited for these coalition operations. The implications are significant for a CIO as it greatly expands the who, the what, and the how of information sharing and collaboration. It also requires a new way of thinking about effectiveness, increasing the emphasis we place on agility, which, as is explained in this book, is the necessary response to uncertainty and complexity. What is this “new way of thinking?” From Chapter 1. Introduction The economics of communications and information technologies has created enormous opportunities to leverage the power of information and collaboration cost effectively by adopting Power to the Edge principles and network-centric concepts. Fine to use these terms, but what do we really mean by them? Platforms Exploration and exploitation

11. 11 From Chapter 2. Key Concepts Complicated Systems Systems that have many moving parts or actors and are highly dynamic, that is, the elements of these systems constantly interact with and impact upon one another. However, the cause and effect relationships within a complicated situation are generally well understood, which allows planners to predict the consequences of specific actions with some confidence. I think this misses the point. Most systems and interactions are (eventually) “well understood.” Complicated systems are often fully entrained with one locus of control.

12. 12 From Chapter 2. Key Concepts Complex Endeavors (Systems) Complex endeavors involve changes and behaviors that cannot be predicted in detail, although those behaviors and changes can be expected to form recognizable patterns. Complex endeavors are also characterized by circumstances in which relatively small differences in initial conditions or relatively small perturbations (seemingly tactical actions) are associated with very large changes in the resulting patterns of behavior and/or strategic outcomes. Some complex situations develop into complex adaptive systems (CAS), which tend to be robust—to persist over time and across a variety of circumstances. These are often observed in nature in the form of biological or ecological systems. However, while these systems are thought of as robust, they can be pushed out of balance even to the point of collapse through cascades of negatively reinforcing conditions and behaviors. Such perturbations are what ecologists fear when a habitat is reduced to an isolated geographic area or when invasive, nonnative species are introduced. Note biological reference I disagree—although sometimes the only way to predict it is to run (a model of) it. But that’s true of the 3-body problem too. Both chaos and phase transitions

13. 13 Like many networks, this complex system involves many independently operating elements (the trains) that together enable one to get from any one station to any other without a massive number of point-to-point connections. Simon Patterson is fascinated by the information which orders our lives. He humorously dislocates and subverts sources of information such as maps, diagrams and constellation charts; one of his best known works is The Great Bear, in which he replaced the names of stations on the London Underground map with names of philosophers, film stars, explorers, saints and other celebrities. By transforming authoritative data with his own associations he challenges existing rationales.

14. 14 The world in a grain of sand To see the world in a grain of sand, and heaven in a wild flower, to hold infinity in the palm of your hands, and eternity in an hour. –William Blake Do you understand what that means? A primary objective of this talk is to say in as plain a way as possible what many people have been groping for when talking about complex systems. Do you understand what that means? It’s profound, but it uses poetry to make its point. Many of the ideas may seem like common sense. It’s just that we’ll be looking at them more closely.

15. 15 Introduction to Complex Systems: How to think like nature Unintended consequences; mechanism, function, and purpose Russ Abbott This segment introduces some basic concepts.

16. 16 A fable Once upon a time, a territory in India had too many snakes. To solve this problem the government instituted an incentive- based program to encourage its citizens to kill snakes. It created the No Snake Left Alive program. –Anyone who brings a dead snake into a field office of the Dead Snake Control Authority (DSCA) will be paid a generous Dead Snake Bounty (DSB). Once upon a time, a territory in India had too many snakes. To solve this problem the government instituted an incentive- based program to encourage its citizens to kill snakes. It created the No Snake Left Alive program. –Anyone who brings a dead snake into a field office of the Dead Snake Control Authority (DSCA) will be paid a generous Dead Snake Bounty (DSB).

17. 17 The DSCA mechanism Catch, kill, and submit a dead snake. DSCA Receive money. Dead snake verifier Receive dead snake certificate. Submit certificate to DSCA.

18. 18 A fable (continued) A year later the DSB budget was exhausted. DSCA had paid for a great many dead snakes. But there was no noticeable reduction in the number of snakes plaguing the good citizens of the state. What went wrong? A year later the DSB budget was exhausted. DSCA had paid for a great many dead snakes. But there was no noticeable reduction in the number of snakes plaguing the good citizens of the state. What went wrong?

19. 19 The DSCA mechanism Catch, kill, and submit a dead snake. DSCA Receive money. Dead snake verifier Receive dead snake certificate. Submit certificate to DSCA. What would you do if this mechanism were available in your world? Start a snake farm.

20. 20 Moral: unintended consequences A mechanism is installed in an environment. The mechanism is used/exploited in unanticipated ways. Once a mechanism is installed in the environment, it will be used for whatever purposes “users” can think to make of it … –which may not be that for which it was originally intended. The first lesson of complex systems thinking is that one must always be aware of the relationship between systems and their environments. A second lesson of complex systems thinking is that energy and energy flows are fundamental—and money is a proxy for energy.

21. 21 Parasites that control their hosts Dicrocoelium dendriticum causes host ants to climb grass blades where they are eaten by grazing animals, where D. dendriticum lives out its adult life. Toxoplasma gondii causes host mice not to fear cats, where T. gondii reproduces. Spinochordodes tellinii causes host insects to jump into the water and drown, where S. tellinii grows to adulthood. Dicrocoelium dendriticum causes host ants to climb grass blades where they are eaten by grazing animals, where D. dendriticum lives out its adult life. Toxoplasma gondii causes host mice not to fear cats, where T. gondii reproduces. Spinochordodes tellinii causes host insects to jump into the water and drown, where S. tellinii grows to adulthood. It’s amazing how far exploitation of environmental mechanisms can go. Mechanism, function, and purpose Talking about dynamic entities. Can’t have any of this without energy. More indirect effects

22. 22 Each tumble reorients the cell and sets it off in a new direction. Cells that are moving up the gradient of an attractant tumble less frequently than cells wandering in a homogeneous medium or moving away from the source. In consequence, cells take longer runs toward the source and shorter ones away. Locomotion in E. coli E. coli movements consist of short straight runs, each lasting a second or less, punctuated by briefer episodes of random tumbling. Harold, Franklyn M. (2001) The Way of the Cell: Molecules, Organisms, and the Order of Life, Oxford University Press.The Way of the Cell Exploitation Exploration Gain benefit Microcosm, Carl Zimmer

23. 23 Mechanism, function, and purpose Mechanism: The physical processes within an entity. –The chemical reactions built into E.coli that result in its flagella movements. –The internal bureaucratic DSCA mechanism. –The chemical mechanisms stimulated by the parasites. Function: The effect of a mechanism on the environment and on the relationship between an entity and its environment. –E. coli moves about. In particular, it moves up nutrient gradients. –Snakes are killed. –The actions of the hosts. Purpose: The (presumably positive) consequence for the entity of the change in its environment or its relationship with its environment. (But Nature is not teleological.) –E. coli is better able to feed, which is necessary for its survival. –Snake farming is encouraged? –Survival and reproduction. Compare to Measures of Performance, Effectiveness, and Utility Wikipedia Commons Socrates

24. 24 Most of the design steps require significant conceptualization abilities. In both cases, the world will be changed by the addition of the new functionality. The purpose is more likely to be achieved by nature since it’s only a purpose if it succeeds. In both cases, the world will be changed by the addition of the new functionality. The purpose is more likely to be achieved by nature since it’s only a purpose if it succeeds. Teleology: building “purpose” E.g., E. coli locomotion to food Evolve a new mechanism Experience the resulting functionality If the functionality enhances survival, keep the mechanism “ Purpose ” has been created (and by definition achieved) implicitly E.g., E. coli locomotion to food Evolve a new mechanism Experience the resulting functionality If the functionality enhances survival, keep the mechanism “ Purpose ” has been created (and by definition achieved) implicitly E.g., Reduce snake population Identify a purpose (need) Imagine how a function can achieve that purpose Design and develop a mechanism to perform that function Deploy the mechanism and hope the purpose is achieved E.g., Reduce snake population Identify a purpose (need) Imagine how a function can achieve that purpose Design and develop a mechanism to perform that function Deploy the mechanism and hope the purpose is achieved DesignedNature

25. 25 MechanismFunctionPurpose Nature Design

26. 26 NetLogo: let’s try it File > Models Library > Biology > Ants Click Open

27. 27 Two levels of emergence No individual chemical reaction—or line of code—inside the ants is responsible for making them follow the rules that describe their behavior. What the internal chemical reactions together do is an example of emergence. No individual rule and no individual ant is responsible for the ant colony gathering food. That the ants together bring about that result is a second level of emergence. Colony results Ant behaviors Ant chemistry Each layer is a level of abstraction Notice the similarity to layered communication protocols

28. 28 Two levels of emergence No individual chemical reaction inside the ants is responsible for making them follow the rules that describe their behavior. That the internal chemical reactions together do is an example of emergence. No individual rule and no individual ant is responsible for the ant colony gathering food. That the ants together bring about that result is a second level of emergence. Colony results Ant behaviors Ant chemistry Presentation Session Transport Network Physical WWW (HTML) — browsers + servers Applications, e.g., email, IM, Wikipedia Each layer is a level of abstraction Notice the similarity to layered communication protocols

29. 29 Principles of Complex Systems: How to think like nature Emergence: what’s right and what’s wrong with reductionism Presumptuous? Russ Abbott

30. 30 How macroscopic behavior arises from microscopic behavior. Emergent entities (properties or substances) ‘arise’ out of more fundamental entities and yet are ‘novel’ or ‘irreducible’ with respect to them. Stanford Encyclopedia of Philosophy http://plato.stanford.edu/entries/properties-emergent/ Emergence: the holy grail of complex systems The ‘scare’ quotes identify problematic areas. Plato Emergence: Contemporary Readings in Philosophy and Science Mark A. Bedau and Paul Humphreys (Eds.), MIT Press, April 2008.

31. 31 Why is there anything except physics? The reductionist challenge If a higher level explanation can be related to physical processes, it becomes redundant since the explanatory work can be done by physics. — Maurice Schouten and Huib Looren de Jong, The Matter of the Mind, 2007 Why is there anything except physics? — Fodor, 1998 Well, I admit that I don’t know why. I don’t even know how to think about why. I expect to figure out why there is anything except physics the day before I figure out why there is anything at all.

32. 32 Phenomena that arise from and depend on some more basic phenomena yet are simultaneously autonomous from that base. When we finally understand what emergence truly is [we will know] whether there are any genuine examples of emergence. How should emergence be defined? … irreducibility, unpredictability, conceptual novelty, ontological novelty, supervenience? In what ways are emergent phenomena autonomous from their emergent bases? … irreducible to their bases, inexplicable from them, unpredictable from them, supervenient on them, multiply realizable in them? Does emergence necessarily involve novel causal powers, especially powers that produce “downward causation?” Emergence … is simultaneously palpable and confusing. The very idea of emergence seems opaque, and perhaps even incoherent. Paul HumphreysMark Bedau Emergence 2008

33. 33 Cosma Shalizi http://cscs.umich.edu/~crshalizi/reviews/holland-on-emergence/ Someplace … where quantum field theory meets general relativity and atoms and void merge into one another, we may take “the rules of the game” to be given. But the rest of the observable, exploitable order in the universe benzene molecules, PV = nRT, snowflakes, cyclonic storms, kittens, cats, young love, middle-aged remorse, financial euphoria accompanied with acute gullibility, prevaricating candidates for public office, tapeworms, jet-lag, and unfolding cherry blossoms Where do all these regularities come from? Call this emergence if you like. It’s a fine-sounding word, and brings to mind southwestern creation myths in an oddly apt way.

34. 34 Living matter, while not eluding the ‘laws of physics’ … is likely to involve ‘other laws,’ [which] will form just as integral a part of [its] science. [Starting with the basic laws of physics] it ought to be possible to arrive at … the theory of every natural process, including life, by means of pure deduction. All of nature is the way it is … because of simple universal laws, to which all other scientific laws may in some sense be reduced. There are no principles of chemistry that simply stand on their own, without needing to be explained reductively from the properties of electrons and atomic nuclei, and … there are no principles of psychology that are free-standing. [Starting with the basic laws of physics] it ought to be possible to arrive at … the theory of every natural process, including life, by means of pure deduction. — Einstein Living matter, while not eluding the ‘laws of physics’ … is likely to involve ‘other laws,’ [which] will form just as integral a part of [its] science. — Schrödinger All of nature is the way it is … because of simple universal laws, to which all other scientific laws may in some sense be reduced. There are no principles of chemistry that simply stand on their own, without needing to be explained reductively from the properties of electrons and atomic nuclei, and … there are no principles of psychology that are free-standing. — Weinberg The ability to reduce everything to simple fundamental laws [does not imply] the ability to start from those laws and reconstruct the universe. — The ability to reduce everything to simple fundamental laws [does not imply] the ability to start from those laws and reconstruct the universe. — Anderson

35. 35 Are there autonomous higher level laws of nature? Fodor cites Gresham’s law. The fundamental dilemma of science How can that be if everything can be reduced to the fundamental laws of physics? The functionalist claim The reductionist position It can all be explained in terms of levels of abstraction. My answer Emergence

36. 36 Game of Life Gliders A 2-dimensional cellular automaton. The Game of Life rules determine everything that happens on the grid. A dead cell with exactly three live neighbors becomes alive. A live cell with either two or three live neighbors stays alive. In all other cases, a cell dies or remains dead. The “glider” pattern Nothing really moves. Just cells going on and off.

37. 37 The Game of Life Click Open File > Models Library > Computer Science > Cellular Automata > LifeLife

38. 38 Gliders are causally powerless. –A glider does not change how the rules operate or which cells will be switched on and off. A glider doesn’t “ go to a cell and turn it on. ” –A Game of Life run will proceed in exactly the same way whether one notices the gliders or not. A very reductionist stance. But … –One can write down equations that characterize glider motion and predict whether—and if so when—a glider will “turn on” a particular cell. –What is the status of those equations? Are they higher level laws? Gliders Like shadows, they don’t “do” anything. The rules are the only “forces!” Good GoL website

39. 39 Amazing as they are, gliders are also trivial. –Once we know how to build a glider, it’s simple to make as many of them as we want. Can build a library of Game of Life patterns and their interaction APIs. By suitably arranging these patterns, one can simulate a Turing Machine. Paul Rendell. http://rendell.server.org.uk/gol/tmdetails.htm Game of Life as a Programming Platform A second level of emergence. Emergence is not particularly mysterious. What does it mean to compute with shadows?

40. 40 Downward causation The unsolvability of the TM halting problem entails the unsolvability of the GoL halting problem. –How strange! We can conclude something about the GoL because we know something about Turing Machines. –Yet Turing Machines are just shadows in the GoL world. –And the theory of computation is not derivable from GoL rules. Downward causation entailment “Reduce” GoL unsolvability to TM unsolvability by constructing a TM within the GoL. Paul Davies, “The physics of downward causation” in Philip Clayton (Claremont Graduate University), Paul Davies (Macquarie/NSW/Arizona State University), The re-emergence of emergence, 2006

41. 41 A GoL Turing machine … … is an entity. –Like a glider, it is recognizable; it has reduced entropy; it persists and has coherence—even though it is nothing but patterns created by cells going on and off. … obeys laws from the theory of computability. … is a GoL phenomenon that obeys laws that are independent of the GoL rules while at the same time being completely determined by the GoL rules. Reductionism holds. Everything that happens on a GoL grid is a result of the application of the GoL rules and nothing else. Computability theory is independent of the GoL rules. Just as Schrödinger said.said Living matter, while not eluding the ‘laws of physics’ … is likely to involve ‘other laws,’ [which] will form just as integral a part of [its] science. — Schrödinger