Complex System Science John Finnigan CSIRO Atmospheric Research Where in 30 minutes I will have time only to sketch in the freeways

Contents Complex systems Science Three Approaches to Understanding Complexity-the idea of emergent structure Farming systems as ‘Complex Adaptive Systems’ Three Approaches to Understanding Network Theory Cellular Automata Agent Based Models Summary The CSIRO Centre for Complex Systems Science Start by introducing the two aspects of Css Then very briefly talk about conventional approaches Before spending most of my time discussing some aspects of what we are now justified in regarding as mainstream CSS I’ll stick my neck out with some new ideas on what is the key to understanding emergent structure before a BRIEF SUMMARY

Complex System Science Has two elements: Systems-collections of interacting things Complexity-the essence of which is the property of self-organisation or emergence of structure from the interaction between the constituent parts of the system The systematic study of systems qua systems really got underway around the time of the second World War with operations research. It developed as a field in science after the 1940’s with the “General Systems Theory” of the biologist von Bertalanffy[i] and the development of the related fields of cybernetics[ii] , Systems Analysis and Hierarchy theory[iii]. i] von Bertalanffy (1975) Perspectives on General Systems Theory. Braziller, New York.[ii] Wiener, Norbert, (1986) Cybernetics, Science and Society. URL http://www.amazon.com/exec/obidos/external-search/002-3394835-1683226?keyword=Norbert+Wiener&tag=principiacyberne[iii] Alan, TFH, and Starr, TB: (1982) Hierarchy: Perspective for Explaining Ecological Complexity, U. Chicago, Chicagoand others-general systems theory, systems analysis, hierarchy theory etc. To define systems as objects of study we need to define them by drawing control volumes sensibly so that the flow of energy and info across the boundaries is small and also to iodentify what interacts with what and how. Complexity is the science behind systems. At its most basic it asserts that there are universal principles governing the behaviour of all systems and which we can discover. The antecedents of complexity are at a certain level entangled with those of systems theory as in cybernetics but follow a different line through chaos theory and the Santa Fe school. The subject has now coalesced to the state that we can view it as a valid area of study It will be the concern of our workshop. So what do we mean by emergence

Emergence or Self-Organisation We recognise this phenomenon over a vast range of physical scales and degrees of complexity From Galaxies ~ 106 Light Years We mean the appearance of structure, organisation or intelligent behaviour through the interaction of simpler or less intelligent objects. We recognise it over a vast range of scales and contexts

Spiral galaxy in Hercules ~10^5 LY diameter balance between gravity and angular momentum

To cyclones ~ 100 km Cyclone ~100km balance between buoyancy and coriolis Cyclone Douglas off Florida coast last year

And Chemical reactions ~ 10 cm The Belousev-Zhabotinskii reaction is the catalyzed oxidation of citric acid. Reaction initiated by touching a dish of reagents with a hot wire Structures such as spiral travelling waves spontaneously emerge from the initial homogeneous mixture of reagents And Chemical reactions ~ 10 cm

To Gene expression and cell interaction Amoeba Ribosome Root Tip From the molecular level, where the genome codes for linear sequences of amino acids which then fold up autonomously into the required complex 3D shape, to organisation of prokaryotes, eukaryotes and the assembly of cells into organisms is now assumewd to involve a substantial measure of Self organisation as the amount of information required to determine all these processes is not encoded explicitly into the genome. One surprise of genome sequencing is how little info there is there. E Coli

The processing of information by the brain Processing of information by animal Brains-Memory, Consciousness

To animal societies and the emergence of culture Self organisation between animals-societies-much of the behaviour of ’simple’ societies like social insects can be modelled using simple rules governing individual behaviour. Human society is more complex, it seems

And the social artefacts of human society such as economies This is a Bangkok street market

The Concept of Self-Organisation has consequences at several levels At the whole system level (in the present case, farming systems) it means that ‘no one is in charge’ and optimal or command and control solutions to system problems usually fail (sooner or later) At the level of analysis, self organising processes provide us with powerful tools

Foot and Mouth Disease in the UK An example of failure caused by focussing on one part of the system and ignoring the links between biophysics and economics Economic rationalization of abattoirs and bizarre EU subsidies increased the connections between herds to a critical point. Changes to F&M reporting rules may have delayed the isolation of infectious animals. The relationship between these actions and the epidemiology of F&M was not appreciated in advance (at least where it mattered) because the livestock industry was not viewed as an integrated system. There have been F&M outbreaks before in UK. The last big one in the 60’s. Why did this one spread so far and so fast? The economic rationalization of the livestock industry has led to a series of actions that run counter to those required to maintain the bio-ecological resilience of the livestock industry . UK exports and imports an equal number of pigs to Holland. As the pigs cross the channel both ways, they each attract an EU export subsidy. British consumers like to eat Scottish beef and welsh lamb-send your herd for a month’s holiday in Wales or Scotland before slaughter and they qualify Treating the economics and the bio-ecology of the system in isolation led in the end to an economic and social disaster. 750,000 cattle, 4.5m sheep slaughtered, 3Bn pounds in damage to tourism. A crisis of support for the rural lifestyle in UK. City-country divide. The parameters that were increased to maximize economic return were precisely those that maximize the rate of spread of an epidemic. Small world phenomenon

Farming Systems at the gross level involve Economics, People, their Social Networks as well as Biophysics such as hydrology, soil science, Agronomy and Biology. We can attempt to understand and model the whole system or parts of it To model the whole system we need first a mental map and then some techniques to capture the parts and interactions of the mental map

A regional scale social-ecological system including farming, as a complex adaptive system In a CAS, there is no Fat Controller. The system behaviour is an emergent property Climate The Market

We can build models of Complex adaptive Systems using techniques like ‘Agent based Modelling’ but to understand and predict their behaviour, we need a science of systems The understanding we need is coming from a evolving blend of at least three different approaches: Dynamical Systems Theory Network Theory Evolutionary or adaptive computing

1 Studying Ecosystems as dynamical systems A minimal model of an ecosystem describes the change over time of ecosystem state. The trajectories indicate stable states of the ecosystem as external conditions change Ecosystems that display two (or more) alternative stable states include : lakes(oligotrophic/eutrophic), grasslands/woodlands, coral reefs (pristine/algal covered), marine ecosystems as measured in fish catch…. Diff equations or dynamical systems theory replaces the behaviour of many interacting agents with smooth functions. E.g. gases, liquids go from interacting particles to intrinsic properties like T and P or visc with a stupendous loss of degrees of freedom 6 10^23->3. Why this can be done is connected with irreversibility. The realms of stat mech and non equil thermo. Intrinsic props are emergent phenomena of the many interactions of particles. The lines represent equilibrium states of the ecosystem so that if the system is perturbed it returns to this state. When two equilibrium states exist for the same external conditions they are divided by an unstable equilibrium. (Figs from Scheffer et al, 2001, Nature) Two things to note: Hysteresis- have to reverse conditions a long way to reverse the state As you approach the cliff, the ecosystem state hasn’t changed much-little warning (Figs from Scheffer et al, 2001, Nature)

Basins of attraction and Ecosystem Resilience A minimal model of an ecosystem describes the change over time of an unwanted ecosystem property, x such as lake turbidity a represents an environmental factor that promotes x, b represents the rate at which x decays in the system, r is the rate at which x recovers again as a function f of x The form of f(x) determines whether multiple stable states or attractors will exist For lakes, one can think of x as phytoplankton suspended in nutrients causing turbidity, of a as nutrient loading, of b as nutrient removal rate and of r as internal nutrient recycling. For desertification, one could interpret x as barren soil, a as vegetation destruction, b as recolonization of barren soil by plants and r as erosion by wind and runoff. Following Holling, we here use the term 'resilience' to refer the size of the valley, or basin of attraction, around a state, which corresponds to the maximum perturbation that can be taken without causing a shift to an alternative stable state. In systems with multiple stable states, gradually changing conditions may have little effect on the state of the ecosystem, but nevertheless reduce the size of the attraction basin (Fig. 3). This loss of resilience makes the system more fragile in the sense that can easily be tipped into a contrasting state by stochastic events. Such stochastic fluctuations may often be driven externally; however, they can also result from internal system dynamics. The latter can happen if the alternative attractors are 'cycles' or 'strange attractors', rather than equilibria. A system that moves along a strange attractor fluctuates chaotically even in the absence of an external stochastic forcing. These fluctuations can lead to a collision with the boundary of the basin of attraction, and consequently induce a switch to an alternative state. This system can also serve as an example of stochastic resonance where a weak forcing can emerge if it is reinforced by background noise that takes peaks in the forcing over system thresholds-e.g. ice age periods in climate records. (Figs from Scheffer et al, 2001, Nature)

What do we mean by stable states? For linear systems stable attractors will result in the system state settling down to an equilibrium or oscillating around it However, most real systems in nature are described by non-linear equations and most attractors are not of this stable kind. Strange attractors may contain periodic attractors within themselves but mostly, the system never revisits an earlier state. Stability in real systems means staying within some ‘basin of attraction’. The boundary of a strange attractor is a fractal. The system may flip between states as a result of an external perturbation or internal dynamics-stochastic resonance and ice ages. Dynamical systems theory builds on traditional mathematical approaches using continuous systems and de’s. Despite some elegant theory it has not told us much yet about self organization and emergence. We will return to consider how Dynamical systems theory fits within the corpus of CSS later Linear dynamics Non-linear dynamics Boundaries of Strange Periodic attractor Strange Attractor Attractors are Fractal

We can represent most systems as networks with interactions across the links- Network Topologies control System behaviour Regular Network: each node has the same number of connections Homogeneous network: Number of connections per node varies but there is a clear average value. Networks like this can result from randomly connecting nodes. Near the phase transition they are vulnerable to random removal of links There is a good deal of information about the possible behaviour of systems encoded in the topology of the network The small world factor suggests that more detailed topological properties can be important. Delving a little deeper into network structure we can make links between the way a network forms and the behaviour of the system it represents Heterogeneous or ‘scale free’ network: There is no average number of connections per node: Living networks that grow by accretion often have this dendritic form. They are resilient to random removal of links but vulnerable to a targeted attack that removes a key node

3. Adaptive Systems can be illustrated simply using Cellular Automata 3 Adaptive Systems can be illustrated simply using Cellular Automata. CAs are Systems that evolve on lattices according to local interaction rules The simplest rules: the state of a cell at time T+1 is determined by its own state and that of its two neighbours at time T So far we have taken a static view of these middle-ground systems by concentrating on their network topology but understanding emergence requires a dynamic view. Let’s start with the simplest abstraction of a dynamic interacting system, a cellular automaton. Here we start from a single black cell and let the CA evolve according to the rule shown. This is one of 256 ‘3-cell’ rules This is a 1D CA but we can have CAs of any dimension

Discretization of PDEs yields Cellular Automata Advection-diffusion equation -1 +1 X= We can derive CAs from differential equation, in fact we do whenever we solve a de numerically We can translate continuous differential equations as cellular automata by discretizing them. This of course is how we solve them numerically on digital computers. Look at the dicretized advection-diffusion equation as a CA Diffusion of smoke from a chimney, pheremones from an animal,… Plume grows like t^1/2 or x^1/2 after initially growing like t or x. t=n t=n+1

We can form new types of Cellular Automata by changing the interaction rules or the wiring or both Dynamics on networks can evolve either by changes in the interaction rules T=0 T=1 T=2 It is easy to devise new types of cellular automata by changing the rules or the network organisation. The network dynamics is the pattern of changes to the network elements in space and time. The interaction rules determine how elements of a network respond to information transmitted from other elements along the connections between the elements or nodes. CAs have very simple rules. In deciding what state they will adopt at the next time step, they consult only their near neighbours (and their own state) at the present time step. Clearly much more complex interaction rules are possible. There are 120 possible states of a cell and its 4 neighbours. In response to these states the rule can say turn black or white next step so there are 2^n+1 possible rules on 3 cells. We can change the way that a given initial state (pattern of cells at time zero in our CA) will evolve by changing the interaction rules or the connections between the elements. We can change which and how many cells interact and confine ourselves to the present or present + past time steps, i.e., introduce memory. Or by changes in the ‘wiring’ of the network

The Cellular Automaton as a computer: Evolving the local rules that will perform a computational task by applying a global selection pressure T=0 T=1 T=2 The colour that a cell adopts at the next timestep depends only on the colours of itself and its neighbours at the present time step An example of emergence-evolving slightly intelligent behaviour by applying global selection to local interaction rules. Here we have a system, a cellular automation (CA), that is going to teach itself how to solve a problem At time zero the cells are in a random pattern of black and white. Each cell adopts a colour-black or white at the next time step according to local interaction rules-they look at their nearest neighbours and the rule tells them whether to be black or white at the next time step. The problem to solve is-if most of the initial cells are white, then find a local rule so all cells are white after a given number of steps Density problem The local rules are evolved using a genetic algorithm-Darwinian selection on a computer-and the rules that solve the problem quickly emerge Wim Hordjik SFI 2000 In this case the program is in the rules and wiring. Thje data are the first row at time zero. Note that just as in the Von Neumann CA, groups of cells organize as the computation proceeds. This organisation on many scales is a characteristic feature of systems that have spontaneously evolved to an ordered state SOC or edge of chaos Rules are recombined (bred) and selected according to Darwinian principles to find the set of local rules that will solve the density problem

Moving Away from Classical Mathematics With complete freedom to stipulate rules and wiring between elements of our CAs in the virtual world of the computer and then to let them evolve as part of the computation, we can form mathematical objects that are very difficult to capture using the approaches of conventional mathematics but which match very well what we observe in living systems. Agent Based Models exploit this freedom Analysis using network theory and similar techniques is leading to increasing understanding of these systems-but so far we have few general principles It is very easy to go beyond the space of CAs that can be generated by translating classical mathematics-des or statistical or probabilistic approaches to the CA space. This is especially true when we add Darwinian ideas such as breeding and selection to evolve the rules of the CAs. At a less abstract level we encounter the field of agent based models (MASs). In these cases the objects are much more sophisticated than cells that can be on or off or B or W BUT as we shall see later, these much more complex systems can be reduced (in principle) to CAs. Use the CA as a generic term-we can make the elements of our virtual systems much more complex than a binary cell but we will see that all such objects can be reduced to an equivalent CA.

Summary Complex Systems Science brings together systems approaches and a rapidly developing science of systems Rather than being any particular set of techniques it is primarily the adoption of a different point of view That is, to admit the prevalence of self- organisation in complex systems together with the behaviours that flow from that and the techniques necessary to study it.

The CSIRO Centre for Complex Systems Science-A Virtual Centre The Core Group comprises a permanent Science Director, a Communication and Training Manager, Post Docs, PhDs and visitors. It interacts with Division based projects to do basic research in CSS. Projects are located within CSIRO Divisions (and partner Institutions). A key function of the Core group is to manage interaction between the projects. The Core and the Division-based Projects are closely networked. We have called it a virtual Centre because we see it being heavily reliant on new telecollaboration methods to connect its parts. The projects in CSIRO Divisions and other GRO’s are the motivation and the means of application. Nevertheless, we expect that for the first 2-3 years we will be operating in a theory-limited rather than a data-limited mode and so we need to manage for maximum creativity.

The Compass of Complex System Science: Projects in the CSIRO Centre for CSS-1 Inference of complex systems properties from fragmentary information (Mantle dynamics and mineralization: State Space reconstruction) Ensemble Prediction of Atmospheric and Ocean-Atmosphere Regime Transitions (Dynamical Systems theory) The stability of the Southern Ocean overturning circulation (Dynamical Systems theory) Critical states in bushfires (Dynamical Systems theory, Agent Based Modelling) The effects of model structure and dimensionality on the emergent properties of ecosystem models (Estuarine systems: Dynamical Systems theory, ABM) Rapid shifts in state and resilience in river systems (ABM) Targeting Drug-like properties in Chemical libraries (Genetic Algorithms, Evolutionary computing) What kind of problems have emergence of structure through self oprganisation as a characteristic feature?

The Compass of Complex System Science: Projects in the CSIRO Centre for CSS-2 Tracking Air Borne Chemical Signals (Fractal turbulence + AI, ABM) Adaptation and resilience in regional socio-economic systems (Managed rangelands: ABM) Multiscale modelling in Industrial and Natural systems (Lattice-Boltzmann methods, Non-Equil Thermodynamics) Interactions, information sharing and simulated reasoning of fishers in an agent-based, Bayesian network model of fishing behaviour (ABM) The Future of the Swan River: Governance and Agent Based Modeling (ABM, Evolutionary game theory) Our National Electricity Market as a Complex Adaptive System (ABM) Links between resilience and information in complex adaptive systems (ABM, Evolutionary game theory) All of these projects are linked intellectually by a focus on emergence and practically by a project and funding structure that both encourages and mandates interaction across discplines

The Purpose of This Workshop Is to bring together workers with awareness of the problems and workers with knowledge of CSS Techniques And to start a process of developing projects for future joint funding We plan a funding round built around the 2nd CSIRO CSS workshop in Sydney 27-29 August, 2003.