Complex Systems Modeling and Networks Summary of NECSI Course CX202 11 February 2009 John M. Linebarger, PhD Sandia National Laboratories Interactive Systems Simulation & Analysis (06344) Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

Outline Overview What are Complex Systems? Complex Networks Metrics Types of Networks Modeling and Simulation of Complex Systems How to Model Types of Models Further Reading Class Project Overview of Design for Tractable Analysis (DTA) framework Essentially, a “mind dump” of the class with a few practical reflections at the end

Overview I attended a Complex Systems Modeling and Networks (CX202) course from 23-27 June 2008 I used all 32 of my FY 2008 Strategic Education Initiative (SEI) hours on the course; the balance was covered by the Systems Analysis LDRD. The course was held at the New England Complex Systems Institute (NECSI) on the campus of the Sloan School at MIT, in Cambridge, MA The course prerequisite is CX201, Complex Physical, Biological and Social Systems I fulfilled the prerequisite by reading Making Things Work by Yaneer Bar-Yam, and Linked by Albert-László Barabási, before the course began The follow-on course is CX203, Methods for the Study of Complex Systems, which is mathematically-oriented

Overview (cont’d) Over 30 attendees from all over the world, predominately PhD students and academics Only three or four people from “industry” My project group consisted of a Norwegian PhD student, a South Korean health care researcher (working in the Boston area), an Iranian PhD student (studying in Maryland), and a Bulgarian PhD student (studying in France) The course was divided into two parts The first part was complex network analysis, taught by Dan Braha of NECSI and UMass Dartmouth The second part was modeling and simulation of complex systems, taught by Hiroki Sayama of NECSI and SUNY Binghamton Project groups met each evening, often until quite late, to work on their course project Projects (and the associated papers) were presented to the class on Friday, followed by the final exam for the course

NECSI Course Participants

What are Complex Systems? Complex systems are characterized by emergent, self-organizing, collective behavior Such emergent collective behavior often arises due to the combination of simple individual behavior patterns The components of a complex system are interdependent Complex systems exhibit multiscale variety The structure of the system differs depending on the level at which it is viewed A common pattern is a mixture of competition and cooperation at different levels of the system Complex systems arise due to evolutionary, not deterministic, processes Engineered systems should be implemented this way too Examples of complex systems include the health care system, the education system, and recent military conflict From Making Things Work by Yaneer Bar-Yam

Another Characterization of Complex Systems Complex systems contain many constituents interacting nonlinearly The constituents of a complex system are interdependent A complex system possesses a structure spanning several scales A complex system is capable of emerging behavior Complexity involves an interplay between chaos and non-chaos “Complex systems dance on the edge of chaos” Complexity involves an interplay between cooperation and competition From “Chaos, Complexity, and Entropy” by Michel Baranger

Wicked Problems Every wicked problem is essentially unique There is no definitive formulation of a wicked problem Wicked problems are never solved A wicked problem is not understood until after the formulation of a solution Solutions to wicked problems change the problem itself Solutions to wicked problems are not true or false, but better or worse Stakeholders have radically different world views and different frames for understanding the problem Conclusion Wicked problems are a proper subset of complex systems All wicked problems are complex systems, but not all complex systems are wicked problems

Dancing on the Edge of Chaos Two fundamental assumptions of most systems analysis are that the system is continuous, and that if it is decomposed far enough it becomes simple It can then be analyzed without loss of fidelity These assumptions may be driven by the predominant analytical tool of 20th century physics, which was calculus For both derivatives and integrals, if you zoom in far enough a curve becomes a straight line Thus a calculus problem is transformed into a geometry problem, which has been well understood since Euclid However, chaotic systems are often discontinuous, and the decomposition of chaotic systems generally yields more complexity e.g., fractals, Sierpiński triangles “The arrogance of analysis” Yet chaotic systems often are simple, not in terms of their decomposition, but in terms of the rules that govern their behavior e.g., “boids,” where simple rules lead to complex flocking behaviors

Complex Networks The interdependencies of complex systems are often modeled by networks Networks can be analyzed using several important metrics from graph theory Directed (nodes and arcs) vs. undirected graphs (nodes and edges) Number of nodes (N): The number of nodes in a network Degree (k): The number of edges or arcs incident on a node Average degree is denoted as <k> Can distinguish between in degree and out degree Node density is actual connections between nodes divided by potential connections that could exist between nodes Note that real-world networks are quite sparse, and have an average density of 10-4 Characteristic path length (L): The average length of the shortest path in the network between two nodes Clustering coefficient (C): Number of actual links between neighbors of a node divided by the number of possible links between those neighbors This can be thought of as local density, the probability that two neighbors know each other The clustering coefficient is a measure of modularity

Random Networks For years (since the late 1950’s) the dominant model of networks was the random network, otherwise known as the Erdős-Rényi model, even though it is rarely seen in nature Random graphs have a fixed number of nodes which are connected randomly by undirected edges. The number of edges depend on a specified probability. The seminal paper is “On Random Graphs,” by Erdős and Rényi (1959)

Small-World Networks Adding a small number of links to a regular ring network, in which only the nearest neighbors are connected, can have dramatic effects The characteristic path length becomes quite small with only a small number of such cross-neighborhood links Small-world networks are pervasive in nature: Friend networks, power grids, predator-prey networks, and even neural networks of the brain. The seminal paper is “Collective Dynamics of ‘Small-World’ Networks,” by Watts and Strogatz (1998)

Scale-Free Networks Scale-free networks are so-called because they have no characteristic scale, or average number of links per node Instead, they are characterized by numerous hubs that hold together highly connected regions of the network However, direct connections between hubs are rare (for reasons of resilience) Scale-free networks are also pervasive in nature: Disease propagation, terrorist networks, and the World-Wide Web The seminal paper is “Emergence of Scaling in Random Networks,” by Barabási and Albert (1999)

Scale-Free Networks (cont’d) Scale-free networks seem to follow a power law distribution (which is a variant of a Pareto distribution, or the 80/20 rule) Earlier models (such as Erdős-Rényi) described networks using a bell curve, with many nodes with the same number of links and no highly connected nodes The scale free model describes networks as having many nodes with only a few links, and a few nodes with many links Power law functions appear as straight lines on a log-log plot Power Law Distribution Bell Curve Many nodes with few links On a log-log plot, if the function is not a power law you get a banana, not a straight line Most nodes have the same number of links # of nodes with k links # of nodes with k links A few nodes with many links No highly connected nodes # of links (k) # of links (k)

Scale-Free Networks (cont’d) Probability is of the number of nodes with exactly k links. “Banana” vs. straight line when plotted on a log-log scale. Pk ~ e-k Pk ~ k-γ

Scale-Free Networks (cont’d) In a more formal sense, a scale-free network is characterized by two properties Continuous growth (the other network models did not consider growth) Preferential attachment (e.g., “the rich get richer”) The number of nodes with exactly k links follows a power law for any value of k P(k) ~ k-γ Often 2 < γ < 3 However, many real-world networks actually exhibit “stretched exponential” behavior, becoming exponential past a threshold Cost of adding new links Aging From Faloutsos, M., P., & C., “On Power-Law Relationships of the Internet Topology” (1999)

Topology is Destiny! Characterizing a network quantitatively gives insight into the dynamics (performance) and vulnerabilities of the network Random graphs have a short path length (good), but also a small clustering coefficient (not good) Regular ring networks have a large clustering coefficient (good) but long path length (not good) Small-world networks are a balance between those two extremes, but do not have the robustness of scale-free networks Scale-free networks (which are similar in some ways to small-world networks) are robust against random attacks (good) but not against targeted attacks (not good) The hubs are vulnerable when directly targeted The network can become disconnected fairly rapidly in this kind of attack The moral of the story: When it comes to networks (and to everything else in life), you can’t have it all …

How Complex Network Research is Done A network under study is represented and analyzed mathematically A randomly generated “ensemble” of networks is created with which to compare to the network under study Ensembles of several types of networks are generated, such as random graphs, small world, regular ring, and scale-free The network ensembles are analyzed mathematically and compared to the network under study Both the network under study and the generated ensembles are subjected to the same process (e.g., attack), generally via simulation, and the results are compared

Tools for Complex Network Analysis Pajek, an open source tool that generates and visualizes networks A useful book is Exploratory Social Network Analysis with Pajek, by de Nooy et al. UCINET, a commercial tool from Analytic Technologies that quantitatively analyzes networks Python with the NetworkX package MATLAB, a commercial tool from MathWorks, which can generate, visualize, and quantitatively analyze networks after “a simple matter of programming” Free network analysis packages are available for MATLAB

Modeling and Simulation of Emergent Behavior Fundamentals of Modeling Modeling and Simulation of Dynamical Systems (complex interdependent systems that can exhibit emergent behavior) Non-Spatial Modeling Cellular Automata Agent-Based Models Network Models Other Simulation Paradigms for Dynamical Systems (not covered in the course) Systems Dynamics Numerous commercial packages exist, such as Vensim Can be described in terms of difference or differential equations, so can be addressed by non-spatial modeling Spatial models with partial differential equations “Soft computing” (e.g., evolutionary computation, artificial neural networks, or support vector machines)

Fundamentals of Modeling Scientific “laws” are really hypotheses, models of observed behavior There is no way to prove anything about nature, although it is easy to disprove something A model is a simplified representation of a system Rule-based modeling allows the system’s possible or future states to be extrapolated, unlike descriptive modeling, which just specifies the system’s actual or past states A good model is simple, correct, and robust Correct means that reality corresponds to predictions. Robust means that the predictions are stable even if the assumptions undergo minor perturbations.

Modeling Procedure In general Observe the system of interest Reflect on the possible rules that might cause the system’s characteristics Derive predictions from those rules and compare them with reality Repeat to modify the rules until satisfied For complex systems Define the key questions and choose the right scale of modeling Identify the structure (the parts and their connections) of the system Define the possible states for each part (state space) Describe how the state of each part changes over time through interactions with other parts Models must be evaluated and validated, in terms of simplicity, correctness, and robustness

Tools Used for Modeling and Simulation Python NumPy and SciPy (MATLAB-like numerical packages for Python) matplotlib (MATLAB-like visualization library for Python) NetworkX (network graphing package) Agent-based modeling toolkits, such as NetLogo, StarLogo, or Swarm (see the Swarm Wiki for other tools) And, of course, MATLAB …

Non-Spatial Modeling The simplest mathematical model Called “non-spatial” because the spatial location of the elements of the system is unimportant A dynamical system is a system whose state is uniquely specified by a finite set of variables and whose behavior is uniquely determined by predetermined rules Examples are simple population growth, simple pendulum swing, motion of celestial bodies Two schools of modeling dynamical systems Discrete-time model (using difference/recurrence equations or iterative maps) Continuous-time model (using differential equations) Non-autonomous, higher-order equations can always be converted into autonomous, first order equations, by introducing additional variables Non-linear systems are generally simulated because they cannot be solved analytically Exercise: Logistic growth (logisticgrowthmodel.py)

Cellular Automata Adds spatio-temporal dynamics, especially for locally connected networks, to the modeling process Cells in a regular grid interact with their nearest neighbors according to simple interaction rules Transition from one state to another depends on simple rules based on interactions with neighbors The resulting system is a linear system Example: Simple diffusion, panic in a gym Exercise: Decide whether or not to panic based on the state of a simple majority of your neighbors (majority.py)

Agent-Based Models The only way to study the behavior of agent-based models is to simulate, since no global equation exists, only equations at the agent level Agent characteristics (what makes an agent an agent) Local state Spatial localization Local interaction with other agents based on predefined rules No central supervisor May learn autonomously May produce non-trivial collective behavior

Agent-Based Models (cont’d) To develop an agent-based model Identify the agents and get a theory of agent behavior Identify the agent relationships and get a theory of agent interaction Get the requisite agent-related data Validate the agent behavior models in addition to the model as a whole (thus you must justify your model relative to the real-world, including emergent behavior) Run the model and analyze the output Exercise: Garbage collection by ants (ants.py) Eventually, only one pile of garbage remains No simulation of the environment is needed, just tracking of the environment; however, if pheremones were emitted and decayed with time, the environment would need to be simulated as well, based on a model of pheremone decay An excellent two-part tutorial was presented by Macal and North at the Winter Simulation Conference in 2005 and 2006, respectively

Network Models Simulation of the types of complex networks we learned about in the first part of the course Uses the NetworkX package from LANL Has methods to create random networks, small-world networks, and scale-free networks Exercise: A disease epidemic on a random network (networkepidemics.py) Two parameters: Infection probability and recovery probability Red means infected; yellow means susceptible Note that the disease is never completely eradicated from a social network; this is called the fixation of a disease Can experiment with different network topologies

Further Reading Complexity and Chaos Making Things Work, by Yaneer Bar-Yam (2005) Ubiquity: The Science of History … or Why the World is Simpler than We Think (a.k.a. “Ubiquity: Why Catastrophes Happen”), by Mark Buchanan (2001, 2002) Chaos: Making a New Science, by James Gleick (1988) The Dynamics of Complex Systems, by Yaneer Bar-Yam (1997) Complex networks Linked, by Albert-László Barabási (2003) Small Worlds: The Dynamics of Networks between Order and Randomness, by D. J. Watts (2003) Exploratory Social Network Analysis with Pajek, by de Nooy, et al. (2005) Modeling and simulation of dynamical systems Cellular Automata, by Andrew Ilachinski (2001) Python Scripting for Computational Science, 3rd ed., by Hans Petter Langtangen (2006) Simulation Modeling and Analysis, 4th ed., by Averill Law (2007)

Class Project