1. Complex Systems Modeling and Networks
Summary of NECSI Course CX202
11 February 2009
John M. Linebarger, PhD
Sandia National Laboratories
Interactive Systems Simulation & Analysis (06344)
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

2. Outline Overview What are Complex Systems? Complex Networks Metrics
Types of Networks
Modeling and Simulation of Complex Systems
How to Model
Types of Models
Further Reading
Class Project
Overview of Design for Tractable Analysis (DTA) framework
Essentially, a “mind dump” of the class with a few practical reflections at the end

3. Overview
I attended a Complex Systems Modeling and Networks (CX202) course from June 2008
I used all 32 of my FY 2008 Strategic Education Initiative (SEI) hours on the course; the balance was covered by the Systems Analysis LDRD.
The course was held at the New England Complex Systems Institute (NECSI) on the campus of the Sloan School at MIT, in Cambridge, MA
The course prerequisite is CX201, Complex Physical, Biological and Social Systems
I fulfilled the prerequisite by reading Making Things Work by Yaneer Bar-Yam, and Linked by Albert-László Barabási, before the course began
The follow-on course is CX203, Methods for the Study of Complex Systems, which is mathematically-oriented

4. Overview (cont’d)
Over 30 attendees from all over the world, predominately PhD students and academics
Only three or four people from “industry”
My project group consisted of a Norwegian PhD student, a South Korean health care researcher (working in the Boston area), an Iranian PhD student (studying in Maryland), and a Bulgarian PhD student (studying in France)
The course was divided into two parts
The first part was complex network analysis, taught by Dan Braha of NECSI and UMass Dartmouth
The second part was modeling and simulation of complex systems, taught by Hiroki Sayama of NECSI and SUNY Binghamton
Project groups met each evening, often until quite late, to work on their course project
Projects (and the associated papers) were presented to the class on Friday, followed by the final exam for the course

5. NECSI Course Participants

6. What are Complex Systems?
Complex systems are characterized by emergent, self-organizing, collective behavior
Such emergent collective behavior often arises due to the combination of simple individual behavior patterns
The components of a complex system are interdependent
Complex systems exhibit multiscale variety
The structure of the system differs depending on the level at which it is viewed
A common pattern is a mixture of competition and cooperation at different levels of the system
Complex systems arise due to evolutionary, not deterministic, processes
Engineered systems should be implemented this way too
Examples of complex systems include the health care system, the education system, and recent military conflict
From Making Things Work by Yaneer Bar-Yam

7. Another Characterization of Complex Systems
Complex systems contain many constituents interacting nonlinearly
The constituents of a complex system are interdependent
A complex system possesses a structure spanning several scales
A complex system is capable of emerging behavior
Complexity involves an interplay between chaos and non-chaos
“Complex systems dance on the edge of chaos”
Complexity involves an interplay between cooperation and competition
From “Chaos, Complexity, and Entropy” by Michel Baranger

8. Wicked Problems Every wicked problem is essentially unique
There is no definitive formulation of a wicked problem
Wicked problems are never solved
A wicked problem is not understood until after the formulation of a solution
Solutions to wicked problems change the problem itself
Solutions to wicked problems are not true or false, but better or worse
Stakeholders have radically different world views and different frames for understanding the problem
Conclusion
Wicked problems are a proper subset of complex systems
All wicked problems are complex systems, but not all complex systems are wicked problems

9. Dancing on the Edge of Chaos
Two fundamental assumptions of most systems analysis are that the system is continuous, and that if it is decomposed far enough it becomes simple
It can then be analyzed without loss of fidelity
These assumptions may be driven by the predominant analytical tool of 20th century physics, which was calculus
For both derivatives and integrals, if you zoom in far enough a curve becomes a straight line
Thus a calculus problem is transformed into a geometry problem, which has been well understood since Euclid
However, chaotic systems are often discontinuous, and the decomposition of chaotic systems generally yields more complexity
e.g., fractals, Sierpiński triangles
“The arrogance of analysis”
Yet chaotic systems often are simple, not in terms of their decomposition, but in terms of the rules that govern their behavior
e.g., “boids,” where simple rules lead to complex flocking behaviors

10. Complex Networks
The interdependencies of complex systems are often modeled by networks
Networks can be analyzed using several important metrics from graph theory
Directed (nodes and arcs) vs. undirected graphs (nodes and edges)
Number of nodes (N): The number of nodes in a network
Degree (k): The number of edges or arcs incident on a node
Average degree is denoted as <k>
Can distinguish between in degree and out degree
Node density is actual connections between nodes divided by potential connections that could exist between nodes
Note that real-world networks are quite sparse, and have an average density of 10-4
Characteristic path length (L): The average length of the shortest path in the network between two nodes
Clustering coefficient (C): Number of actual links between neighbors of a node divided by the number of possible links between those neighbors
This can be thought of as local density, the probability that two neighbors know each other
The clustering coefficient is a measure of modularity

11. Random Networks
For years (since the late 1950’s) the dominant model of networks was the random network, otherwise known as the Erdős-Rényi model, even though it is rarely seen in nature
Random graphs have a fixed number of nodes which are connected randomly by undirected edges. The number of edges depend on a specified probability.
The seminal paper is “On Random Graphs,” by Erdős and Rényi (1959)

12. Small-World Networks
Adding a small number of links to a regular ring network, in which only the nearest neighbors are connected, can have dramatic effects
The characteristic path length becomes quite small with only a small number of such cross-neighborhood links
Small-world networks are pervasive in nature: Friend networks, power grids, predator-prey networks, and even neural networks of the brain.
The seminal paper is “Collective Dynamics of ‘Small-World’ Networks,” by Watts and Strogatz (1998)

13. Scale-Free Networks
Scale-free networks are so-called because they have no characteristic scale, or average number of links per node
Instead, they are characterized by numerous hubs that hold together highly connected regions of the network
However, direct connections between hubs are rare (for reasons of resilience)
Scale-free networks are also pervasive in nature: Disease propagation, terrorist networks, and the World-Wide Web
The seminal paper is “Emergence of Scaling in Random Networks,” by Barabási and Albert (1999)

14. Scale-Free Networks (cont’d)
Scale-free networks seem to follow a power law distribution (which is a variant of a Pareto distribution, or the 80/20 rule)
Earlier models (such as Erdős-Rényi) described networks using a bell curve, with many nodes with the same number of links and no highly connected nodes
The scale free model describes networks as having many nodes with only a few links, and a few nodes with many links
Power law functions appear as straight lines on a log-log plot
Power Law Distribution
Bell Curve
Many nodes with few links
On a log-log plot, if the function is not a power law you get a banana, not a straight line
Most nodes have the same number of links
# of nodes with k links
# of nodes with k links
A few nodes with many links
No highly connected nodes
# of links (k)
# of links (k)

15. Scale-Free Networks (cont’d)
Probability is of the number of nodes with exactly k links.
“Banana” vs. straight line when plotted on a log-log scale.
Pk ~ e-k
Pk ~ k-γ

16. Scale-Free Networks (cont’d)
In a more formal sense, a scale-free network is characterized by two properties
Continuous growth (the other network models did not consider growth)
Preferential attachment (e.g., “the rich get richer”)
The number of nodes with exactly k links follows a power law for any value of k
P(k) ~ k-γ
Often 2 < γ < 3
However, many real-world networks actually exhibit “stretched exponential” behavior, becoming exponential past a threshold
Cost of adding new links
Aging
From Faloutsos, M., P., & C., “On Power-Law Relationships of the Internet Topology” (1999)

17. Topology is Destiny!
Characterizing a network quantitatively gives insight into the dynamics (performance) and vulnerabilities of the network
Random graphs have a short path length (good), but also a small clustering coefficient (not good)
Regular ring networks have a large clustering coefficient (good) but long path length (not good)
Small-world networks are a balance between those two extremes, but do not have the robustness of scale-free networks
Scale-free networks (which are similar in some ways to small-world networks) are robust against random attacks (good) but not against targeted attacks (not good)
The hubs are vulnerable when directly targeted
The network can become disconnected fairly rapidly in this kind of attack
The moral of the story: When it comes to networks (and to everything else in life), you can’t have it all …

18. How Complex Network Research is Done
A network under study is represented and analyzed mathematically
A randomly generated “ensemble” of networks is created with which to compare to the network under study
Ensembles of several types of networks are generated, such as random graphs, small world, regular ring, and scale-free
The network ensembles are analyzed mathematically and compared to the network under study
Both the network under study and the generated ensembles are subjected to the same process (e.g., attack), generally via simulation, and the results are compared

19. Tools for Complex Network Analysis
Pajek, an open source tool that generates and visualizes networks
A useful book is Exploratory Social Network Analysis with Pajek, by de Nooy et al.
UCINET, a commercial tool from Analytic Technologies that quantitatively analyzes networks
Python with the NetworkX package
MATLAB, a commercial tool from MathWorks, which can generate, visualize, and quantitatively analyze networks after “a simple matter of programming”
Free network analysis packages are available for MATLAB

20. Modeling and Simulation of Emergent Behavior
Fundamentals of Modeling
Modeling and Simulation of Dynamical Systems (complex interdependent systems that can exhibit emergent behavior)
Non-Spatial Modeling
Cellular Automata
Agent-Based Models
Network Models
Other Simulation Paradigms for Dynamical Systems (not covered in the course)
Systems Dynamics
Numerous commercial packages exist, such as Vensim
Can be described in terms of difference or differential equations, so can be addressed by non-spatial modeling
Spatial models with partial differential equations
“Soft computing” (e.g., evolutionary computation, artificial neural networks, or support vector machines)

21. Fundamentals of Modeling
Scientific “laws” are really hypotheses, models of observed behavior
There is no way to prove anything about nature, although it is easy to disprove something
A model is a simplified representation of a system
Rule-based modeling allows the system’s possible or future states to be extrapolated, unlike descriptive modeling, which just specifies the system’s actual or past states
A good model is simple, correct, and robust
Correct means that reality corresponds to predictions. Robust means that the predictions are stable even if the assumptions undergo minor perturbations.

22. Modeling Procedure In general Observe the system of interest
Reflect on the possible rules that might cause the system’s characteristics
Derive predictions from those rules and compare them with reality
Repeat to modify the rules until satisfied
For complex systems
Define the key questions and choose the right scale of modeling
Identify the structure (the parts and their connections) of the system
Define the possible states for each part (state space)
Describe how the state of each part changes over time through interactions with other parts
Models must be evaluated and validated, in terms of simplicity, correctness, and robustness

23. Tools Used for Modeling and Simulation
Python
NumPy and SciPy (MATLAB-like numerical packages for Python)
matplotlib (MATLAB-like visualization library for Python)
NetworkX (network graphing package)
Agent-based modeling toolkits, such as NetLogo, StarLogo, or Swarm (see the Swarm Wiki for other tools)
And, of course, MATLAB …

24. Non-Spatial Modeling The simplest mathematical model
Called “non-spatial” because the spatial location of the elements of the system is unimportant
A dynamical system is a system whose state is uniquely specified by a finite set of variables and whose behavior is uniquely determined by predetermined rules
Examples are simple population growth, simple pendulum swing, motion of celestial bodies
Two schools of modeling dynamical systems
Discrete-time model (using difference/recurrence equations or iterative maps)
Continuous-time model (using differential equations)
Non-autonomous, higher-order equations can always be converted into autonomous, first order equations, by introducing additional variables
Non-linear systems are generally simulated because they cannot be solved analytically
Exercise: Logistic growth (logisticgrowthmodel.py)

25. Cellular Automata
Adds spatio-temporal dynamics, especially for locally connected networks, to the modeling process
Cells in a regular grid interact with their nearest neighbors according to simple interaction rules
Transition from one state to another depends on simple rules based on interactions with neighbors
The resulting system is a linear system
Example: Simple diffusion, panic in a gym
Exercise: Decide whether or not to panic based on the state of a simple majority of your neighbors (majority.py)

26. Agent-Based Models
The only way to study the behavior of agent-based models is to simulate, since no global equation exists, only equations at the agent level
Agent characteristics (what makes an agent an agent)
Local state
Spatial localization
Local interaction with other agents based on predefined rules
No central supervisor
May learn autonomously
May produce non-trivial collective behavior

27. Agent-Based Models (cont’d)
To develop an agent-based model
Identify the agents and get a theory of agent behavior
Identify the agent relationships and get a theory of agent interaction
Get the requisite agent-related data
Validate the agent behavior models in addition to the model as a whole (thus you must justify your model relative to the real-world, including emergent behavior)
Run the model and analyze the output
Exercise: Garbage collection by ants (ants.py)
Eventually, only one pile of garbage remains
No simulation of the environment is needed, just tracking of the environment; however, if pheremones were emitted and decayed with time, the environment would need to be simulated as well, based on a model of pheremone decay
An excellent two-part tutorial was presented by Macal and North at the Winter Simulation Conference in 2005 and 2006, respectively

28. Network Models
Simulation of the types of complex networks we learned about in the first part of the course
Uses the NetworkX package from LANL
Has methods to create random networks, small-world networks, and scale-free networks
Exercise: A disease epidemic on a random network (networkepidemics.py)
Two parameters: Infection probability and recovery probability
Red means infected; yellow means susceptible
Note that the disease is never completely eradicated from a social network; this is called the fixation of a disease
Can experiment with different network topologies

29. Further Reading Complexity and Chaos
Making Things Work, by Yaneer Bar-Yam (2005)
Ubiquity: The Science of History … or Why the World is Simpler than We Think (a.k.a. “Ubiquity: Why Catastrophes Happen”), by Mark Buchanan (2001, 2002)
Chaos: Making a New Science, by James Gleick (1988)
The Dynamics of Complex Systems, by Yaneer Bar-Yam (1997)
Complex networks
Linked, by Albert-László Barabási (2003)
Small Worlds: The Dynamics of Networks between Order and Randomness, by D. J. Watts (2003)
Exploratory Social Network Analysis with Pajek, by de Nooy, et al. (2005)
Modeling and simulation of dynamical systems
Cellular Automata, by Andrew Ilachinski (2001)
Python Scripting for Computational Science, 3rd ed., by Hans Petter Langtangen (2006)
Simulation Modeling and Analysis, 4th ed., by Averill Law (2007)

30. Class Project