Complex network of the brain I Small world vs. scale-free networks Jaeseung Jeong, Ph.D. Department of Bio and Brain Engineering, KAIST

What is complexity? The complexity of a physical system or a dynamical process expresses the degree to which components engage in organized structured interactions. High complexity is achieved in systems that exhibit a mixture of order and disorder (randomness and regularity) and that have a high capacity to generate emergent phenomena. There is a broad spectrum of ‘measures of complexity’ that apply to specific types of systems or problem domains. Despite the heterogeneous approaches taken to defining and measuring complexity, the belief persists that there are properties common to all complex systems.

General features of complexity Herbert Simon was among the first to discuss the nature and architecture of complex systems (Simon, 1981) and he suggested to define complex systems as those that are “made up of a large number of parts that have many interactions.” Simon goes on to note that “in such systems the whole is more than the sum of the parts in the […] sense that, given the properties of the parts and the laws of their interaction, it is not a trivial matter to infer the properties of the whole.”

Components. Many complex systems can be decomposed into components (elements, units) having local dynamics, which are decomposable into subcomponents, resulting in systems that are complex at multiple levels of organization, forming hierarchies of nearly, but not completely decomposable components. Interactions. Components of complex systems engage in dynamic interactions, resulting in integration or binding of these components across space and time into an organized whole. Interactions often modulate the individual actions of the components, thus altering their local functionality by relaying global context. Emergence. Interactions between components in integrated systems often generate phenomena, functions, or effects that cannot be trivially reduced to properties of the components alone.

Hierarchical structure of the brain

Vertex (node) and edge (link) Any system can be expressed as a network with nodes and links.

Examples of complex networks: geometric, regular Eileen Kraemer

Examples of complex networks: semi-geometric, irregular Eileen Kraemer

Structural metrics: Average path length

Structural Metrics: Degree distribution(connectivity)

Structural Metrics: Clustering coefficient A clustering coefficient is a measure of the degree to which nodes in a graph tend to cluster together.

Milgram’s experiment: Six degrees of separation Milgram's experiment was designed to measure these path lengths by developing a procedure to count the number of ties between any two people. Milgram's experiment developed out of a desire to learn more about the probability that two randomly selected people would know each other, as one way of looking at the small world problem. An alternative view of the problem is to imagine the population as a social network and attempt to find the average path length between any two nodes.

Milgram’s experiment: Procedure Milgram chose individuals in an U.S. city (Omaha) to be the starting points and Boston, Massachusetts, to be the end point of a chain of correspondence. Letters were initially sent to "randomly" selected individuals in Omaha. The letter detailed the study's purpose, and basic information about a target contact person in Boston. It additionally contained a roster on which they could write their own name, as well as business reply cards that were pre-addressed to Harvard. Upon receiving the invitation to participate, the recipient was asked whether he or she personally knew the contact person described in the letter. If so, the recipient could forward the letter directly to that person. (Knowing someone "personally" was defined as knowing them on a first-name basis.) These cities were selected because they were thought to represent a great distance in the United States, both socially and geographically.

Milgram’s experiment: Procedure If the recipient did not personally know the target person, the recipient was to think of a friend he knew personally who was more likely to know the target. He was then directed to sign his name on the roster and forward the letter to that person. A postcard was also mailed to the researchers at Harvard so that they could track the chain's progression toward the target. When and if the package eventually reached the target person in Boston, the researchers could examine the roster to count the number of times it had been forwarded from person to person. (Additionally, for packages that never reached the destination, the incoming postcards helped identify the break point in the chain.)

Milgram’s experiment: Results Six degrees of separation In one case, 232 of the 296 letters never reached the destination because the recipients refulsed to participate in this study. However, 64 of the letters eventually did reach the target contact. Among these chains, the average path length fell around ‘five and a half or six.’ Hence, the researchers concluded that people in the United States are separated by about six people on average.

Six degrees of Kevin Bacon

Kevin Bacon game: Six degrees of Kevin Bacon

Kevin Bacon Game

D. J. Watts and Steven Strogatz (June 1998) D. J. Watts and Steven Strogatz (June 1998). "Collective dynamics of 'small-world' networks". Nature 393 (6684): 440–442.

D. J. Watts and Steven Strogatz (June 1998) D. J. Watts and Steven Strogatz (June 1998). "Collective dynamics of 'small-world' networks". Nature 393 (6684): 440–442.

D. J. Watts and Steven Strogatz (June 1998) D. J. Watts and Steven Strogatz (June 1998). "Collective dynamics of 'small-world' networks". Nature 393 (6684): 440–442.

A small-world network is a type of mathematical graph in which most nodes are not neighbors of one another, but most nodes can be reached from every other by a small number of hops or steps.

Scale-free network

The scale-free nature of the web of sexual contacts. They analyze data gathered in a 1996 Swedish survey of sexual behavior. The survey--involving a random sample of 4781 Swedish individuals (ages 18-74 yr)--used structured personal interviews and questionnaires to collect information. The response rate was 59 percent, corresponding to 2810 respondents. Connections in the network of sexual contacts appear and disappear as sexual relations are initiated and terminated. To analyze the connectivity of this dynamic network, whose links may be quite short lived, we first analyze the number k of sex partners over a relatively short time window--the twelve months prior to the survey.

Preferential attachment (Albert-Lazlo Barabasi) A preferential attachment process is any of processes in which links are distributed among a number of nodes according to how much they already have links, so that those who are already wealthy receive more than those who are not. (The rich get richer)

Preferential attachment (Albert-Lazlo Barabasi)

Models Erdös-Rényi  Homogeneous Each possible link exists with probability p Scale-free  Heterogeneous The network grows a node at a time The probability i that the new node is connected to node i is proportional to know many links node i owns (preferential attachment)

Brain and complex network (graph) theory Undirected graph Directed graph Weighted graph Boccaletti et al., 2006 Liu, 2008 Node (vertex) : Brain region or voxel, channel of EEG/MEG Link (edge) : Functional or anatomical connection between nodes Network analysis can reveal structural and functional organization of the brain (Liu, 2008)

Constructing Brain Networks Bullmore and Sporns, 2009

Brain is a small-world network Watts and Strogatz, 1998 high C high C low C high L low L low L high clustering coefficient (C) – high resilience to damage in local structures low average path length (L) – high level of global communication efficiency Brain functional network has small-world structure, while this property may be disrupted in damaged brain such as AD (vulnerability to damages, decreased communication efficiency between distant brain regions … )

Small-world and scale-free organization of voxel-based resting state functional connectivity in the human brain van den Heuvel et al., Neuroimage, 2008 normal, resting-state, voxel-based(N=10,000), zero-lag temporal correlation, bandpass-filtered (0.01-0.1Hz), unweighted, small-world and scale-free : optimal network organization balance between maximum communication efficiency and minimum wiring AD, damage modeling, weighted graph, efficiency

A resilient, low-frequency, small-world human brain functional network with highly connected association cortical hubs (Achard et al., The Journal of Neuroscience, 2006) MODWT (Maximal Overlap Discrete Wavelet Transform) at 6 frequency scales healthy young subjects, resting-state, Parcellation (90 region-based), unweighted, small-world, NOT scale-free resilient to targeted attack than SF network AD, voxel-based, weighted, efficiency, region attack

Alzheimer vs. Healthy subjects

Why does the brain process information so quickly?

What do we do in complex network? Basic Properties – Small World – Scale Free – Other measurements What do we do in complex network? • We analyze the structure of (big) networks from the real-world to understand which properties are underlying them. • If a general class of network has a given property then we can use it to reason about any unknown network of this class. • There is a tremendous number of applications and since the main two properties were discovered in 1998, there has been hundreds if not thousands of papers on complex networks. Property: some individuals are very social compared to other ones. Goal: spread a saucy rumor! Biological network What is the influence of A over C in the social network? How strong is the connection from A to C? Social Network Idea: whatever the network as long as it’s a social network, try to target the social individuals. How likely is it that if A is infected by a virus then C will get infected? Blogs Facebook Population 2

Basic Properties – Small World – Scale Free – Other measurements Transitivity 2 1 1 1 1 triangle 2 1 8 connected triples 2 C = 3/8 = 0,375 ►Transitivity measures the probability that if A is connected to B and B is connected to C, then A is connected to C. « There are high chances that a husband knows the family of his wife. » 3

Basic Properties – Small World – Scale Free – Other measurements Network Motifs • When we looked for transitivity, we basically counted the number of subgraphs of a particular type (triangles and triples). • We can generalize this approach to see which patterns are ‘very frequent’ in the network. Those patterns are called network motifs. • To measure the frequency, we compare with how expected it is to see such patterns in a random network. • For each subgraph, we measure its relative frequency in the network. • As we are measuring for the 13 possible directed connected graphs of 3 vertices, it is called a triad significance profile (TSP). • The significance profile (SP) of the network is a vector of those frequencies. • 4 networks of different micro-organisms are shown to have very similar TSPs, and in particular the triad 7 called « feed-forward loop ». 4

The Small-World property Basic Properties – Small World – Scale Free – Other measurements The Small-World property • Through this Kevin Bacon’s experiment, we know that although the network of actors is quite big, the average distance is very small. Global efficiency of small-world networks. • A network is said to have the small-world property if the average shortest path L is at most logarithmically on the network size N. Efficient to exchange information at a local scale. → An e-mail network of 59 812 nodes… L = 4.95 ! Efficient to exchange information at a global scale. → Actor network or 225 226 actors… L = 3.65 ! • It tells you that transmitting information in small-world networks will be very fast. At a local level, we have strongly connected communities. And so, transmitting viruses will be fast too… • Some authors defined the small-world property with an additional constraint with the presence of a high clustering. It’s a choice… 6

The Scale-Free property Basic Properties – Small World – Scale Free – Other measurements The Scale-Free property However, there are things that have an enormous variation in the distribution. Many of the things we measure are centered around a particular value. If we plot this histogram with logarithmic horizontal and vertical axis, a pattern will clearly emerge: a line. In a normal histogram, this line is p(x) = -αx + c. Here it’s log-log, so: This value is the typical size. ln p(x) = -α ln x + c apply exponent e p(x) = ecx c -α We say that this distribution follows a power-law, with exponent α. A power law is the only distribution that is the same whatever scale we look at it on, i.e. p(bx) = g(b)p(x). So, it’s also called scale-free. 7

The Scale-Free property Basic Properties – Small World – Scale Free – Other measurements The Scale-Free property We found that the population has the scale-free property! In 1955, Herbert Simon already showed that many systems follow a power law distribution, so that’s neither new nor unique. • Sizes of earthquakes • Wars • Moon craters • Number of citations received / paper • Solar flares • Number of hits on web pages • Computer files • People’s annual incomes It has been found that the distribution of the degree of nodes follows a power-law in many networks, i.e. many networks are scale-free… What is important is not so much to find a power-law as it’s common, but to understand why and which other structural parameters can be there. 8

The Scale-Free property Basic Properties – Small World – Scale Free – Other measurements The Scale-Free property Myth and reality • Scaling distributions are a subset of a larger family of heavy-tailed distributions that exhibit high variability. • One mechanism was used to build scale-free networks, called preferential attachment, or « the rich get richer ». • It was said that « the most highly connected nodes represent an Achilles’ heel »: delete them and the graph breaks into pieces. • One important claim of the litterature for scale-free networks was the presence of highly connected central hubs. • It is only one of several, and not less than 7 other mechanisms give the same result, so preferential attachment gives little or no insight in the process. • Recent research have shown that complex networks that claimed to be scale-free have a power-law but not this Achilles’ heel. • However, it only requires high variability and not strict scaling… 9

Basic Properties – Small World – Scale Free – Other measurements • We have the clustering, distribution of degree, etc. Are there other global characteristics relevant to the performances of the network, in term of searchability or stability? • Rozenfeld has proposed in his PhD thesis to study the cycles, with algorithms to approximate their counting (as it’s exponential otherwise). • Using cycles as a measure for complex networks has received attention: Inhomogeneous evolution of subgraphs and cycles in complex networks (Vazquez, Oliveira, Barabasi. Phys. Rev E71, 2005). Degree-dependent intervertex separation in complex networks (Dorogovtsev, Mendes, Oliveira. Phys Rev. E73 2006) • See also studies on the correlation of degree (i.e. assortativity). 10