1. News and Notes, 2/17 New Kleinberg article added to SNT readings –Watts chapters plus four articles Homework 2 distributed today, due Feb 26 –heading towards 4-5 homeworks total Reminder: midterm on Thursday, March 4

2. News and Notes, 2/10 Two new articles added to required reading Homework 1 graded; returned today at end

3. News and Notes, 2/5 Homework 1 due now Midterm date: Thursday, March 4 Read Watts, chapters 2-5

4. Social Network Theory Networked Life CSE 112 Spring 2004 Prof. Michael Kearns

5. “Natural” Networks and Universality Consider the many kinds of networks we have examined: –social, technological, business, economic, content,… These networks tend to share certain informal properties: –large scale; continual growth –distributed, organic growth: vertices “decide” who to link to –interaction restricted to links –mixture of local and long-distance connections –abstract notions of distance: geographical, content, social,… Do natural networks share more quantitative universals? What would these “universals” be? How can we make them precise and measure them? How can we explain their universality? This is the domain of social network theory Sometimes also referred to as link analysis

6. Some Interesting Quantities Connected components: –how many, and how large? Network diameter: –maximum (worst-case) or average? –exclude infinite distances? (disconnected components) –the small-world phenomenon Clustering: –to what extent to links tend to cluster “locally”? –what is the balance between local and long-distance connections? –what roles do the two types of links play? Degree distribution: –what is the typical degree in the network? –what is the overall distribution?

7. A “Canonical” Natural Network has… Few connected components: –often only 1 or a small number independent of network size Small diameter: –often a constant independent of network size (like 6) –or perhaps growing only logarithmically with network size –typically exclude infinite distances A high degree of clustering: –considerably more so than for a random network –in tension with small diameter A heavy-tailed degree distribution: –a small but reliable number of high-degree vertices –quantifies Gladwell’s connectors –often of power law form

8. Some Models of Network Generation Random graphs (Erdos-Renyi models): –gives few components and small diameter –does not give high clustering and heavy-tailed degree distributions –is the mathematically most well-studied and understood model Watts-Strogatz and related models: –give few components, small diameter and high clustering –does not give heavy-tailed degree distributions Preferential attachment: –gives few components, small diameter and heavy-tailed distribution –does not give high clustering Hierarchical networks: –few components, small diameter, high clustering, heavy-tailed Affiliation networks: –models group-actor formation Nothing “magic” about any of the measures or models

9. Approximate Roadmap Examine a series of models of network generation –macroscopic properties they do and do not entail –pros and cons of each model Examine some “real life” case studies Study some dynamics issues (e.g. navigation) Move into in-depth study of the web as network

10. Probabilistic Models of Networks All of the network generation models we will study are probabilistic or statistical in nature They can generate networks of any size They often have various parameters that can be set: –size of network generated –average degree of a vertex –fraction of long-distance connections The models generate a distribution over networks Statements are always statistical in nature: –with high probability, diameter is small –on average, degree distribution has heavy tail Thus, we’re going to need some basic statistics and probability theory

11. Statistics and Probability Theory: The Absolute, Bare Minimum Essentials

12. Probability and Random Variables A random variable X is simply a variable that probabilistically assumes values in some set –set of possible values sometimes called the sample space S of X –sample space may be small and simple or large and complex S = {Heads, Tails}, X is outcome of a coin flip S = {0,1,…,U.S. population size}, X is number voting democratic S = all networks of size N, X is generated by preferential attachment Behavior of X determined by its distribution (or density) –for each value x in S, specify Pr[X = x] –these probabilities sum to exactly 1 (mutually exclusive outcomes) –complex sample spaces (such as large networks): distribution often defined implicitly by simpler components might specify the probability that each edge appears independently this induces a probability distribution over networks may be difficult to compute induced distribution

13. Some Basic Notions and Laws Independence: –let X and Y be random variables –independence: for any x and y, Pr[X = x & Y = y] = Pr[X=x]Pr[Y=y] –intuition: value of X does not influence value of Y, vice-versa –dependence: e.g. X, Y coin flips, but Y is always opposite of X Expected (mean) value of X: –only makes sense for numeric random variables –“average” value of X according to its distribution –formally, E[X] =  (Pr[X = x] X), sum is over all x in S –often denoted by  –always true: E[X + Y] = E[X] + E[Y] –true only for independent random variables: E[XY] = E[X]E[Y] Variance of X: –Var(X) = E[(X –  )^2]; often denoted by  ^2 –standard deviation is sqrt(Var(X)) =  Union bound: –for any X, Y, Pr[X=x & Y=y] <= Pr[X=x] + Pr[Y=y]

14. Convergence to Expectations Let X1, X2,…, Xn be: –independent random variables –with the same distribution Pr[X=x] –expectation  = E[X] and variance  ^2 –independent and identically distributed (i.i.d.) –essentially n repeated “trials” of the same experiment –natural to examine r.v. Z = (1/n)  Xi, where sum is over i=1,…,n –example: number of heads in a sequence of coin flips –example: degree of a vertex in the random graph model –E[Z] = E[X]; what can we say about the distribution of Z? Central Limit Theorem: –as n becomes large, Z becomes normally distributed with expectation  and variance  ^2/n –here’s a demodemo

15. The Normal Distribution The normal or Gaussian density: –applies to continuous, real-valued random variables –characterized by mean (average)  and standard deviation  –density at x is defined as (1/(  sqrt(2  ))) exp(-(x-  )^2/2  ^2) special case  = 0,  = 1: a exp(-x^2/b) for some constants a,b > 0 –peaks at x = , then dies off exponentially rapidly –the classic “bell-shaped curve” exam scores, human body temperature, –here are some examplesexamples –remarks: can control mean and standard deviation independently can make as “broad” as we like, but always have finite variance

16. The Binomial Distribution The binomial distribution: –coin with Pr[heads] = p, flip n times –probability of getting exactly k heads: choose(n,k) p^k (1-p)^(n-k) –for large n and p fixed: approximated well by a normal with  = pn,  = sqrt(np(1-p))   0 as n grows leads to strong large deviation boundslarge deviation bounds

17. The Poisson Distribution The Poisson distribution: –like binomial, applies to variables taken on integer values > 0 –often used to model counts of events number of phone calls placed in a given time period number of times a neuron fires in a given time period –single free parameter –probability of exactly x events: exp(- ) ^x/x! mean and variance are both here are some examplesexamples –binomial distribution with n large, p = /n ( fixed) converges to Poisson with mean

18. Heavy-tailed Distributions Pareto or power law distributions: –for variables assuming integer values > 0 –probability of value x ~ 1/x^  –typically 0 <  < 2; smaller  gives heavier tail –here are some examplesexamples –sometimes also referred to as being scale-free For binomial, normal, and Poisson distributions the tail probabilities approach 0 exponentially fast Inverse polynomial decay vs. inverse exponential decay What kind of phenomena does this distribution model? What kind of process would generate it?

19. Distributions vs. Data All these distributions are idealized models In practice, we do not see distributions, but data Thus, there will be some largest value we observe Also, can be difficult to “eyeball” data and choose model So how do we distinguish between Poisson, power law, etc? Typical procedure: –might restrict our attention to a range of values of interest –accumulate counts of observed data into equal-sized bins –look at counts on a log-log plot –note that power law: –log(Pr[value x]) = log(1/x^  ) = -  log(x) –linear, slope – linear, slope –  Normal: –log(Pr[value x]) = log(a exp(-x^2/b)) = log(a) – x^2/b –non-linear, concave near meannon-linear, concave near mean Poisson: –log(Pr[value x]) = log(exp(- ) ^x/x!) –also non-linear

20. Zipf’s Law Look at the frequency of English words: –“the” is the most common, followed by “of”, “to”, etc. –claim: frequency of the n-th most common ~ 1/n (power law,  = 1) General theme: –rank events by their frequency of occurrence –resulting distribution often is a power law! Other examples: –North America city sizes –personal income –file sizes –genus sizes (number of species) –let’s look at log-log plots of theselog-log plots People seem to dither over exact form of these distributions (e.g. value of  ), but not heavy tails

21. Models of Network Generation and Their Properties

22. The Erdos-Renyi (ER) Model (Random Graphs) A model in which all edges –are equally probable –appear independently NW size N > 1 and probability p: distribution G(N,p) –each edge (u,v) chosen to appear with probability p –N(N-1)/2 trials of a biased coin flip The usual regime of interest is when p ~ 1/N, N is large –e.g. p = 1/2N, p = 1/N, p = 2/N, p=10/N, p = log(N)/N, etc. –in expectation, each vertex will have a “small” number of neighbors –will then examine what happens when N  infinity –can thus study properties of large networks with bounded degree Degree distribution of a typical G drawn from G(N,p): –draw G according to G(N,p); look at a random vertex u in G –what is Pr[deg(u) = k] for any fixed k? –Poisson distribution with mean = p(N-1) ~ pN –Sharply concentrated; not heavy-tailed Especially easy to generate NWs from G(N,p)

23. A Closely Related Model For any fixed m <= N(N-1)/2, define distribution G(N,m): –choose uniformly at random from all graphs with exactly m edges –G(N,m) is “like” G(N,p) with p = m/(N(N-1)/2) ~ 2m/N^2 –this intuition can be made precise, and is correct –if m = cN then p = 2c/(N-1) ~ 2c/N –mathematically trickier than G(N,p)

24. Another Closely Related Model Graph process model: –start with N vertices and no edges –at each time step, add a new edge –choose new edge randomly from among all missing edges Allows study of the evolution or emergence of properties: –as the number of edges m grows in relation to N –equivalently, as p is increased For all of these models: –high probability  “almost all” large graphs of a given density

25. The Evolution of a Random Network We have a large number n of vertices We start randomly adding edges one at a time At what time t will the network: –have at least one “large” connected component? –have a single connected component? –have “small” diameter? –have a “large” clique? –have a “large” chromatic number? How gradually or suddenly do these properties appear?

26. Recap Model G(N,p): –select each of the possible edges independently with prob. p –expected total number of edges is pN(N-1)/2 –expected degree of a vertex is p(N-1) –degree will obey a Poisson distribution (not heavy-tailed) Model G(N,m): –select exactly m of the N(N-1)/2 edges to appear –all sets of m edges equally likely Graph process model: –starting with no edges, just keep adding one edge at a time –always choose next edge randomly from among all missing edges Threshold or tipping for (say) connectivity: –fewer than m(N) edges  graph almost certainly not connected –more than m(N) edges  graph almost certainly is connected –made formal by examining limit as N  infinity

27. Combining and Formalizing Familiar Ideas Explaining universal behavior through statistical models –our models will always generate many networks –almost all of them will share certain properties (universals) Explaining tipping through incremental growth –we gradually add edges, or gradually increase edge probability p –many properties will emerge very suddenly during this process size of police force crime rate number of edges prob. NW connected

28. Monotone Network Properties Often interested in monotone graph properties: –let G have the property –add edges to G to obtain G’ –then G’ must have the property also Examples: –G is connected –G has diameter <= d (not exactly d) –G has a clique of size >= k (not exactly k) –G has chromatic number >= c (not exactly c) –G has a matching of size >= m –d, k, c, m may depend on NW size N (How?) Difficult to study emergence of non-monotone properties as the number of edges is increased –what would it mean?

29. Formalizing Tipping: Thresholds for Monotone Properties Consider Erdos-Renyi G(N,m) model –select m edges at random to include in G Let P be some monotone property of graphs –P(G) = 1  G has the property –P(G) = 0  G does not have the property Let m(N) be some function of NW size N –formalize idea that property P appears “suddenly” at m(N) edges Say that m(N) is a threshold function for P if: –let m’(N) be any function of N –look at ratio r(N) = m’(N)/m(N) as N  infinity –if r(N)  0: probability that P(G) = 1 in G(N,m’(N)):  0 –if r(N)  infinity: probability that P(G) = 1 in G(N,m’(N)):  1 A purely structural definition of tipping –tipping results from incremental increase in connectivity

30. So Which Properties Tip? Just about all of them! The following properties all have threshold functions: –having a “giant component” –being connected –having a perfect matching (N even) –having “small” diameter Demo: look at the following progression –giant component  connectivity  small diameter –in graph process model (add one new edge at a time) –[example 1] [example 2] [example 3] [example 4] [example 5][example 1][example 2][example 3][example 4][example 5] With remarkable consistency (N = 50): –giant component ~ 40 edges, connected ~ 100, small diameter ~ 180

31. Ever More Precise… Connected component of size > N/2: –threshold function is m(N) = N/2 (or p ~ 1/N) –note: full connectivity impossible Fully connected: –threshold function is m(N) = (N/2)log(N) (or p ~ log(N)/N) –NW remains extremely sparse: only ~ log(N) edges per vertex Small diameter: –threshold is m(N) ~ N^(3/2) for diameter 2 (or p ~ 2/sqrt(N)) –fraction of possible edges still ~ 2/sqrt(N)  0 –generate very small worlds

32. Other Tipping Points? Perfect matchings –consider only even N –threshold function is m(N) = (N/2)log(N) (or p ~ log(N)/N) –same as for connectivity! Cliques –k-clique threshold is m(N) = (1/2)N^(2 – 2/(k-1)) (p ~ 1/N^(2/k-1)) –edges appear immediately; triangles at N/2; etc. Coloring –k colors required just as k-cliques appear

33. Erdos-Renyi Summary A model in which all connections are equally likely –each of the N(N-1)/2 edges chosen randomly & independently As we add edges, a precise sequence of events unfolds: –graph acquires a giant component –graph becomes connected –graph acquires small diameter –etc. Many properties appear very suddenly (tipping, thresholds) All statements are mathematically precise But is this how natural networks form? If not, which aspects are unrealistic? –may all edges are not equally likely!

34. The Clustering Coefficient of a Network Let nbr(u) denote the set of neighbors of u in a graph –all vertices v such that the edge (u,v) is in the graph The clustering coefficient of u: –let k = |nbr(u)| (i.e., number of neighbors of u) –choose(k,2): max possible # of edges between vertices in nbr(u) –c(u) = (actual # of edges between vertices in nbr(u))/choose(k,2) –0 <= c(u) <= 1; measure of cliquishness of u’s neighborhood Clustering coefficient of a graph: –average of c(u) over all vertices u k = 4 choose(k,2) = 6 c(u) = 4/6 = 0.666…

35. Erdos-Renyi: Clustering Coefficient Generate a network G according to G(N,p) Examine a “typical” vertex u in G –choose u at random among all vertices in G –what do we expect c(u) to be? Answer: exactly p! In G(N,m), expect c(u) to be 2m/N(N-1) Both cases: c(u) entirely determined by overall density Baseline for comparison with “more clustered” models –Erdos-Renyi has no bias towards clustered or local edges

36. Caveman and Solaria Erdos-Renyi: –sharing a common neighbor makes two vertices no more likely to be directly connected than two very “distant” vertices –every edge appears entirely independently of existing structure But in many settings, the opposite is true: –you tend to meet new friends through your old friends –two web pages pointing to a third might share a topic –two companies selling goods to a third are in related industries Watts’ Caveman world: –overall density of edges is low –but two vertices with a common neighbor are likely connected Watts’ Solaria world –overall density of edges low; no special bias towards local edges –“like” Erdos-Renyi

37. Making it (Somewhat) Precise: the  -model The  -model has the following parameters or “knobs”: – N: size of the network to be generated – k: the average degree of a vertex in the network to be generated – p: the default probability two vertices are connected –  : adjustable parameter dictating bias towards local connections For any vertices u and v: –define m(u,v) to be the number of common neighbors (so far) Key quantity: the propensity R(u,v) of u to connect to v –if m(u,v) >= k, R(u,v) = 1 (share too many friends not to connect) –if m(u,v) = 0, R(u,v) = p (no mutual friends  no bias to connect) –else, R(u,v) = p + (m(u,v)/k)^  (1-p) –here are some plots for different  (see Watts page 77)plots for different  Generate NW incrementally –using R(u,v) as the edge probability; details omitted Note:  = infinity is “like” Erdos-Renyi (but not exactly)

38. Small Worlds and Occam’s Razor For small , should generate large clustering coefficients –we “programmed” the model to do so –Watts claims that proving precise statements is hard… But we do not want a new model for every little property –Erdos-Renyi  small diameter –  -model  high clustering coefficient –etc. In the interests of Occam’s Razor, we would like to find –a single, simple model of network generation… –… that simultaneously captures many properties Watt’s small world: small diameter and high clustering –here is a figure showing that this can be captured in the  -modelfigure

39. Meanwhile, Back in the Real World… Watts examines three real networks as case studies: –the Kevin Bacon graph –the Western states power grid –the C. elegans nervous system For each of these networks, he: –computes its size, diameter, and clustering coefficient –compares diameter and clustering to best Erdos-Renyi approx. –shows that the best  -model approximation is better –important to be “fair” to each model by finding best fit Overall moral: –if we care only about diameter and clustering,  is better than p

40. Case 1: Kevin Bacon Graph Vertices: actors and actresses Edge between u and v if they appeared in a film together Here is the datadata

41. Case 2: Western States Power Grid Vertices: power stations in Western U.S. Edges: high-voltage power transmission lines Here is the network and datanetwork and data

42. Case 3: C. Elegans Nervous System Vertices: neurons in the C. elegans worm Edges: axons/synapses between neurons Here is the network and datanetwork and data

43. Two More Examples M. Newman on scientific collaboration networks –coauthorship networks in several distinct communities –differences in degrees (papers per author) –empirical verification of giant components small diameter (mean distance) high clustering coefficient Alberich et al. on the Marvel Universe –purely fictional social network –two characters linked if they appeared together in an issue –“empirical” verification of heavy-tailed distribution of degrees (issues and characters) giant component rather small clustering coefficient

44. One More (Structural) Property… A properly tuned  -model can simultaneously explain –small diameter –high clustering coefficient But what about heavy-tailed degree distributions? –  -model and simple variants will not explain this –intuitively, no “bias” towards large degree evolves –all vertices are created equal Can concoct many bad generative models to explain –generate NW according to Erdos-Renyi, reject if tails not heavy –describe fixed NWs with heavy tails all connected to v1; N/2 connected to v2; etc. not clear we can get a precise power law not modeling variation –why would the world evolve this way? As always, we want a “natural” model

45. Preferential Attachment Start with (say) two vertices connected by an edge For i = 3 to N: –for each 1 <= j < i, let d(j) be degree of vertex j (so far) –let Z =  d(j) (sum of all degrees so far) –add new vertex i with k edges back to {1,…,i-1}: i is connected back to j with probability d(j)/Z Vertices j with high degree are likely to get more links! “Rich get richer” Natural model for many processes: –hyperlinks on the web –new business and social contacts –transportation networks Generates a power law distribution of degrees –exponent depends on value of k

46. Two Out of Three Isn’t Bad… Preferential attachment explains –heavy-tailed degree distributions –small diameter (~log(N), via “hubs”) Will not generate high clustering coefficient –no bias towards local connectivity, but towards hubs Can we simultaneously capture all three properties? –probably, but we’ll stop here –soon there will be a fourth property anyway…

47. Search and Navigation

48. Finding Short Paths Milgram’s experiment, Columbia Small Worlds,  -model… –all emphasize existence of short paths between pairs How do individuals find short paths –in an incremental, next-step fashion –using purely local information about the NW and location of target This is not a structural question, but an algorithmic one –statics vs. dynamics Navigability may impose additional restrictions on model! Briefly investigate two alternatives: –variation on the  -model –a “social identity” model

49. Kleinberg’s Model Similar in spirit to the  -model Start with an n by n grid of vertices (so N = n^2) –add local connections: all vertices within grid distance p (e.g. 2) –add distant connections: q additional connections probability of connection at distance d: ~ 1/d^r –so full model given by choice of p, q and r –small r: heavy bias towards “more local” long-distance connections –large r: approach uniformly random Kleinberg’s question: –what of r permits effective search? Assume parties know only: –grid address of target –addresses of their own direct links Algorithm: pass message to neighbor closest to target

50. Kleinberg’s Result Intuition: –if r is too small (strong local bias), then “long-distance” connections never help much; short paths may not even exist –if r is too large (no local bias), we may quickly get close to the target; but then we’ll have to use local links to finish think of a transport system with only long-haul jets or donkey carts –effective search requires a delicate mixture of link distances The result (informally): –r = 2 is the only value that permits rapid navigation (~log(N) steps) –any other value of r will result in time ~ N^c for 0 < c <= 1 –a critical value phenomenon Note: locality of information crucial to this argument –centralized algorithm may compute short paths at large r –can recognize when “backwards” steps are beneficial

51. Navigation via Identity Watts et al.: –we don’t navigate social networks by purely “geographic” information –we don’t use any single criterion; recall Dodds et al. on Columbia SW –different criteria used a different points in the chain Represent individuals by a vector of attributes –profession, religion, hobbies, education, background, etc… –attribute values have distances between them (tree-structured) –distance between individuals: minimum distance in any attribute –only need one thing in common to be close! Algorithm: –given attribute vector of target –forward message to neighbor closest to target Permits fast navigation under broad conditions –not as sensitive as Kleinberg’s model all jobs scientists athletes chemistry CS baseball tennis

52. Next Up: The Web as Network