1. COMS 6998-06 Network Theory Week 6: February 28, 2008 Dragomir R. Radev Thursdays, 6-8 PM 233 Mudd Spring 2008

2. (11) Community identification (12) Spectral clustering

3. Community structure

4. Assortative mixing and community structure Community structure usually implies assortative mixing Assortative mixing doesn’t necessarily imply community structure (e.g., assortative mixing by age)

5. Building the graph Two nodes are linked if the two end points share a lot of attributes Examples: Hamming distance, Pearson correlation, cosine

6. Traditional methods Spectral bisection Kernighan-Lin (1970) O(n 3 ) Fiduccia-Mattheyses heuristic: O(m)

7. Spectral algorithms The spectrum of a matrix is the list of all eigenvalues of a matrix sorted from smallest to largest. The eigenvectors in the spectrum are sorted by the absolute value of their corresponding eigenvalues. In spectral methods, eigenvectors are based on the Laplacian of the original matrix.

8. Laplacian matrix The Laplacian L of a matrix is a symmetric matrix. L = D – G, where D is the degree matrix corresponding to G. Example: A B C G F E D ABCDEFG A 3000 B 30 00 C 003 0 D 0 3 00 E 00 200 F 0 0020 G 00002

9. Fiedler vector The Fiedler vector is the eigenvector of L(G) with the second smallest eigenvalue. That quantity is called the algebraic connectivity of the matrix. A B C G F E D A -0.3682C1 B -0.2808C1 C 0.3682C2 D 0.2808C2 E 0.5344C2 F 0.0000? G -0.5344C1

10. Spectral bisection algorithm Compute 2 Compute the corresponding v2 For each node n of G –If v2(n) < 0 Assign n to cluster C1 –Else if v2(n) > 0 Assign n to cluster C2 –Else if v2(n) = 0 Assign n to cluster C1 or C2 at random A. Pothen, H. Simon, K.-P. Liou, “Partitioning sparse matrices with eigenvectors of graphs”, SIAM J. Mat. Anal. Appl. 11:430-452 (1990) M. Fiedler, “Algebraic Connectivity of Graphs”, Czech. Math. J., 23:298-305 (1973) M. Fiedler, Czech. Math. J., 25:619-637 (1975)

11. [Figure by Demmel]

12. Kernighan-Lin Q=benefit function = #edges within groups - #edges across groups Start with a bipartition of specified sizes. Let’s call the two parts A and B. Compute  Q for each pair of vertices such that one of them is in A and the other in B. Pick the pair with the largest  Q and swap its elements. Repeat until there are no more swaps possible (no vertex can change partitions more than once). Retrace the algorithm and stop when Q is highest. Note that Q did not have to change monotonically.

13. Kernighan-Lin Disadvantage: need split size –Can be fixed in O(n 3 ) –However, this is likely to produce a lopsided split (of sizes 0 and n).

14. Hierarchical clustering Based on similarity and dendrograms.

15. Centrality-based methods Girvan-Newman method based on edge betweenness Divisive method Computed in O(mn). Edge betweenness is the number of geodesic paths that run along a given edge.

16. Modularity Measures the quality of a split g groups, gxg matrix E such that e ij is the fraction of edges in the original network that connect vertices in group i to vertices in group j. where ||x|| is the sum of all values of x Q is the fraction of all edges that lie within communities – the expected value of the same quantity in a network in which the vertices have the same degrees but edges are placed at random.

17. Alternatives The method is slow – there are m edges to be removed so total runtime is O(m 2 n). Improvement 1 – Monte Carlo speedup (Tyler et al.) – compute EB only using a sample of vertices. An extra advantage – can give probabilistic node assignments (how?) Improvement 2 – local information only (Radicchi) – triangles are unlikely across communities. This method works better for social networks.

18. Wu and Huberman Use voltage gaps. Challenge – pick the poles; randomized method works: pick random pairs of non-neighbors This method can be used to find nearby nodes too (how?) F Wu, BA Huberman: Finding communities in linear time: a physics approach - The European Physical Journal B-Condensed Matter, 2004Finding communities in linear time: a physics approach

19. AB DE HI FG C J Example from Caldarelli 2007] Girvan/Newman example

20. HIDABEFLCG Edge betweenness dendrogram Example from Caldarelli 2007]

21. [Detecting community structure in networks, M. E. J. Newman, Eur. Phys. J. B 38, 321–330 (2004). ]Detecting community structure in networks

22. Existing software http://www.sandia.gov/~bahendr/chaco.ht ml (CHACO)http://www.sandia.gov/~bahendr/chaco.ht ml http://glaros.dtc.umn.edu/gkhome/views/m etis/index.html (METIS)http://glaros.dtc.umn.edu/gkhome/views/m etis/index.html http://www.cs.utexas.edu/users/dml/Softw are/graclus.html (GRACLUS)http://www.cs.utexas.edu/users/dml/Softw are/graclus.html

23. Minimum Spanning Tree A spanning tree for a connected, undirected graph G=(V,E) –a subgraph of G that is –an undirected tree and contains –all the vertices of G. In a weighted graph G=(V,E,W), the weight of a subgraph –the sum of the weights of the edges in the subgraph A minimum spanning tree for a weighted graph –a spanning tree with the minimum weight. [MST and mincut slides by Rada Mihalcea]

24. Prim/Jarnik’s Minimum Spanning Tree Algorithm Select an arbitrary starting vertex (the root) Branch out from the tree constructed so far by –choosing an edge at each iteration –attach the edge to the tree the edge that has minimum weight among all edges that can be attached –add to the tree the vertex associated with the edge Vertices are divided into three disjoint categories: –Tree vertices: in the tree constructed so far, –Fringe vertices: not in the tree, but adjacent to some vertex in the tree, –Unseen vertices: all others

25. Prim’s Algorithm Algorithm PrimMinSpanningTree(G) Initialize all nodes as unseen Select an arbitrary vertex s to start the tree Reclassify it as tree Reclassify all vertices adjacent to s as fringe While there are fringe vertices Select an edge of minimum weight between a tree vertex t and a fringe vertex v Reclassify v as tree; add edge tv to the tree Reclassify all unseen vertices adjacent to v as fringe

26. Prim’s Algorithm v1v2 v3v4v5 v6v7 2 14 1 4 10 6 72 5 8 3 v1v2 v3v4v5 v6v7 2 14 1 4 10 6 72 5 8 3 v1v2 v3v4v5 v6v7 2 14 1 4 10 6 72 5 8 3 v1v2 v3v4v5 v6v7 2 14 1 4 10 6 72 5 8 3

27. Prim’s Algorithm v1v2 v3v4v5 v6v7 2 14 1 4 10 6 72 5 8 3 v1v2 v3v4v5 v6v7 2 14 1 4 10 6 72 5 8 3 v1v2 v3v4v5 v6v7 2 14 1 4 10 6 72 5 8 3

28. Properties of Minimum Spanning Trees Definition: Minimum spanning tree property –Let G be a connected, weighted graph G=(V,E,W), and let T be any spanning tree of G. Suppose that for every edge vw of G that is not in T: if uv is added to T, then it creates a cycle uv is a maximum-weight edge on that cycle –The tree T is said to have the minimum spanning tree property A graph can admit several minimum spanning trees Lemma: In a connected, weighted graph G = (V, E, W), if T1 and T2 are two spanning trees that have the MST property, then they have the same total weight The number of edges in a minimum spanning tree is |V|- 1

29. Flow networks What if weights in a graph are maximum capacities of some flow of material? Examples: –Pipe network to transport fluid (e.g., water, oil) Edges – pipes, vertices – junctions of pipes –Data communication network Edges – network connections of different capacity, vertices – routers (do not produce or consume data just move it) Concepts (informally): –Source vertex s (where material is produced) –Sink vertex t (where material is consumed) –For all other vertices – what goes in must go out –Goal: maximum rate of material flow from source to sink

30. Formalization Flow network – G=(V,E) –Directed, each edge has capacity c(u,v)  0 –Two special vertices: source s, and sink t –For any other vertex v, there is a path s  …  v  …  t Flow – f : V  V  R –Capacity constraint:  u,v  V: f(u,v)  c(u,v) –Skew symmetry:  u,v  V: f(u,v) = –f(v,u) –Flow conservation:  u  V – {s, t}:

31. Cancellation of flows Do we want to have positive flows going in both directions between two vertices? No! such flows cancel (maybe partially) each other. 13 11 5 4 15 10 14 19 3 s t 9 a b c d 2 5 3 3 6 6 13 11 5 4 15 10 14 19 3 s t 9 a b c d 3 3 3 6 6

32. Maximum flow Want to maximize the total value of the flow f: Find a flow of maximum value. 13 11 5 4 15 10 14 19 3 s t 9 a b c d 5 13 3 8 6 10 8 2

33. Augmenting path Idea for the algorithm: –If we have some flow, and can find a path p from s to t (augmenting path), such that there is a > 0, and for each edge (u,v) in p we can add a units of flow: f(u,v) + a  c(u,v) –then mark down that path, to get a better flow Augmenting path in this graph? 13 11 5 4 15 10 14 19 3 s t 9 a b c d 5 13 3 8 6 10 8 2

34. Ford-Fulkerson method Ford-Fulkerson(G,s,t) 1 initialize flow f to 0 everywhere 2 while there is an augmenting path p do 3 augment flow f along p 4 return f How do we find augmenting path? How much additional flow can we send through that path?

35. Residual network How do we find augmenting path? It is any path in residual network: –Residual capacities: c f (u,v) = c(u,v) – f(u,v) –Residual network: G f =(V,E f ), where E f = {(u,v)  V  V : c f (u,v) > 0} Compute residual network: 13 11 5 4 15 10 14 19 3 s t 9 a b c d 5 13 3 8 6 10 8 2

36. Residual capacity of a path How much additional flow can we send through an augmenting path? Residual capacity of a path p in G f : c f (p) = min{c f (u,v): (u,v)  p} Doing augmentation: –  (u,v) in p, add c f (p) to f(u,v) (and subtract from f(v,u)) –Resulting flow is a valid flow with a larger value. 13 11 5 4 15 10 14 19 3 s t 9 a b c d 5 13 3 8 6 10 8 2 What is the residual capacity of the path (s,a,b,t)?

37. Ford-Fulkerson method, with details Ford-Fulkerson(G,s,t) 1 for each edge (u,v)  G.E do 2 f(u,v) = f(v,u) = 0 3 while  path p from s to t in residual network G f do 4 c f = min{c f (u,v): (u,v)  p} 5 for each edge (u,v) in p do 6 f(u,v) = f(u,v) + c f 7 f(v,u) = -f(u,v) 8 return f The algorithms based on this method differ in how they choose p in step 3.

38. Ford-Fulkerson method, example 13 11 5 4 15 10 14 19 3 s t a b c d 3 3 3 13 11 5 4 15 10 14 19 3 s t a b c d 13 3 3 3 11 5 4 15 10 14 19 3 s t a b c d 18 3 8 8 13 5

39. An s-t cut is a partition (S, T) such that s  S, t  T. –The capacity of an s-t cut (S, T) is: Min s-t cut: find an s-t cut of minimum capacity. Cuts s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 Capacity = 30

40. Cuts s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 Capacity = 62 s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 Capacity = 28

41. Flows and Cuts 10 6 6 0 4 8 8 0 4 0 0 0 s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 Value = 24 L1. Let f be a flow, and let (S, T) be a cut. Then, the net flow sent across the cut is equal to the amount reaching t.

42. Max Flow and Min Cut Corollary. Let f be a flow, and let (S, T) be a cut. If |f| = cap(S, T), then f is a max flow and (S, T) is a min cut. Max-flow / min-cut theorem (Ford-Fulkerson): In any network, the value of the max flow is equal to the value of the min cut. s 2 3 4 5 6 7 t 15 5 30 15 10 8 15 9 6 10 15 4 4 10 9 9 15 4 10 4 8 9 1 0 0 Flow value = 28 0 0 Cut capacity = 28