1. CMU SCS Talk 3: Graph Mining Tools – Tensors, communities, parallelism Christos Faloutsos CMU

2. CMU SCS (C) 2011, C. Faloutsos 2 Overall Outline Introduction – Motivation Talk#1: Patterns in graphs; generators Talk#2: Tools (Ranking, proximity) Talk#3: Tools (Tensors, scalability) Conclusions KAIST-2011

3. CMU SCS Outline Task 4: time-evolving graphs – tensors Task 5: community detection Task 6: virus propagation Task 7: scalability, parallelism and hadoop Conclusions KAIST-2011(C) 2011, C. Faloutsos 3

4. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 4 Thanks to Tamara Kolda (Sandia) for the foils on tensor definitions, and on TOPHITS

5. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 5 Detailed outline Motivation Definitions: PARAFAC and Tucker Case study: web mining

6. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 6 Examples of Matrices: Authors and terms dataminingclassif.tree... John Peter Mary Nick...

7. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 7 Motivation: Why tensors? Q: what is a tensor?

8. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 8 Motivation: Why tensors? A: N-D generalization of matrix: dataminingclassif.tree... John Peter Mary Nick... KDD’09

9. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 9 Motivation: Why tensors? A: N-D generalization of matrix: dataminingclassif.tree... John Peter Mary Nick... KDD’08 KDD’07 KDD’09

10. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 10 Tensors are useful for 3 or more modes Terminology: ‘mode’ (or ‘aspect’): dataminingclassif.tree... Mode (== aspect) #1 Mode#2 Mode#3

11. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 11 Notice 3 rd mode does not need to be time we can have more than 3 modes... IP destination Dest. port IP source 80 125

12. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 12 Notice 3 rd mode does not need to be time we can have more than 3 modes –Eg, fFMRI: x,y,z, time, person-id, task-id http://denlab.temple.edu/bidms/cgi-bin/browse.cgi From DENLAB, Temple U. (Prof. V. Megalooikonomou +)

13. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 13 Motivating Applications Why tensors are useful? –web mining (TOPHITS) –environmental sensors –Intrusion detection (src, dst, time, dest-port) –Social networks (src, dst, time, type-of-contact) –face recognition –etc …

14. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 14 Detailed outline Motivation Definitions: PARAFAC and Tucker Case study: web mining

15. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 15 Tensor basics Multi-mode extensions of SVD – recall that:

16. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 16 Reminder: SVD –Best rank-k approximation in L2 A m n  m n U VTVT 

17. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 17 Reminder: SVD –Best rank-k approximation in L2 A m n  + 1u1v11u1v1 2u2v22u2v2

18. KAIST-2011(C) 2011, C. Faloutsos 18 Goal: extension to >=3 modes ~ I x R K x R A B J x R C R x R x R I x J x K +…+=

19. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 19 Main points: 2 major types of tensor decompositions: PARAFAC and Tucker both can be solved with ``alternating least squares’’ (ALS)

20. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 20 = U I x R V J x R W K x R R x R x R Specially Structured Tensors Tucker TensorKruskal Tensor I x J x K = U I x R V J x S W K x T R x S x T I x J x K Our Notation +…+= u1u1 uRuR v1v1 w1w1 vRvR wRwR “core”

21. KAIST-2011(C) 2011, C. Faloutsos 21 Tucker Decomposition - intuition I x J x K ~ A I x R B J x S C K x T R x S x T author x keyword x conference A: author x author-group B: keyword x keyword-group C: conf. x conf-group G : how groups relate to each other

22. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 22 Intuition behind core tensor 2-d case: co-clustering [Dhillon et al. Information-Theoretic Co- clustering, KDD’03]

23. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 23 m m n nl k k l eg, terms x documents

24. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 24 term x term-group doc x doc group term group x doc. group med. terms cs terms common terms med. doc cs doc

25. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 25 Tensor tools - summary Two main tools –PARAFAC –Tucker Both find row-, column-, tube-groups –but in PARAFAC the three groups are identical ( To solve: Alternating Least Squares )

26. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 26 Detailed outline Motivation Definitions: PARAFAC and Tucker Case study: web mining

27. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 27 Web graph mining How to order the importance of web pages? –Kleinberg’s algorithm HITS –PageRank –Tensor extension on HITS (TOPHITS)

28. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 28 Kleinberg’s Hubs and Authorities (the HITS method) Sparse adjacency matrix and its SVD: authority scores for 1 st topic hub scores for 1 st topic hub scores for 2 nd topic authority scores for 2 nd topic from to Kleinberg, JACM, 1999

29. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 29 authority scores for 1 st topic hub scores for 1 st topic hub scores for 2 nd topic authority scores for 2 nd topic from to HITS Authorities on Sample Data We started our crawl from http://www-neos.mcs.anl.gov/neos, and crawled 4700 pages, resulting in 560 cross-linked hosts.

30. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 30 Three-Dimensional View of the Web Observe that this tensor is very sparse! Kolda, Bader, Kenny, ICDM05

31. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 31 Three-Dimensional View of the Web Observe that this tensor is very sparse! Kolda, Bader, Kenny, ICDM05

32. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 32 Three-Dimensional View of the Web Observe that this tensor is very sparse! Kolda, Bader, Kenny, ICDM05

33. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 33 Topical HITS (TOPHITS) Main Idea: Extend the idea behind the HITS model to incorporate term (i.e., topical) information. authority scores for 1 st topic hub scores for 1 st topic hub scores for 2 nd topic authority scores for 2 nd topic from to term term scores for 1 st topic term scores for 2 nd topic

34. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 34 Topical HITS (TOPHITS) Main Idea: Extend the idea behind the HITS model to incorporate term (i.e., topical) information. authority scores for 1 st topic hub scores for 1 st topic hub scores for 2 nd topic authority scores for 2 nd topic from to term term scores for 1 st topic term scores for 2 nd topic

35. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 35 TOPHITS Terms & Authorities on Sample Data TOPHITS uses 3D analysis to find the dominant groupings of web pages and terms. authority scores for 1 st topic hub scores for 1 st topic hub scores for 2 nd topic authority scores for 2 nd topic from to term term scores for 1 st topic term scores for 2 nd topic Tensor PARAFAC w k = # unique links using term k

36. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 36 Conclusions Real data are often in high dimensions with multiple aspects (modes) Tensors provide elegant theory and algorithms –PARAFAC and Tucker: discover groups

37. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 37 References T. G. Kolda, B. W. Bader and J. P. Kenny. Higher-Order Web Link Analysis Using Multilinear Algebra. In: ICDM 2005, Pages 242- 249, November 2005. Jimeng Sun, Spiros Papadimitriou, Philip Yu. Window-based Tensor Analysis on High- dimensional and Multi-aspect Streams, Proc. of the Int. Conf. on Data Mining (ICDM), Hong Kong, China, Dec 2006

38. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 38 Resources See tutorial on tensors, KDD’07 (w/ Tamara Kolda and Jimeng Sun): www.cs.cmu.edu/~christos/TALKS/KDD-07-tutorial

39. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 39 Tensor tools - resources Toolbox: from Tamara Kolda: csmr.ca.sandia.gov/~tgkolda/TensorToolbox 2-39 Copyright: Faloutsos, Tong (2009) 2-39 ICDE’09 T. G. Kolda and B. W. Bader. Tensor Decompositions and Applications. SIAM Review, Volume 51, Number 3, September 2009 csmr.ca.sandia.gov/~tgkolda/pubs/bibtgkfiles/TensorReview-preprint.pdf T. Kolda and J. Sun: Scalable Tensor Decomposition for Multi-Aspect Data Mining (ICDM 2008)

40. CMU SCS Outline Task 4: time-evolving graphs – tensors Task 5: community detection Task 6: virus propagation Task 7: scalability, parallelism and hadoop Conclusions KAIST-2011(C) 2011, C. Faloutsos 40

41. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 41 Detailed outline Motivation Hard clustering – k pieces Hard co-clustering – (k,l) pieces Hard clustering – optimal # pieces Observations

42. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 42 Problem Given a graph, and k Break it into k (disjoint) communities

43. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 43 Problem Given a graph, and k Break it into k (disjoint) communities k = 2

44. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 44 Solution #1: METIS Arguably, the best algorithm Open source, at –http://www.cs.umn.edu/~metishttp://www.cs.umn.edu/~metis and *many* related papers, at same url Main idea: –coarsen the graph; –partition; –un-coarsen

45. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 45 Solution #1: METIS G. Karypis and V. Kumar. METIS 4.0: Unstructured graph partitioning and sparse matrix ordering system. TR, Dept. of CS, Univ. of Minnesota, 1998.

46. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 46 Solution #2 (problem: hard clustering, k pieces) Spectral partitioning: Consider the 2 nd smallest eigenvector of the (normalized) Laplacian

47. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 47 Solutions #3, … Many more ideas: Clustering on the A 2 (square of adjacency matrix) [Zhou, Woodruff, PODS’04] Minimum cut / maximum flow [Flake+, KDD’00] …

48. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 48 Detailed outline Motivation Hard clustering – k pieces Hard co-clustering – (k,l) pieces Hard clustering – optimal # pieces Soft clustering – matrix decompositions Observations

49. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 49 Problem definition Given a bi-partite graph, and k, l Divide it into k row groups and l row groups (Also applicable to uni-partite graph)

50. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 50 Co-clustering Given data matrix and the number of row and column groups k and l Simultaneously –Cluster rows into k disjoint groups –Cluster columns into l disjoint groups

51. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 51 Copyright: Faloutsos, Tong (2009) 2-51 Co-clustering Let X and Y be discrete random variables – X and Y take values in {1, 2, …, m} and {1, 2, …, n} – p(X, Y) denotes the joint probability distribution—if not known, it is often estimated based on co-occurrence data –Application areas: text mining, market-basket analysis, analysis of browsing behavior, etc. Key Obstacles in Clustering Contingency Tables –High Dimensionality, Sparsity, Noise –Need for robust and scalable algorithms Reference: 1.Dhillon et al. Information-Theoretic Co-clustering, KDD’03

52. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 52 m m n nl k k l eg, terms x documents

53. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 53 doc x doc group term group x doc. group med. terms cs terms common terms med. doc cs doc term x term-group

54. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 54 Co-clustering Observations uses KL divergence, instead of L2 the middle matrix is not diagonal –we saw that earlier in the Tucker tensor decomposition s/w at: www.cs.utexas.edu/users/dml/Software/cocluster.html

55. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 55 Detailed outline Motivation Hard clustering – k pieces Hard co-clustering – (k,l) pieces Hard clustering – optimal # pieces Soft clustering – matrix decompositions Observations

56. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 56 Problem with Information Theoretic Co-clustering Number of row and column groups must be specified Desiderata: Simultaneously discover row and column groups Fully Automatic: No “magic numbers” Scalable to large graphs

57. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 57 Cross-association Desiderata: Simultaneously discover row and column groups Fully Automatic: No “magic numbers” Scalable to large matrices Reference: 1.Chakrabarti et al. Fully Automatic Cross-Associations, KDD’04

58. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 58 What makes a cross-association “good”? versus Column groups Row groups Why is this better?

59. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 59 What makes a cross-association “good”? versus Column groups Row groups Why is this better? simpler; easier to describe easier to compress!

60. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 60 What makes a cross-association “good”? Problem definition: given an encoding scheme decide on the # of col. and row groups k and l and reorder rows and columns, to achieve best compression

61. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 61 Main Idea size i * H(x i ) + Cost of describing cross-associations Code Cost Description Cost ΣiΣi Total Encoding Cost = Good Compression Better Clustering Minimize the total cost (# bits) for lossless compression

62. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 62 Algorithm k = 5 row groups k=1, l=2 k=2, l=2 k=2, l=3 k=3, l=3 k=3, l=4 k=4, l=4 k=4, l=5 l = 5 col groups

63. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 63 Experiments “CLASSIC” 3,893 documents 4,303 words 176,347 “dots” Combination of 3 sources: MEDLINE (medical) CISI (info. retrieval) CRANFIELD (aerodynamics) Documents Words

64. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 64 Experiments “CLASSIC” graph of documents & words: k=15, l=19 Documents Words

65. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 65 Experiments “CLASSIC” graph of documents & words: k=15, l=19 MEDLINE (medical) insipidus, alveolar, aortic, death, prognosis, intravenous blood, disease, clinical, cell, tissue, patient

66. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 66 Experiments “CLASSIC” graph of documents & words: k=15, l=19 CISI (Information Retrieval) providing, studying, records, development, students, rules abstract, notation, works, construct, bibliographies MEDLINE (medical)

67. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 67 Experiments “CLASSIC” graph of documents & words: k=15, l=19 CRANFIELD (aerodynamics) shape, nasa, leading, assumed, thin CISI (Information Retrieval) MEDLINE (medical)

68. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 68 Experiments “CLASSIC” graph of documents & words: k=15, l=19 paint, examination, fall, raise, leave, based CRANFIELD (aerodynamics) CISI (Information Retrieval) MEDLINE (medical)

69. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 69 Algorithm Code for cross-associations (matlab): www.cs.cmu.edu/~deepay/mywww/software/CrossAssociations -01-27-2005.tgz Variations and extensions: ‘Autopart’ [Chakrabarti, PKDD’04] www.cs.cmu.edu/~deepay

70. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 70 Algorithm Hadoop implementation [ICDM’08] Spiros Papadimitriou, Jimeng Sun: DisCo: Distributed Co-clustering with Map- Reduce: A Case Study towards Petabyte-Scale End-to-End Mining. ICDM 2008: 512-521

71. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 71 Detailed outline Motivation Hard clustering – k pieces Hard co-clustering – (k,l) pieces Hard clustering – optimal # pieces Observations

72. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 72 Observation #1 Skewed degree distributions – there are nodes with huge degree (>O(10^4), in facebook/linkedIn popularity contests!)

73. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 73 Observation #2 Maybe there are no good cuts: ``jellyfish’’ shape [Tauro+’01], [Siganos+,’06], strange behavior of cuts [Chakrabarti+’04], [Leskovec+,’08]

74. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 74 Observation #2 Maybe there are no good cuts: ``jellyfish’’ shape [Tauro+’01], [Siganos+,’06], strange behavior of cuts [Chakrabarti+,’04], [Leskovec+,’08] ? ?

75. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 75 Jellyfish model [Tauro+] … A Simple Conceptual Model for the Internet Topology, L. Tauro, C. Palmer, G. Siganos, M. Faloutsos, Global Internet, November 25-29, 2001 Jellyfish: A Conceptual Model for the AS Internet Topology G. Siganos, Sudhir L Tauro, M. Faloutsos, J. of Communications and Networks, Vol. 8, No. 3, pp 339- 350, Sept. 2006.

76. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 76 Strange behavior of min cuts ‘negative dimensionality’ (!) NetMine: New Mining Tools for Large Graphs, by D. Chakrabarti, Y. Zhan, D. Blandford, C. Faloutsos and G. Blelloch, in the SDM 2004 Workshop on Link Analysis, Counter-terrorism and Privacy Statistical Properties of Community Structure in Large Social and Information Networks, J. Leskovec, K. Lang, A. Dasgupta, M. Mahoney. WWW 2008.

77. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 77 “Min-cut” plot Do min-cuts recursively. log (# edges) log (mincut-size / #edges) N nodes Mincut size = sqrt(N)

78. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 78 “Min-cut” plot Do min-cuts recursively. log (# edges) log (mincut-size / #edges) N nodes New min-cut

79. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 79 “Min-cut” plot Do min-cuts recursively. log (# edges) log (mincut-size / #edges) N nodes New min-cut Slope = -0.5 For a d-dimensional grid, the slope is -1/d

80. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 80 “Min-cut” plot log (# edges) log (mincut-size / #edges) Slope = -1/d For a d-dimensional grid, the slope is -1/d log (# edges) log (mincut-size / #edges) For a random graph, the slope is 0

81. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 81 “Min-cut” plot What does it look like for a real-world graph? log (# edges) log (mincut-size / #edges) ?

82. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 82 Experiments Datasets: –Google Web Graph: 916,428 nodes and 5,105,039 edges –Lucent Router Graph: Undirected graph of network routers from www.isi.edu/scan/mercator/maps.html; 112,969 nodes and 181,639 edges www.isi.edu/scan/mercator/maps.html –User  Website Clickstream Graph: 222,704 nodes and 952,580 edges NetMine: New Mining Tools for Large Graphs, by D. Chakrabarti, Y. Zhan, D. Blandford, C. Faloutsos and G. Blelloch, in the SDM 2004 Workshop on Link Analysis, Counter-terrorism and Privacy

83. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 83 Experiments Used the METIS algorithm [ Karypis, Kumar, 1995] log (# edges) log (mincut-size / #edges) Google Web graph Values along the y- axis are averaged We observe a “lip” for large edges Slope of -0.4, corresponds to a 2.5- dimensional grid! Slope~ -0.4

84. CMU SCS Google graph KAIST-2011(C) 2011, C. Faloutsos 84 Log(#edges) log (mincut-size / #edges) Log(#edges) All min-cuts averaged

85. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 85 Experiments Same results for other graphs too… Lucent Router graph Clickstream graph

86. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 86 Conclusions – Practitioner’s guide Hard clustering – k pieces Hard co-clustering – (k,l) pieces Hard clustering – optimal # pieces Observations METIS Co-clustering Cross-associations ‘jellyfish’: Maybe, there are no good cuts

87. CMU SCS Outline Task 4: time-evolving graphs – tensors Task 5: community detection Task 6: virus propagation Task 7: scalability, parallelism and hadoop Conclusions KAIST-2011(C) 2011, C. Faloutsos 87

88. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 88 Detailed outline Problem definition Analysis Experiments

89. CMU SCS Immunization and epidemic thresholds Q1: which nodes to immunize? Q2: will a virus vanish, or will it create an epidemic? KAIST-2011(C) 2011, C. Faloutsos 89

90. CMU SCS Q1: Immunization: ? ? Given a network, k vaccines, and the virus details Which nodes to immunize? KAIST-2011 90 (C) 2011, C. Faloutsos

91. CMU SCS Q1: Immunization: ? ? Given a network, k vaccines, and the virus details Which nodes to immunize? KAIST-2011 91 (C) 2011, C. Faloutsos

92. CMU SCS Q1: Immunization: ? ? Given a network, k vaccines, and the virus details Which nodes to immunize? KAIST-2011 92 (C) 2011, C. Faloutsos

93. CMU SCS Q1: Immunization: ? ? Given a network, k vaccines, and the virus details Which nodes to immunize? A: immunize the ones that maximally raise the `epidemic threshold’ [Tong+, ICDM’10] KAIST-2011 93 (C) 2011, C. Faloutsos

94. CMU SCS Q2: will a virus take over? Flu-like virus (no immunity, ‘SIS’) Mumps (life-time immunity, ‘SIR’) Pertussis (finite-length immunity, ‘SIRS’) KAIST-2011(C) 2011, C. Faloutsos 94 ? ?  : attack prob  : heal prob

95. CMU SCS Q2: will a virus take over? Flu-like virus (no immunity, ‘SIS’) Mumps (life-time immunity, ‘SIR’) Pertussis (finite-length immunity, ‘SIRS’) KAIST-2011(C) 2011, C. Faloutsos 95 ? ?  : attack prob  : heal prob  depends on connectivity (avg degree? Max degree? variance? Something else?

96. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 96 The model: SIS ‘Flu’ like: Susceptible-Infected-Susceptible Virus ‘strength’ s=  /  Infected Healthy NN1 N3 N2 Prob.  Prob. β Prob. 

97. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 97 Epidemic threshold  of a graph: the value of , such that if strength s =  /  <  an epidemic can not happen Thus, given a graph compute its epidemic threshold

98. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 98 Detailed outline Problem definition Analysis Experiments

99. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 99 Epidemic threshold  What should  depend on? avg. degree? and/or highest degree? and/or variance of degree? and/or third moment of degree? and/or diameter?

100. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 100 Epidemic threshold [Theorem] We have no epidemic, if β/δ <τ = 1/ λ 1,A

101. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 101 Epidemic threshold [Theorem] We have no epidemic, if β/δ <τ = 1/ λ 1,A largest eigenvalue of adj. matrix A attack prob. recovery prob. epidemic threshold Proof: [Wang+03] (proof: for SIS=flu only)

102. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 102 Beginning of proof Healthy @ t+1: - ( healthy or healed ) - and not attacked @ t Let: p(i, t) = Prob node i is sick @ t+1 1 - p(i, t+1 ) = (1 – p(i, t) + p(i, t) *  ) *  j (1 –  aji * p(j, t) ) Below threshold, if the above non-linear dynamical system above is ‘stable’ (eigenvalue of Hessian < 1 )

103. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 103 Epidemic threshold for various networks Formula includes older results as special cases: Homogeneous networks [Kephart+White] –λ 1,A = ; τ = 1/ ( : avg degree) Star networks (d = degree of center) –λ 1,A = sqrt(d); τ = 1/ sqrt(d) Infinite power-law networks –λ 1,A = ∞; τ = 0 ; [Barabasi]

104. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 104 Epidemic threshold [Theorem 2] Below the epidemic threshold, the epidemic dies out exponentially

105. CMU SCS Recent generalization [Prakash+, arxiv ‘10]: similar threshold, for almost all virus propagation models (VPM) –SIS -> flu –SIR -> mumps –SIRS -> whooping cough (temporary immunity) –SIIR (-> HIV) –… KAIST-2011(C) 2011, C. Faloutsos 105

106. CMU SCS A2: will a virus take over? For all typical virus propagation models (flu, mumps, pertussis, HIV, etc) The only connectivity measure that matters, is  1 the first eigenvalue of the adj. matrix Proof for all VPM: [Prakash+, ‘10, arxiv] KAIST-2011(C) 2011, C. Faloutsos 106 ? ?

107. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 107 Detailed outline Epidemic threshold –Problem definition –Analysis –Experiments

108. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 108 Experiments (Oregon)  /  > τ (above threshold)  /  = τ (at the threshold)  /  < τ (below threshold)

109. KAIST-2011(C) 2011, C. Faloutsos 109 SIS simulation - # infected nodes vs time Time (linear scale) #inf. (log scale) above at below Log - Lin

110. KAIST-2011(C) 2011, C. Faloutsos 110 SIS simulation - # infected nodes vs time Log - Lin Time (linear scale) #inf. (log scale) above at below Exponential decay

111. KAIST-2011(C) 2011, C. Faloutsos 111 SIS simulation - # infected nodes vs time Log - Log Time (log scale) #inf. (log scale) above at below

112. KAIST-2011(C) 2011, C. Faloutsos 112 SIS simulation - # infected nodes vs time Time (log scale) #inf. (log scale) above at below Log - Log Power-law Decay (!)

113. How about other VPMs? KAIST-2011(C) 2011, C. Faloutsos P6-113

114. CMU SCS A2: will a virus take over? (SIRS case) KAIST-2011(C) 2011, C. Faloutsos 114 Fraction of infected Time ticks Below: exp. extinction Above: take-over Graph: Portland, OR 31M links 1.5M nodes

115. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 115 Conclusions λ 1,A : Eigenvalue of adjacency matrix determines the survival of (almost) any virus measure of connectivity (~ # paths) Can answer ‘what-if’ scenarios –May guide immunization policies Can help us avoid expensive simulations

116. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 116 References D. Chakrabarti, Y. Wang, C. Wang, J. Leskovec, and C. Faloutsos, Epidemic Thresholds in Real Networks, in ACM TISSEC, 10(4), 2008 Ganesh, A., Massoulie, L., and Towsley, D., 2005. The effect of network topology on the spread of epidemics. In INFOCOM.

117. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 117 References (cont’d) Hethcote, H. W. 2000. The mathematics of infectious diseases. SIAM Review 42, 599– 653. Hethcote, H. W. AND Yorke, J. A. 1984. Gonorrhea Transmission Dynamics and Control. Vol. 56. Springer. Lecture Notes in Biomathematics.

118. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 118 References (cont’d) Y. Wang, D. Chakrabarti, C. Wang and C. Faloutsos, Epidemic Spreading in Real Networks: An Eigenvalue Viewpoint, in SRDS 2003 (pages 25-34), Florence, Italy

119. CMU SCS Outline Task 4: time-evolving graphs – tensors Task 5: community detection Task 6: virus propagation Task 7: scalability, parallelism and hadoop Conclusions KAIST-2011(C) 2011, C. Faloutsos 119

120. CMU SCS KAIST-2011(C) 2011, C. Faloutsos P8-120 Scalability How about if graph/tensor does not fit in core? How about handling huge graphs?

121. CMU SCS KAIST-2011(C) 2011, C. Faloutsos P8-121 Scalability How about if graph/tensor does not fit in core? [‘MET’: Kolda, Sun, ICMD’08, best paper award] How about handling huge graphs?

122. CMU SCS KAIST-2011(C) 2011, C. Faloutsos P8-122 Scalability Google: > 450,000 processors in clusters of ~2000 processors each [ Barroso, Dean, Hölzle, “Web Search for a Planet: The Google Cluster Architecture” IEEE Micro 2003 ] Yahoo: 5Pb of data [Fayyad, KDD’07] Problem: machine failures, on a daily basis How to parallelize data mining tasks, then?

123. CMU SCS KAIST-2011(C) 2011, C. Faloutsos P8-123 Scalability Google: > 450,000 processors in clusters of ~2000 processors each [ Barroso, Dean, Hölzle, “Web Search for a Planet: The Google Cluster Architecture” IEEE Micro 2003 ] Yahoo: 5Pb of data [Fayyad, KDD’07] Problem: machine failures, on a daily basis How to parallelize data mining tasks, then? A: map/reduce – hadoop (open-source clone) http://hadoop.apache.org/ http://hadoop.apache.org/

124. CMU SCS KAIST-2011(C) 2011, C. Faloutsos P8-124 2’ intro to hadoop master-slave architecture; n-way replication (default n=3) ‘group by’ of SQL (in parallel, fault-tolerant way) e.g, find histogram of word frequency –compute local histograms –then merge into global histogram select course-id, count(*) from ENROLLMENT group by course-id

125. CMU SCS KAIST-2011(C) 2011, C. Faloutsos P8-125 2’ intro to hadoop master-slave architecture; n-way replication (default n=3) ‘group by’ of SQL (in parallel, fault-tolerant way) e.g, find histogram of word frequency –compute local histograms –then merge into global histogram select course-id, count(*) from ENROLLMENT group by course-id map reduce

126. CMU SCS KAIST-2011(C) 2011, C. Faloutsos P8-126 User Program Reducer Master Mapper fork assign map assign reduce read local write remote read, sort Output File 0 Output File 1 write Split 0 Split 1 Split 2 Input Data (on HDFS) By default: 3-way replication; Late/dead machines: ignored, transparently (!)

127. CMU SCS KAIST-2011(C) 2011, C. Faloutsos P8-127 D.I.S.C. ‘Data Intensive Scientific Computing’ [R. Bryant, CMU] –‘big data’ –www.cs.cmu.edu/~bryant/pubdir/cmu-cs-07- 128.pdf

128. CMU SCS KAIST-2011(C) 2011, C. Faloutsos P8-128 ~200Gb (Yahoo crawl) - Degree Distribution: in 12 minutes with 50 machines Many (link spams ?) at out-degree 1200 Analysis of a large graph

129. CMU SCS (C) 2011, C. Faloutsos 129 CentralizedHadoop/PEG ASUS Degree Distr. old Pagerank old Diameter/ANF old DONE Conn. Comp old DONE TrianglesDONE VisualizationSTARTED Outline – Algorithms & results KAIST-2011

130. CMU SCS HADI for diameter estimation Radius Plots for Mining Tera-byte Scale Graphs U Kang, Charalampos Tsourakakis, Ana Paula Appel, Christos Faloutsos, Jure Leskovec, SDM’10 Naively: diameter needs O(N**2) space and up to O(N**3) time – prohibitive (N~1B) Our HADI: linear on E (~10B) –Near-linear scalability wrt # machines –Several optimizations -> 5x faster (C) 2011, C. Faloutsos 130 KAIST-2011

131. CMU SCS ???? 19+ [Barabasi+] 131 (C) 2011, C. Faloutsos Radius Count KAIST-2011 ~1999, ~1M nodes

132. CMU SCS YahooWeb graph (120Gb, 1.4B nodes, 6.6 B edges) Largest publicly available graph ever studied. ???? 19+ [Barabasi+] 132 (C) 2011, C. Faloutsos Radius Count KAIST-2011 ?? ~1999, ~1M nodes

133. CMU SCS YahooWeb graph (120Gb, 1.4B nodes, 6.6 B edges) Largest publicly available graph ever studied. ???? 19+? [Barabasi+] 133 (C) 2011, C. Faloutsos Radius Count KAIST-2011 14 (dir.) ~7 (undir.)

134. CMU SCS YahooWeb graph (120Gb, 1.4B nodes, 6.6 B edges) 7 degrees of separation (!) Diameter: shrunk ???? 19+? [Barabasi+] 134 (C) 2011, C. Faloutsos Radius Count KAIST-2011 14 (dir.) ~7 (undir.)

135. CMU SCS YahooWeb graph (120Gb, 1.4B nodes, 6.6 B edges) Q: Shape? ???? 135 (C) 2011, C. Faloutsos Radius Count KAIST-2011 ~7 (undir.)

136. CMU SCS 136 (C) 2011, C. Faloutsos YahooWeb graph (120Gb, 1.4B nodes, 6.6 B edges) effective diameter: surprisingly small. Multi-modality (?!) KAIST-2011

137. CMU SCS Radius Plot of GCC of YahooWeb. 137 (C) 2011, C. FaloutsosKAIST-2011

138. CMU SCS 138 (C) 2011, C. Faloutsos YahooWeb graph (120Gb, 1.4B nodes, 6.6 B edges) effective diameter: surprisingly small. Multi-modality: probably mixture of cores. KAIST-2011

139. CMU SCS 139 (C) 2011, C. Faloutsos YahooWeb graph (120Gb, 1.4B nodes, 6.6 B edges) effective diameter: surprisingly small. Multi-modality: probably mixture of cores. KAIST-2011 EN ~7 Conjecture: DE BR

140. CMU SCS 140 (C) 2011, C. Faloutsos YahooWeb graph (120Gb, 1.4B nodes, 6.6 B edges) effective diameter: surprisingly small. Multi-modality: probably mixture of cores. KAIST-2011 ~7 Conjecture:

141. CMU SCS Running time - Kronecker and Erdos-Renyi Graphs with billions edges. details

142. CMU SCS (C) 2011, C. Faloutsos 142 CentralizedHadoop/PEG ASUS Degree Distr. old Pagerank old Diameter/ANF old DONE Conn. Comp old DONE TrianglesDONE VisualizationSTARTED Outline – Algorithms & results KAIST-2011

143. CMU SCS Generalized Iterated Matrix Vector Multiplication (GIMV) (C) 2011, C. Faloutsos 143 PEGASUS: A Peta-Scale Graph Mining System - Implementation and ObservationsSystem - Implementation and Observations. U Kang, Charalampos E. Tsourakakis, and Christos Faloutsos. (ICDM) 2009, Miami, Florida, USA. Best Application Paper (runner-up).ICDM KAIST-2011

144. CMU SCS Generalized Iterated Matrix Vector Multiplication (GIMV) (C) 2011, C. Faloutsos 144 PageRank proximity (RWR) Diameter Connected components (eigenvectors, Belief Prop. … ) Matrix – vector Multiplication (iterated) KAIST-2011 details

145. CMU SCS 145 Example: GIM-V At Work Connected Components – 4 observations: Size Count (C) 2011, C. Faloutsos KAIST-2011

146. CMU SCS 146 Example: GIM-V At Work Connected Components Size Count (C) 2011, C. Faloutsos KAIST-2011 1) 10K x larger than next

147. CMU SCS 147 Example: GIM-V At Work Connected Components Size Count (C) 2011, C. Faloutsos KAIST-2011 2) ~0.7B singleton nodes

148. CMU SCS 148 Example: GIM-V At Work Connected Components Size Count (C) 2011, C. Faloutsos KAIST-2011 3) SLOPE!

149. CMU SCS 149 Example: GIM-V At Work Connected Components Size Count 300-size cmpt X 500. Why? 1100-size cmpt X 65. Why? (C) 2011, C. Faloutsos KAIST-2011 4) Spikes!

150. CMU SCS 150 Example: GIM-V At Work Connected Components Size Count suspicious financial-advice sites (not existing now) (C) 2011, C. Faloutsos KAIST-2011

151. CMU SCS 151 GIM-V At Work Connected Components over Time LinkedIn: 7.5M nodes and 58M edges Stable tail slope after the gelling point (C) 2011, C. FaloutsosKAIST-2011

152. CMU SCS KAIST-2011(C) 2011, C. Faloutsos P8-152 Conclusions Hadoop: promising architecture for Tera/Peta scale graph mining Resources: http://hadoop.apache.org/core/ http://hadoop.apache.org/pig/ Higher-level language for data processing

153. CMU SCS KAIST-2011(C) 2011, C. Faloutsos P8-153 References Jeffrey Dean and Sanjay Ghemawat, MapReduce: Simplified Data Processing on Large Clusters, OSDI'04Jeffrey DeanSanjay Ghemawat Christopher Olston, Benjamin Reed, Utkarsh Srivastava, Ravi Kumar, Andrew Tomkins: Pig latin: a not-so-foreign language for data processing. SIGMOD 2008: 1099-1110Benjamin ReedUtkarsh SrivastavaRavi KumarAndrew TomkinsSIGMOD 2008

154. CMU SCS Overall Conclusions Real graphs exhibit surprising patterns (power laws, shrinking diameter, super- linearity on edge weights, triangles etc) SVD: a powerful tool (HITS, PageRank) Several other tools: tensors, METIS, … –But: good communities might not exist… Immunization: first eigenvalue Scalability: hadoop/parallelism KAIST-2011(C) 2011, C. Faloutsos 154

155. CMU SCS (C) 2011, C. Faloutsos 155 Our goal: Open source system for mining huge graphs: PEGASUS project (PEta GrAph mining System) www.cs.cmu.edu/~pegasus code and papers KAIST-2011

156. CMU SCS (C) 2011, C. Faloutsos 156 Project info Akoglu, Leman Chau, Polo Kang, U McGlohon, Mary Tong, Hanghang Prakash, Aditya KAIST-2011 Thanks to: NSF IIS-0705359, IIS-0534205, CTA-INARC ; Yahoo (M45), LLNL, IBM, SPRINT, Google, INTEL, HP, iLab www.cs.cmu.edu/~pegasus Koutra, Danae

157. CMU SCS Extra material E-bay fraud detection Outlier detection KAIST-2011(C) 2011, C. Faloutsos 157

158. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 158 Detailed outline Fraud detection in e-bay Anomaly detection

159. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 159 E-bay Fraud detection w/ Polo Chau & Shashank Pandit, CMU NetProbe: A Fast and Scalable System for Fraud Detection in Online Auction Networks, S. Pandit, D. H. Chau, S. Wang, and C. Faloutsos (WWW'07), pp. 201-210

160. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 160 E-bay Fraud detection lines: positive feedbacks would you buy from him/her?

161. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 161 E-bay Fraud detection lines: positive feedbacks would you buy from him/her? or him/her?

162. CMU SCS KAIST-2011(C) 2011, C. Faloutsos 162 E-bay Fraud detection - NetProbe Belief Propagation gives:

163. CMU SCS Popular press And less desirable attention: E-mail from ‘Belgium police’ (‘copy of your code?’) KAIST-2011(C) 2011, C. Faloutsos 163

164. CMU SCS Extra material E-bay fraud detection Outlier detection KAIST-2011(C) 2011, C. Faloutsos 164

165. CMU SCS OddBall: Spotting A n o m a l i e s in Weighted Graphs Leman Akoglu, Mary McGlohon, Christos Faloutsos Carnegie Mellon University School of Computer Science PAKDD 2010, Hyderabad, India

166. CMU SCS Main idea For each node, extract ‘ego-net’ (=1-step-away neighbors) Extract features (#edges, total weight, etc etc) Compare with the rest of the population (C) 2011, C. Faloutsos 166 KAIST-2011

167. CMU SCS What is an egonet? ego 167 egonet (C) 2011, C. FaloutsosKAIST-2011

168. CMU SCS Selected Features  N i : number of neighbors (degree) of ego i  E i : number of edges in egonet i  W i : total weight of egonet i  λ w,i : principal eigenvalue of the weighted adjacency matrix of egonet I 168 (C) 2011, C. FaloutsosKAIST-2011

169. CMU SCS Near-Clique/Star 169 KAIST-2011(C) 2011, C. Faloutsos

170. CMU SCS Near-Clique/Star 170 (C) 2011, C. FaloutsosKAIST-2011

171. CMU SCS END KAIST-2011(C) 2011, C. Faloutsos 171