1. Social Network Analysis & Network Optimization Dimitrios Katsaros, Ph.D. Koblenz, February 18 th, 2008 @ Dept. of Computer & Communication Engineering, University of Thessaly @ Dept. of Informatics, Aristotle University

2. 2 Outline of the talk A summary of my research Latest results: “ Social Network Analysis for Network Optimization” Web (2 nd round review @ IEEE Transactions on Knowledge & Data Engineering) PRIMITIVE: Community Identification PROTOCOL: Content Outsourcing GOAL: Latency Reduction Wireless Multimedia Sensor Nets (2 nd round review @ ACM Mobile Networks & Applications) PRIMITIVE: “Important” Sensor Nodes Identification PROTOCOL: Cooperative Caching GOAL: Latency Reduction Collective Intelligence: Latest step of cyberspace

3. 3 My Research Areas (chronological info) WIRELESS NETWORKS Mobile & Pervasive Computing Data Management Caching ( ’04 ) Air-Indexing ( ’07 ) Data Dissemination Broadcast Scheduling ( ’04 ) Prediction Mobility Prediction ( ’03+’08 ) Prefetching ( ’03 ) Mobile Ad Hoc Networks Content-based Multimedia Retrieval ( ’05+’08 ) Broadcasting ( ’06+’08 ) Wireless Sensor Networks Sensor Network Clustering ( ’07 ) (Distr+Local) Data Indexing ( ’06+’08 ) Cooperative Caching ( ’07+’08 ) Data Dissemination ( ’08 ) WIRED NETWORKS Conventional and Streaming Media Distribution in the Web Replication ( ’03 ) Prefetching ( ’01+’02+03 ) Caching ( ’04 ) Overlay and P2P Networks Content Distribution Networks ( ’05+’06 ) Content Placement in CDNs ( ’07+’08 ) Indexing & Query Routing in P2P ( progress ) Distributed Structures over P2P ( progress ) Web Information Retrieval and Data Mining Web Link Mining ( ’05 ) Web Ranking ( ’07+’08 ) Rank Aggregation ( ’07+’08 ) Social Network Analysis ( ’07+’08 ) Bibliometrics (’06+’07+’08)

4. 4 Research areas: Ultimately  ??? Overlay Nets Mobile/Pervasive Computing Sensors Ad Hoc Information Retrieval Web Location Tracking Caching & Air-Indexing Peer-to-Peer Networks Content Distribution Networks Caching & Prefetching & Replication & Semistructured Data & Web views Web Ranking & Search Engines Social Network Analysis Cooperative Caching & Sensor Node Clustering & Distributed Indexing & Coverage/Connectivity & Flash storage & Content-Based MIR Broadcasting & Data Dissemination Webcasting INTELLIGENCE Pervasive Web

5. 5 Social Network Analysis A social network is a social structure to describe social relations (wikipedia) The history of Social Network is older than everybody who is here More than 100 years (Cooley 1909, Durkheim 1893) Focusing on small groups Information Techniques give it a new life [ book: Stanley Wasserman & Katherine Faust ] 1.Mathematical Representation 2.Structural & Locational Properties 3.Roles & Positions 4.Dyadic & Triadic Methods

6. 6 Social Network Analysis [Stanley Wasserman & Katherine Faust] 1.Mathematical Representation 2.Structural & Locational Properties 1.Centrality 1.Betweenness Centrality 3.Roles & Positions 4.Dyadic & Triadic Methods

7. 7 Betweenness Centrality Let σ uw = σ wu denote the number of shortest paths from u  V to w  V (by definition, σ uu = 0) Let σ uw (v) denote the number of shortest paths from u to w that some vertex v  V lies on The Betweenness Centrality index NI(v) of a vertex v is defined as: Large values for the NI index of a node v indicate that this node can reach others on relatively short paths, or that v lies on considerable fractions of shortest paths connecting others

8. 8 The NI index in sample graphs In parenthesis, the NI index of the respective node; i.e., 7(156): node with ID 7 has NI equal to 156. Nodes with large NI:  Articulation nodes (in bridges), e.g., 3, 4, 7, 16, 18  With large fanout, e.g., 14, 8, U Therefore: geodesic nodes

9. 9 The NI index in a localized algorithm For any node v, the NI indexes of the nodes in N 12 (v) calculated only for the subgraph of the 2-hop (in general, k -hop) neighborhood reveal the relative importance of the nodes in N 12 For a node u (of the 2-hop neighbourhood of a node v ), the NI index of u will be denoted as NI v (u)

10. 10 Betweenness Centrality in … [ WEB ] Performing graph clustering and recognizing communities in Web site graphs [ WIRELESS MULTIMEDIA SENSOR NETWORKS ] Recognizing (in a distributed fashion) important sensor nodes, the mediators, that coordinate cooperative caching decisions

11. Community Identification & Content Outsourcing for the Web

12. 12 The need for content outsourcing

13. 13 CiBC Method Target: is true CiBC method: Building cliques and clusters around representative (pole) nodes (with low CB) Earlier methods have Defined “hard communities”:  node deg(inCom)>deg(outCom) exploited “edge betweenness” to perform hierarchical agglomerative clustering

14. 14 CiBC Method IDNI index 1020.68 219.61 611.38 110.28 72.06 01.73 90.99 8 40.75 50.00 110.00 0 1 2 3 4 5 6 7 8 10 9 11 Phase 1: NI Computation -O(nm) Phase 2: Initialization of cliques O(n)

15. 15 CiBC Method IDNI index 1020.68 219.61 611.38 110.28 72.06 01.73 90.99 8 40.75 50.00 110.00 0 1 2 3 4 5 6 7 8 10 9 11 Phase 2: Initialization of cliques O(n)

16. 16 CiBC Method IDNI index 1020.68 219.61 611.38 110.28 72.06 01.73 90.99 8 40.75 50.00 110.00 0 1 2 3 4 5 6 7 8 10 9 11 Phase 2: Initialization of cliques O(n)

17. 17 CiBC Method IDNI index 1020.68 219.61 611.38 110.28 72.06 01.73 90.99 8 40.75 50.00 110.00 0 1 2 3 4 5 6 7 8 10 9 11 Phase 2: Initialization of cliques O(n)

18. 18 CiBC Method IDNI index 1020.68 219.61 611.38 110.28 72.06 01.73 90.99 8 40.75 50.00 110.00 0 1 2 3 4 5 6 7 8 10 9 11 Phase 2: Initialization of cliques O(n)

19. 19 CiBC Method A B ABCD A3300 B3311 C0134 D0143 0 1 2 3 4 5 6 7 8 10 9 11 CD Phase 3: Clique Merging & Creation of Communities Complexity: O(l 2 ) l is the number of cliques

20. 20 CiBC Method A B ABCD A3300 B3311 C01 34 D0143 0 1 2 3 4 5 6 7 8 10 9 11 CD Phase 3: Clique Merging & Creation of Communities 4343

21. 21 CiBC Method A B ABC A330 B332 C0210 0 1 2 3 4 5 6 7 8 9 11 C Phase 3: Clique Merging & Creation of Communities

22. 22 CiBC Method A B ABC A 33 0 B332 C0210 0 1 2 3 4 5 6 7 8 9 11 C Phase 3: Clique Merging & Creation of Communities

23. 23 CiBC Method A AC A92 C210 0 1 2 3 4 5 6 7 8 9 11 C Phase 3: Clique Merging & Creation of Communities Phase 4: Check constraints

24. 24 CiBC vs. Clique Percolation Method, LRU

25. Cooperative Caching in Wireless Multimedia Sensor Networks

26. 26 The NICoCa protocol Each node is aware of its 2-hop neighborhood Uses NI to characterize some neighbors as mediators A node can be either a mediator or an ordinary node Each sensor node stores the dataID, and the actual multimedia datum the data size, TTL interval for each cached item, the timestamps of the K most recent accesses each cached item is characterized either as O (i.e., own) or H (i.e., hosted)

27. 27 The cache discovery protocol (1/2) A sensor node issues a request for a multimedia item Searches its local cache and if it is found ( local cache hit ) then the K most recent access timestamps are updated Otherwise ( local cache miss ), the request is broadcasted and received by the mediators These check the 2-hop neighbors of the requesting node whether they cache the datum ( proximity hit ) If none of them responds ( proximity cache miss ), then the request is directed to the Data Center

28. 28 The cache discovery protocol (2/2) When a mediator receives a request, searches its cache If it deduces that the request can be satisfied by a neighboring node ( remote cache hit ), forwards the request to the neighboring node with the largest residual energy If the request can not be satisfied by this mediator node, then it does not forward it recursively to its own mediators, since this will be done by the routing protocol, e.g., AODV If none of the nodes can help, then requested datum is served by the Data Center ( global hit )

29. 29 Cache vs. hits (MB files & uniform access) in a sparse WMSN (d = 4) HYBRID: appears at: L. Yin and G. Cao, “Supporting cooperative caching in ad hoc networks”, IEEE Transactions on Mobile Computing, 5(1):77-89, 2006

30. 30 Cache vs. hits (MB files & uniform access) in a dense WMSN (d = 7) HYBRID: appears at: L. Yin and G. Cao, “Supporting cooperative caching in ad hoc networks”, IEEE Transactions on Mobile Computing, 5(1):77-89, 2006

31. 31 Evolution of cyberspace … Semantic Web + Pervasive Computing WWW + Broadband + WIFI + grid computing Unicode + XML + RDF + Ontologies Internet + Multimedia + URL + HTTP + HTML Servers + Telecom Networks + PCs + TCP-IP + e-mail + FTP Computers + Micro-chips + Application Software + WYSIWYG Interfaces Transistors+Formal Logic+Digital Coding+Program. Languages Collective Intelligence Net Semantic Web WWW Internet PC Computer

32. 32 Why Collective Intelligence? Users/ devices generate data at an unprecedented rate Blogs Tags Sensor measurements Web pages Rankings by search engines They could be treated as “opinions” or “votes” Under some conditions: group IQ > individual IQ [So far] Opinion/Vote fusion: PageRank (i.e., collective linking preferences) Metasearching (ranked list merging) Collaborative filtering ( what is interesting from what other people say, what people like you say) …..

33. 33 Collective Intelligence: Some challenges Statistical analysis of social networks Identification of influential opinions and/or producers Discover social context to provide personalization Opinion spam Bias filtering

34. 34 Collective Intelligence: Some challenges Finding high-quality content Opinion mining Dealing with controversies Metadata from data analysis Storage of metadata ……………. MOST IMPORTANTLY In Centralized and/or Distributed settings

35. Thank you for your attention! Questions?

36. 36 References Our work D. Katsaros , G. Pallis, K. Stamos, A. Sidiropoulos, A. Vakali, Y. Manolopoulos. “ CDNs Content Outsourcing via Generalized Communities ”. IEEE Transactions on Knowledge and Data Engineering, (under second round review), December, 2007. N. Dimokas, D. Katsaros, and Y. Manolopoulos, “ Cooperative Caching in Wireless Multimedia Sensor Networks ” ACM Mobile Networks and Applications, (under second round review), February, 2008. Competing methods [ CPM community identification method ] G. Palla, I.Derenyi, I.Farkas, and T.Vicsek. Uncovering the overlapping community structure of complex networks in nature and society. Nature, 435(7043):814–818, 2005. [ Hybrid cooperative caching method ] L. Yin and G. Cao. Supporting cooperative caching in ad hoc networks. IEEE Transactions on Mobile Computing, 5(1):77–89, 2006.