1. Practical Applications of Complex Network Theory Niloy Ganguly (IIT Kharagpur)

2. Complex Network Society and nature of human interaction is a Complex System Complex Network: A generic tool to model complex systems  There is a growing body of work on CNT Theory  Applied to a variety of fields – Social, Biological, Physical & Cognitive sciences, Engineering & Technology

3. Objective of this Tutorial Familiarize with tools and techniques of complex Network Present practical examples where complex network concepts are used  Peer to peer networks  Language modeling

4. Outline of the Tutorial Part I: Introduction  Network Analysis Techniques  Network Synthesis Techniques Part II: Case Studies  Topology management of peer to peer networks  Self-organization of Sound Systems Conclusion and Discussions

5. Complex System Non-trivial properties and patterns emerging from the interaction of a large number of simple entities Self-organization: The process through which these patterns evolve without any external intervention or central control Emergent Property or Emergent Behavior: The pattern that emerges due to self-organization

6. The best example from nature A termite "cathedral" mound produced by a termite colony

7. Emergence of a networked life Atom Molecule Cell Tissue Organs Organisms Communities

8. Complex Network Theory Handy toolbox for modeling mesoscopy Marriage of Graph theory, social networks and Statistics Complex because:  Non-trivial topology  Difficult to specify completely  Usually large (in terms of nodes and edges)  Dynamicity Provides insight into the nature and evolution of the system being modeled

9. Internet and www

10. Genetic interaction network

11. 9-11 Terrorist Network Social Network Analysis is a mathematical methodology for connecting the dots -- using science to fight terrorism. Connecting multiple pairs of dots soon reveals an emergent network of organization.

12. CNT Examples: Road and Airlines Network

13. What Questions can be asked Does these networks display some symmetry? Are these networks creation of intelligent objects or they have emerged? How have these networks emerged ? What are the underlying simple rules leading to their complex formation?

14. Bi-directional Approach Analysis of the real-world networks  Global topological properties  Community structure  Node-level properties Synthesis of the network by means of some simple rules  Preferential attachment models  Small-world models

15. Topological Characterization of Networks

16. Types Of Networks and Representation UnipartiteBinary/ Weighted Undirected/ Directed BipartiteBinary/ Weighted Undirected/ Directed Representation 1.Adjacency Matrix 2.Adjacency List a{b,c} b{a,c} c{a,b} abc a011 b101 c110

17. Properties of Adjacency Matrix A={a ij }, where i and j are nodes and a ij =1 if there is an edge between i an j. A 2 = A*A; Entries denote number paths of length 2 between any two node (Σa ik *a kj ) In general, A n denotes number of paths of length n Trace(A) = Σa ii How is the trace of A 3 related to the number of triangles in the network? k

18. Characterization of Complex Networks They have a non-trivial topological structure Properties:  Heavy tail in the degree distribution (non-negligible probability mass towards the tail; more than in the case of an exp. distribution)  High clustering coefficient  Centrality Properties  Social Roles & Equivalence  Assortativity  Community Structure  Random Graphs & Small avg. path length  Preferential attachment  Small World Properties

19. Degree Distribution (DD) Let p k be the fraction of vertices in the network that has a degree k. The k versus p k plot is defined as the degree distribution of a network For most of the real world networks these distributions are right skewed with a long right tail showing up values far above the mean – p k varies as k -α Due to noisy and insufficient data sometimes the definition is slightly modified  Cumulative degree distribution is plotted Probability that the degree of a node is greater than or equal to k

20. A Few Examples Power law: P k ~ k -α

21. Friend of Friends Consider the following scenario  Sourish and Ravi are friends  Sourish and Shaunak are friends  Are Shaunak and Ravi friends?  If so then … This property is known as transitivity Ravi SaurishSaunak

22. Measuring Transitivity: Clustering Coefficient The clustering coefficient for a vertex ‘v’ in a network is defined as the ratio between the total number of connections among the neighbors of ‘v’ to the total number of possible connections between the neighbors High clustering coefficient means my friends know each other with high probability – a typical property of social networks

23. Mathematically… The clustering coefficient of a vertex i is The clustering coefficient of the whole network is the average Alternatively, Ci =Ci = # of links between ‘n’ neighbors n(n-1)/2 C=C= 1 N ∑C i C = # triangles in the n/w # triples in the n/w

24. Centrality Centrality measures are commonly described as indices of 4 Ps -- prestige, prominence, importance, and power  Degree – Count of immediate neighbors  Betweenness – Nodes that form a bridge between two regions of the n/w Where σ st is total number of shortest paths between s and t and σ st (v) is the total number of shortest paths from s to t via v

25. Eigenvector centrality – Bonacich (1972) It is not just how many people knows me counts to my popularity (or power) but how many people knows people who knows me – this is recursive! In context of HIV transmission – A person x with one sex partner is less prone to the disease than a person y with multiple partners  But imagine what happens if the partner of x has multiple partners  The basic idea of eigenvector centrality

26. Definition Eigenvector centrality is defined as the principal eigenvector of the adjacency matrix Eigenvector of any symmetric matrix A = {a ij } is any vector e such that  Where λ is a constant and e i is the centrality of the node i What does it imply – centrality of a node is proportional to the centrality of the nodes it is connected to (recursively)… Practical Example: Google PageRank

27. Node Equivalence Social Roles – Nodes (actors) in a social n/w who have similar patterns of relations (ties) with other nodes. Three Different ways to find equivalence classes:  Structural Equivalence  Automorphic Equivalence  Regular Equivalence

28. Structural Equivalence Two nodes are said to be exactly structurally equivalent if they have the same relationships to all other nodes. Computation: Let A be the adjacency matrix. Compute the Euclidean Distance /Pearson Correlation between a pair or rows/columns representing the neighbor profile of two nodes (say i and j). This value shows how much structurally similar i and j are.

29. Automorphic Equivalence The idea of automorphic equivalence is that sets of actors can be equivalent by being embedded in local structures that have the same patterns of ties -- "parallel" structures. Top Boss Managers of 3 Diff. stores Workers Swap(B,D) with all their neighbors:The distances among all the actors in the graph would be exactly identical Path vectors of i: how many nodes are at distance 1, 2, … from node i. Amount of Equivalence: Distance between path vectors

30. Regular Equivalence Two nodes are said to be regularly equivalent if they have the same profile of ties with members of other sets of actors that are also regularly equivalent. Class III Class II Class I No tie with Class I 1 tie with Class II 1 tie with Class I 1/2 tie(s) with Class III 1 tie with Class II No tie with Class III

31. Rich goes with the rich (selective linking)  A famous actor (e.g., Shah Rukh Khan) would prefer to pair up with some other famous actor (e.g., Rani Mukherjee) in a movie rather than a new comer in the film industry. Assortativity (homophily) Assortative Scale-free network Disassortative Scale-free network

32. Measures of Assortativity  ANND (Average nearest neighbor degree) Find the average degree of the neighbors of each node i with degree k Find the Pearson correlation (r) between the degree of i and the average degree of its neighbors For further reference see the supplementary material

33. Community structure  Community structure: a group of vertices that have a high density of edges within them and a low density of edges in between groups  Example : Friendship n/w of children Citation n/ws: research interest World Wide Web: subject matter of pages Metabolic networks: Functional units Linguistic n/ws: similar linguistic categories

34. Some Examples Community Structure in Political Books Community structure in a Social n/w of Students (American High School)

35. Community Identification Algorithms Hierarchical Girvan-Newman Radicchi et al. Chinese Whispers Spectral Bisection See (Newman 2004) for a comprehensive survey (you will find the ref. in the supplementary material)

36. Girvan-Newman Algorithm Bisection Method  Calculate the betweenness for all edges in the network.  Remove the edge with the highest betweenness.  Recalculate betweennesses for all edges affected by the removal.  Repeat from step 2 until no edges remain.

37. Evolution of Networks Processes on Networks

38. Random Graphs & Small Average Path Length Q: What do we mean by a ‘random graph’? A: Erdos-Renyi random graph model: For every pair of nodes, draw an edge between them with equal probability p. Poisson distribution P(k)~ e - k /k! N nodes z neighbors per node, on average, z = D degrees of separation Degrees of Separation in a Random Graph

39. Degree Distributions

40. (a)World-Wide Web (b)Coauthorship networks: computer science, high energy physics, condensed matter physics, astrophysics (c)Power grid of the western United States and Canada (d)Social network of 43 Mormons in Utah Degree distributions for various networks

41. How do Power law DDs arise? (1) GROWTH : Starting with a small number of nodes (m 0 ) at every timestep we add a new node with m (<=m 0 ) edges (connected to the nodes already present in the system). (2) PREFERENTIAL ATTACHMENT : The probability Π that a new node will be connected to node i depends on the connectivity k i of that node A.-L.Barabási, R. Albert, Science 286, 509 (1999) Barabási-Albert Model of Preferential Attachment (Rich gets Richer)

42. Growth analysis Markov chain representation Probability that the new edge is attached to any of the vertices of degree k where total number of edges

43. Growth analysis Markov chain representation Growth dynamics at time (t+1) Number of nodes of degree (k-1) at t Number of nodes of degree k at t+1 Number of nodes of degree k at t

44. Growth analysis Markov chain representation The net change in np k per vertex added In the stationary solution, we find Which results

45. The World is Small! “Registration fee for ICDCN 2008 are being waived for all participants – get it collected from the registration counter” How long do you think the above information will take to spread among yourselves Experiments say it will spread very fast – within 6 hops from the initiator it would reach all This is the famous Milgram’s six degrees of separation

46. The Small World Effect Even in very large social networks, the average distance between nodes is usually quite short. Milgram’s small world experiment: Target individual in Boston Initial senders in Omaha, Nebraska Each sender was asked to forward a packet to a friend who was closer to the target Friends asked to do the same Result: Average of ‘six degrees’ of separation. S. Milgram, The small world problem, Psych. Today, 2 (1967), pp. 60-67.

47. Measure of Small-Worldness Low average geodesic path length High clustering coefficient Geodesic path – Shortest path through the network from one vertex to another Mean path length  ℓ = 2∑ i≥j d ij /n(n+1) where d ij is the geodesic distance from vertex i to vertex j  Most of the networks observed in real world have ℓ ≤ 6 Film actors3.48 Company Directors4.60 Emails4.95 Internet3.33 Electronic circuits4.34

48. Clustering C = Probability that two of a node’s neighbors are themselves connected In a random graph: C rand ~ 1/N (if the average degree is held constant)

49. Watts-Strogatz ‘Small World’ Model Watts and Strogatz introduced this simple model to show how networks can have both short path lengths and high clustering. D. J. Watts and S. H. Strogatz, Collective dynamics of “small-world” networks, Nature, 393 (1998), pp. 440–442.

50. Small-world model Used for modeling network transitivity Many networks assume some kind of geographical proximity Small-world model:  Start with a low-dimensional regular lattice  Rewire: Add/remove edges to create shortcuts to join remote parts of the lattice For each edge with prob p move the other end to a random vertex Rewiring allows to interpolate between regular lattice and random graph

51. Small-world model Regular lattice ( p=0 ):  Clustering coefficient C=(3k- 3)/(4k-2)=3/4  Mean distance L/4k Almost random graph ( p=1 ):  Clustering coefficient C=2k/L  Mean distance log L / log k No power-law degree distribution Rewiring probability p Degree distribution

52. Case Study I Analyzing the Vulnerability of Superpeer Networks Against Churn and Attack

53. Poster - Developing Analytical Framework to Measure Stability of P2P Networks, ACM Sigcomm 2006 Pisa, Italy Brief Communication - Measuring Robustness of Superpeer Topologies, PODC 2007, Portland, USA How stable are large superpeer networks against attack? The Seventh IEEE Conference on Peer-to-Peer Computing, Galway, Ireland, 2007 Full paper - Analyzing the Vulnerability of the Superpeer Networks Against Attack, ACM CCS, 14th ACM Conference on Computer and Communications Security, Alexandria, USA, 29 October - 2 Nov, 2007.

54. Client/Server architecture Servers: Provide services. Clients : Request services from servers Very successful architecture  WWW (HTTP), FTP, Web services, etc. Server Client Internet

55. Client/Server architecture Limitations Scalability : Hard to achieve Poor fault tolerance : Single point of failure Administration : Highly required

56. Peer to Peer architecture All peers act as both clients and servers i.e. Servent (SERVer+cliENT)  Provide and consume data  Any node can initiate a connection No centralized data source  “The ultimate form of democracy on the Internet” File sharing and other applications like IP telephony, distributed storage, publish subscribe system etc Node Internet

57. Peer to peer and overlay network  An overlay network is built on top of physical network  Nodes are connected by virtual or logical links  Underlying physical network becomes unimportant  Interested in the complex graph structure of overlay

58. Dynamicity of overlay networks Peers in the p2p system leave network randomly without any central coordination (user churn) Important peers are targeted for attack  DoS attack drown important nodes in fastidious computation Fail to provide services to other peers  Importance of a node is defined by centrality measures Like degree centrality, betweenness centraliy etc

59. Peers in the p2p system leave network randomly without any central coordination (user churn) Important peers are targeted for attack  Makes overlay structures highly dynamic in nature  Frequently it partitions the network into smaller fragments  Communication between peers become impossible Dynamicity of overlay networks

60. Problem definition Investigating stability of the networks against the churn and attack Network Topology+ Dynamicity = How (long) stable Developing an analytical framework Examining the impact of different structural parameters upon stability  Peer contribution  degree of peers, superpeers  their individual fractions

61. Steps followed to analyze Modeling of  Overlay topologies pure p2p networks, superpeer networks, hybrid networks  Various kinds of failures and attacks Defining stability metric Developing the analytical framework Validation through simulation Understanding impact of structural parameters

62. Modeling overlay topologies Topologies are modeled by various random graphs characterized by degree distribution p k = Fraction of nodes having degree k Examples: Erdos-Renyi graph Scale free network Superpeer networks

63. Modeling overlay topologies: E-R graph, scale free networks Erdos-Renyi graph  Degree distribution follows Poisson distribution. Scale free network  Degree distribution follows power law distribution Average degree

64. Modeling overlay topologies: Superpeer networks Superpeer networks emerge as most widely used network  Small fraction of nodes are superpeers and rest are peers  KaZaA adopted this kind of topology  Can be modeled using bimodal degree distribution  Mathematically if otherwise Superpeer Node Peer node

65. Modeling peer dynamics We propose a generalized model for peer dynamics Probability of removal of a node having degree k is f k  k ,  models peer dynamics By changing the value of , we can obtain various peer dynamics like  random failure, degree dependent failure  deterministic and degree dependent attack q k models the probability of survival of a node of degree k after the disrupting event q k =1-f k 

66. Generalized model for peer dynamics  = 0 (degree independent failure)  Probability of removal of a node ( f k ) is constant & degree independent i.e. q k =q  < 0 (degree dependent failure)  Probability of removal of a node ( f k ) is inversely proportional to the degree of that node (1/k)  Peers having lower connectivity or bandwidth are less stable because they enter and leave network frequently  > 0 (Attack)  Peers with high degrees are targeted.

67. Modeling: Attack Deterministic attack  Nodes having high degrees are progressively removed q k =0 when k>k max 0< q k < 1 when k=k max q k =1 when k<k max Degree dependent attack  Nodes having high degrees are likely to be removed  Probability of removal of node having degree k

68. Stability Metric: Percolation Threshold Initially all the nodes in the network are connected Forms a single component Nodes in the network are connected and form a single giant component

69. Stability Metric: Percolation Threshold Initial single connected component f fraction of nodes removed Giant component still exists Initially all the nodes in the network are connected Forms a single component Size of the giant component is the order of the network size Giant component carries the structural properties of the entire network

70. Stability Metric: Percolation Threshold Initial single connected component f fraction of nodes removed Giant component still exists f c fraction of nodes removed The entire graph breaks into smaller fragments Therefore f c =1-q c becomes the percolation threshold

71. Ordinary Percolation on Lattices Fill in each link (bond percolation) or site (site percolation) with probability p and ask questions about the sizes of connected components.

72.  Generating function:  Formal power series whose coefficients encode information Here encode information about a sequence  Used to understand different properties of the graph  generates probability distribution of the vertex degrees.  Average degree Development of the analytical framework

73. specifies the probability of a node having degree k to be present in the network after f k = (1-q k ) fraction of nodes removed. becomes the corresponding generating function. Development of the analytical framework ( 1-q k ) fraction of nodes removed

74. specifies the probability of a node having degree k to be present in the network after (1-q k ) fraction of nodes removed. becomes the corresponding generating function. Distribution of the outgoing edges of first neighbor of a randomly chosen node Development of the analytical framework Random node First neighbor

75. Development of the analytical framework H 1 (x) generates the distribution of the size of the components that are reached through random edge H 1 (x) satisfies the following condition

76. generates distribution for the component size to which a randomly selected node belongs to Average size of the components Average component size becomes infinity when Development of the analytical framework

77. Average component size becomes infinity when With the help of generating function, we derive the following critical condition for the stability of giant component The critical condition is applicable  For any kind of topology (modeled by p k )  Undergoing any kind of dynamics (modeled by 1-q k ) Degree distribution Peer dynamics Development of the analytical framework

78. Stability metric: simulation The theory is developed based on the concept of infinite graph At percolation point  theoretically ‘infinite’ size graph reduces to the ‘finite’ size components In practice we work on finite graph  cannot simulate the phenomenon directly We approximate the percolation phenomenon on finite graph with the help of condensation theory

79. How to determine percolation point during simulation? Let s denotes the size of a component and n s determines the number of components of size s at time t At each timestep t a fraction of nodes is removed from the network Calculate component size distribution If becomes monotonically decreasing function at the time t  t becomes percolation point Initial condition (t=1)Intermediate condition (t=5) Percolation point (t=10)

80. Outline of the results Networks under consideration Disrupting events Superpeer networks (Characterized by bimodal degree distribution ) Degree independent failure or random failure Degree dependent failure Degree dependent attack Deterministic attack (special case of degree dependent attack ??)

81. Outline of the results Networks under consideration Disrupting events Superpeer networks (Characterized by bimodal degree distribution ) Degree independent failure or random failure Degree dependent failure Degree dependent attack Deterministic attack (special case of degree dependent attack ??)

82. Stability against various failures Degree independent random failure : Percolation threshold For superpeer networks Average degree of the network Superpeer degree Fraction of peers

83. Stability against random failure (superpeer networks) Comparative study between theoretical and experimental results We keep average degree fixed

84. Stability against random failure (superpeer networks) Comparative study between theoretical and experimental results Increase of the fraction of superpeers (specially above 15% to 20%) increases stability of the network

85. Stability against random failure (superpeer networks) Comparative study between theoretical and experimental results There is a sharp fall of f c when fraction of superpeers is less than 5%

86. Stability of superpeer networks against deterministic attack Two different cases may arise Case 1:  Removal of a fraction of high degree nodes are sufficient to breakdown the network Case 2:  Removal of all the high degree nodes are not sufficient to breakdown the network  Have to remove a fraction of low degree nodes

87. Stability of superpeer networks against deterministic attack Two different cases may arise Case 1:  Removal of a fraction of high degree nodes are sufficient to breakdown the network Case 2:  Removal of all the high degree nodes are not sufficient to breakdown the network  Have to remove a fraction of low degree nodes  Interesting observation in case 1  Stability decreases with increasing value of peers – counterintuitive

88. Peer contribution Controls the total bandwidth contributed by the peers  Determines the amount of influence superpeer nodes exert on the network Peer contribution where is the average degree We investigate the impact of peer contribution upon the stability of the network

89. Impact of peer contribution for deterministic attack The influence of high degree peers increases with the increase of peer contribution This becomes more eminent as peer contribution

90. Impact of peer contribution for deterministic attack Stability of the networks ( ) having peer contribution primarily depends upon the stability of peer ( )

91. Impact of peer contribution for deterministic attack Stability of the network increases with peer contribution for peer degree k l =3,5 Gradually reduces with peer contribution for peer degree k l =1

92. Stability of superpeer networks against degree dependent attack Probability of removal of a node is directly proportional to its degree  Hence  Calculation of normalizing constant C Minimum value This yields an inequality

93. Stability of superpeer networks against degree dependent attack Probability of removal of a node is directly proportional to its degree  Hence  Calculation of normalizing constant C Minimum value The solution set of the above inequality can be  either bounded  or unbounded

94. Degree dependent attack: Impact of solution set Three situations may arise  Removal of all the superpeers along with a fraction of peers – Case 2 of deterministic attack  Removal of only a fraction of superpeer – Case 1 of deterministic attack  Removal of some fraction of peers and superpeers

95. Degree dependent attack: Impact of solution set Three situations may arise  Case 2 of deterministic attack Networks having bounded solution set If,  Case 1 of deterministic attack Networks having unbounded solution set If,  Degree Dependent attack is a generalized case of deterministic attack

96. Case Study : Superpeer network with k l =3, km=25,  k  =5  Performed simulation on graphs with N=5000 and 500 cases Bounded solution set with  Removal of any combination of where disintegrates the network  At, all superpeer need to be removed along with a fraction of peers  Good agreement between theoretical and simulation results Impact of critical exponent  c Validation through simulation

97. Summarization of the results Random failure  Stability increases with superpeer degree and its fraction  Drastic fall of the stability when fraction of superpeers is less than 5% In deterministic attack, networks having small peer degrees are very much vulnerable Increase in peer degree improves stability Superpeer degree is less important here! In degree dependent attack,  Stability condition provides the critical exponent Amount of peers and superpeers required to be removed is dependent upon  More general kind of attack

98. Limitations We have not considered the change in the degree distribution in the network due to disrupting events Assumed that nodes are turned OFF during disrupting events Topological change in the network should be included in the theory

99. Deformation of the network due to node removal Removal of a node along with its adjacent links changes the degrees of its neighbors  Hence changes the topology of the network Let initial degree distribution of the network be pk Probability of removal of a node having degree k is fk We represent the new degree distribution pk’ as a function of pk and fk

100. Deformation of the network due to node removal  In this diagram, left node set denotes the survived nodes and right node set denotes the removed nodes  The change in the degree distribution is due to the edges removed from the left set  We calculate the number of edges connecting left and right set (E)

101. Deformation of the network due to node removal  The total number of tips in the surviving node set is  The probability of finding a random tip that is going to be removed is  The total number of edges running between two subset

102. Deformation of the network due to node removal Probability of finding an edge in the surviving (left) subset that is connected to a node of removed (right) subset

103. Deformation of the network due to node removal Removal of a node reduces the degree of the survived nodes node having degree > k becomes a node having degree k by losing one or more edges Probability that a survived node will lose one edge becomes

104. Deformation of the network due to node removal Probability of finding a node having degree k (pk’) after removal of nodes following fk, depends upon 1. Probability that nodes having degree k, k+1, k+2 … will lose 0, 1, 2 etc edges respectively to become a node having degree k 2. Probability that nodes having degree k, k+1, k+2 … will sustain k number of edges with them Hence Where denotes the fraction of nodes in the survived (left) node set having degree q

105. Deformation of the network due to node removal For random failure, Hence the new degree distribution after random failure becomes

106. Deformation of the network due to node removal Degree distribution of the Poisson and power law networks after the attack of the form

107. Percolation Threshold Initially all the nodes in the network are connected Forms a single giant component Size of the giant component is the order of the network size Giant component carries the structural properties of the entire network Nodes in the network are connected and form a single giant component

108. Percolation Threshold Initial single connected component f fraction of nodes removed Giant component still exists

109. Percolation Threshold Initial single connected component f fraction of nodes removed Giant component still exists f c fraction of nodes removed The entire graph breaks into smaller fragments Therefore f c =1-q c becomes the percolation threshold

110. Percolation threshold Percolation condition of a network having degree distribution pk can be given as After removal of fk fraction of nodes, if the degree distribution of the network becomes pk’, then the condition for percolation becomes Which leads to the following critical condition for percolation

111. Percolation threshold From this equation, the percolation threshold fq can be obtained numerically for any kind of network characterized by degree distribution pk under any kind of disrupting event checterized by fk For random failure, percolation threshold becomes

112. Conclusion Contribution of the work Development of general framework to analyze the stability of superpeer networks Modeling the dynamic behavior of the peers using degree independent failure as well as attack. Comparative study between theoretical and simulation results to show the effectiveness of our theoretical model.

113. CASE STUDY II: Self-Organization of the Sound Inventories

114. Rediscovering the Co-occurrence Principles of the Vowel Inventories: A Complex Network Approach, Advances in Complex System Modeling the Co-occurrence Principles of the Consonant Inventories: A Complex Network Approach, Int. Jour. of Modern Phy. C Redundancy Ratio: An Invariant Property of the Consonant Inventories of the World's Languages, ACL(07), Prague, Czech Republic. Analysis and Synthesis of the Distribution of Consonants over Languages: A Complex Network Approach, COLING-ACL(06), Sydney, Australia.

115. Human Communication Human beings and many other living organisms produce sound signals Unlike other organisms, they can concatenate these sounds to produce new messages – Language Language is one of the primary cause/effect of human intelligence

116. Human Speech Sounds Human speech sounds are called phonemes – the smallest unit of a language Phonemes are characterized by certain distinctive features like Mermelstein’s Model I.Place of articulation II.Manner of articulation III.Phonation

117. Types of Phonemes Vowels Consonants Diphthongs /ai/ L /a/ /i/ /u//p/ /t/ /k/

118. Choice of Phonemes How a language chooses a set of phonemes in order to build its sound inventory? Is the process arbitrary? Certainly Not! What are the forces affecting this choice?

119. Forces of Choice /a/ Speaker Listener / Learner /a/ Desires “ease of articulation” Desires “perceptual contrast” / “ease of learnability” A Linguistic System – How does it look? The forces shaping the choice are opposing – Hence there has to be a non-trivial solution

120. Vowels: A (Partially) Solved Mystery Languages choose vowels based on maximal perceptual contrast. For instance if a language has three vowels then in more than 95% of the cases they are /a/,/i/, and /u/. Maximally Distinct /u/ /a/ /i/

121. Consonants: A puzzle Research: From 1929 – Date No single satisfactory explanation of the organization of the consonant inventories  The set of features that characterize consonants is much larger than that of vowels  No single force is sufficient to explain this organization  Rather a complex interplay of forces goes on in shaping these inventories J i g s a w

122. The Approach & Objective We adopt a Complex Network Approach to attack the problem of consonant inventories We try to figure out the principle of the distribution of the occurrence of consonants over languages We also attempt to figure out the co- occurrence patterns (if any) that are found across the consonant inventories

123. Principle of Occurrence PlaNet – The “Phoneme-Language Network”  A bipartite network N=(VL,VC,E)  VL : Nodes representing languages of the world  VC : Nodes representing consonants  E : Set of edges which run between VL and VC There is an edge e Є E between two nodes vl Є VL and vc Є VC if the consonant c occurs in the language l. L1L1 L4L4 L2L2 L3L3 /m/ /ŋ//ŋ/ /p/ /d/ /s/ /θ//θ/ Consonants Languages The Structure of PlaNet

124. Construction of PlaNet Data Source : UCLA Phonological Inventory Database (UPSID) Number of nodes in VL is 317 Number of nodes in VC is 541 Number of edges in E is 7022

125. Degree Distribution of PlaNet 0 50100 150 0.02 0.04 0.06 0.08 Language inventory size (degree k) pkpk p k = beta(k) with α = 7.06, and β = 47.64 p k = Γ(54.7) k 6.06 (1-k) 46.64 Γ(7.06) Γ(47.64) k min = 5, k max = 173, k avg = 21 200 PkPk 1000 Degree of a consonant, k P k = k -0.71 Exponential Cut-off 1 10 100 0.001 0.01 0.1 1 DD of the language nodes follows a β- distribution DD of the consonant nodes follows a power-law with an exponential cut-off Distribution of Consonants over Languages follow a power-law

126. Synthesis of PlaNet Given: V L = {L 1, L 2,..., L 317 } sorted in the ascending order of their degrees and 541 unlabeled nodes in V C. Step 0: All nodes in V C have degree 0. Step t+1: Choose a language node L j (in order) with cardinality k j (inventory size) for c running from 1 to k j do Pr(C i ) = d i α + ε ∑ x  V* (d x α + ε) Connect L j preferentially with a consonant node C i  V C, to which it is already not connected, with a probability where, d i = degree of node C i at step t and V* = subset of V C not connected to L j at t and ε is the smoothing parameter.

127. L1L1 L3L3 L2L2 L4L4 L1L1 L3L3 L2L2 L4L4 The Preferential Mechanism of Synthesis After step 3 After step 4

128. Simulation Result The parameters α and ε are 1.44 and 0.5 respectively. The results are averaged over 100 runs PlaNet rand PlaNet PlaNet syn 1 10 100 1000 1.1.01.001 Degree (k) PkPk

129. Principle of Co-occurrence Consonants tend to co-occur in groups or communities These groups tend to be organized around a few distinctive features (based on: manner of articulation, place of articulation & phonation) – Principle of feature economy If a language has in its inventory then it will also tend to have voiced voiceless bilabial dental /b//p/ /d//t/ plosive

130. How to Capture these Co-occurrences? PhoNet – “Phoneme Phoneme Network”  A weighted network N=(VC,E)  VC : Nodes representing consonants  E : Set of edges which run between the nodes in VC There is an edge e Є E between two nodes vc1,vc2 Є VC if the consonant c 1 and c 2 co-occur in a language. The number of languages in which c 1 and c 2 co-occurs defines the edge-weight of e. The number of languages in which c 1 occurs defines the node-weight of v c1. /k w / /k′/ /k/ /d′/ 42 14 38 13 283 17 50 39

131. Construction of PhoNet Data Source : UPSID Number of nodes in V C is 541 Number of edges is 34012 PhoNet

132. Community Structures in PhoNet Radicchi et al. algorithm (for unweighted networks) – Counts number of triangles that an edge is a part of. Inter-community edges will have low count so remove them. Modification for a weighted network like PhoNet  Look for triangles, where the weights on the edges are comparable.  If they are comparable, then the group of consonants co- occur highly else it is not so.  Measure strength S for each edge (u,v) in PhoNet where S is,  Remove edges with S less than a threshold η S = w uv √Σ i Є V c -{u,v} (w ui – w vi ) 2 if √Σ i Є V c -{u,v} (w ui – w vi ) 2 >0 else S = ∞

133. 3 1 2 4 100 110 101 10 5 6 46 52 45 3 1 2 4 11.11 10.94 7.14 0.06 5 6 3.77 5.17 7.5 S η >1 3 1 2 6 4 5 Community Formation For different values of η we get different sets of communities

134. Consonant Societies! η=1.25 η=0.72 η=0.60 η=0.35

135. The Binding Force of the Communities: Feature Economy Feature Entropy: The idea is borrowed from information theory For a community C of size N, let there be pf consonants for which a particular feature f is present and qf other consonants for which f is absent – probability that a consonant chosen from C has f is pf /N and that it does have f is qf /N or (1- pf /N) Feature entropy can be therefore defined as where F is the set of all features present in the consonants in C Essentially the number of bits needed to transmit the entire information about C through a channel. Σ FЄf (-(p f /N)log(p f /N) – (q f /N)log(q f /N)) F E =

136. Computing Feature Entropy Lower F E -> C 1 economizes on the number of features Higher F E -> C 2 does not economize on the number of features

137. If the Inventories had Evolved by Chance! Construction of PhoNetrand  For each consonant c let the frequency of occurrence in UPSID be denoted by f c.  Let there be 317 bins each corresponding to a language in UPSID.  f c bins are then chosen uniformly at random and the consonant c is packed into these bins without repetition.  Thus the consonant inventories of the 317 languages corresponding to the bins are generated.  PhoNetrand can be constructed from these new consonant inventories similarly as PhoNet. Cluster PhoNet rand by the method proposed earlier

138. PhoNet PhoNet rand 0 5 10 15 20 10 5 0 Average Feature Entropy Community Size The curve shows the average feature entropy of the communities of a particular size versus the community size Comparison between PhoNet and PhoNet rand

139. Conclusion Covered  Understanding topological properties of complex networks  Case Study – peer to peer networks and language modeling Not covered  Processes on networks Search, Epidemics etc.

140. Contact Niloy Ganguly  niloy@cse.iitkgp.erent.in  http://www.facweb.iitkgp.ernet.in/~niloy/ http://www.facweb.iitkgp.ernet.in/~niloy/ Workshop  Dynamics on and of complex network http://www.cel.iitkgp.ernet.in/~eccs07/  Sequel : Book Volume to be published in May 2008 from Birkhauser Boston. Thank you!!