Section 7.12: Similarity By: Ralucca Gera, NPS

We talked about global properties We talked about local properties: Motivation We talked about global properties Average degree, average clustering, ave path length We talked about local properties: Some node centralities Next we’ll look at pairwise properties of nodes: Node equivalence and similarity, Correlation between pairs of nodes Why care? How do items get suggested to users? Based on past history of the user Based on the similarity of the user with the other people

Similarity In complex network, one can measure similarity Between vertices Between networks We consider the similarity between vertices in the same network. Why and how can this similarity be quantified? Differentiating vertices helps tease apart the types and relationships of vertices Useful for “click here for pages/movies/books similar to this one” We determine similarities based on the network structure There are two types of similarities: Structural equivalence (main one is the Pearson Corr. Coeff) Regular equivalence

Structural vs. Regular Equivalent nodes Defn: Two vertices are structurally equivalent if they share many neighbors. Defn: Two vertices are regularly equivalent if they have many neighbors that are also equivalent. 𝑎 𝑏 Such as two math students 𝑎 and 𝑏 that know the same professors 𝑎 𝑏 Such as two Deans 𝑎 and 𝑏 that have similar ties: provost, president, department chairs (some could actually be common)

Structural Equivalence

Structural Equivalent nodes Two vertices are structurally equivalent if they share many neighbors How can we measure this similarity? 𝑎 𝑏

Structural similarity 𝑛 𝑖𝑗 1. Count of common neighbors 𝑛 𝑖𝑗 : 𝑛 𝑖𝑗 =|𝑁[𝑖]∩𝑁[𝑗]| = 𝑘 𝐴 𝑖𝑘 𝐴 𝑘𝑗 = 𝐴 2 𝑖𝑗 2. Normalized common neighbors (divide by constant): by the number of vertices: 𝑛 𝑖𝑗 |𝑉(𝐺)| by the max number of common neighbors 𝑛 𝑖𝑗 𝑉 𝐺 −2 Jaccard similarity is the quotient of number of common to the union of all neighbors: 𝐽 𝑖𝑗 = |𝑁 𝑖 ∩𝑁 𝑗 | |𝑁 𝑖 ∪ 𝑁 𝑗 | = 𝑛 𝑖𝑗 |𝑁 𝑖 ∪ 𝑁 𝑗 | Fast to compute (thus popular) as it just counts neighbors

Structural similarity (2) 3. Cosine similarity (divide by variable): Let x be the row corresponding to vertex i in A, and Let y be the column corresponding to vertex j in A: cos α= 𝒙 ∙𝒚 𝒙 ∙|𝒚| = 𝑘 𝐴 𝑖𝑘 𝐴 𝑘𝑗 𝑘 𝐴 𝑖𝑘 2 ∙ 𝑘 𝐴 𝑗𝑘 2 = ? ? simplifies to = 𝑛 𝑖𝑗 deg 𝑖 deg 𝑗 = 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑜𝑚𝑚𝑜𝑛 𝑛𝑒𝑖𝑔ℎ𝑏𝑜𝑟𝑠 deg 𝑖 deg 𝑗 Convention: If deg i = 0 (denominator is 0), then cos 𝛼 = 0. Range: 0≤ cos α≤1

Cosine similarity is an example of a technique used in information retrieval, text analysis, or any comparison of 𝑥 to 𝑦, where each of 𝑥 and 𝑦 can be vectorized based on their components For example: To find the closest document(s)/website(s) to another document or a query…

Technique used in text analysis A collection of n documents ( 𝐷 1 , 𝐷 2 ,…, 𝐷 𝑛 ) can be represented in the vector space model by a term-document matrix. A list of t terms of interest: 𝑇 1 , 𝑇 2 , …, 𝑇 𝑡 An entry in the matrix corresponds to the “weight” of a term in the document; zero means the term has no significance in the document or it simply doesn’t exist in the document. T1 T2 …. Tt D1 w11 w21 … wt1 D2 w12 w22 … wt2 : : : : Dn w1n w2n … wtn Source: Raymond J. Mooney University of Texas at Austin

Graphic Representation Example: D1 = 2T1 + 3T2 + 5T3 D2 = 3T1 + 7T2 + T3 Q = 0T1 + 0T2 + 2T3 T3 T1 T2 D1 = 2T1+ 3T2 + 5T3 D2 = 3T1 + 7T2 + T3 Q = 0T1 + 0T2 + 2T3 7 3 2 5 Is D1 or D2 more similar to Q? How to measure the degree of similarity? Distance? Angle? Projection? Source: Raymond J. Mooney University of Texas at Austin

Cosine Similarity Measure 2 t3 t1 t2 D1 D2 Q 1 Cosine similarity measures the cosine of the angle between two vectors. Inner product normalized by the vector lengths. CosSim(dj, q) = D1 = 2T1 + 3T2 + 5T3 CosSim(D1 , Q) = 10 / (4+9+25)(0+0+4) = 0.81 D2 = 3T1 + 7T2 + 1T3 CosSim(D2 , Q) = 2 / (9+49+1)(0+0+4) = 0.13 Q = 0T1 + 0T2 + 2T3 D1 is 6 times better than D2 using cosine similarity Source: Raymond J. Mooney University of Texas at Austin

Illustration of 3 Nearest Neighbor for Text It works the same with vectors of the adjacency matrix, where the terms are just the neighbors, and similarity is how many common neighbors there are query Source: Raymond J. Mooney University of Texas at Austin

So far: Structural similarity 𝑛 𝑖𝑗 1. Count of common neighbors: 𝑛 𝑖𝑗 =|𝑁[𝑖]∩𝑁[𝑗]| = 𝑘 𝐴 𝑖𝑘 𝐴 𝑘𝑗 = 𝐴 2 𝑖𝑗 2. Normalized common neighbors (divide by the): number of vertices: 𝑛 𝑖𝑗 |𝑉(𝐺)| = 𝐴 2 𝑖𝑗 |𝑉(𝐺)| max count of common neighbors possible 𝑛 𝑖𝑗 𝑉 𝐺 −2 = 𝐴 2 𝑖𝑗 𝑉 𝐺 −2 exact count of common neighbors (Jaccard similarity) 𝐽 𝑖𝑗 = 𝑛 𝑖𝑗 |𝑁 𝑖 ∪ 𝑁 𝑗 | = 𝐴 2 𝑖𝑗 |𝑁 𝑖 ∪ 𝑁 𝑗 | 3. Cosine similarity: 𝑛 𝑖𝑗 deg 𝑖 deg 𝑗 = 𝐴 2 𝑖𝑗 deg 𝑖 deg 𝑗

Pearson Correlation Coeff (1) 4. Pearson Correlation Coefficient (compare to the expected value of common neighbors in a network in which vertices choose their neighbors at random): Let i and j be two vertices of deg i and deg j The probability of j to choose at random one of i’s neighbors is 𝑑𝑒𝑔 𝑖 𝑛−1 (or 𝑑𝑒𝑔 𝑖 𝑛 if loops in G) Thus the probability that all of j’s neighbors are neighbors of i is 𝑑𝑒𝑔 𝑖 ∙ deg 𝑗 𝑛−1

Pearson Correlation Coeff (2) 4. Pearson Correlation Coefficient (continued) is the actual number of common neighbors minus the expected common neighbors that will be normalized: 𝑘 𝐴 𝑖𝑘 𝐴 𝑗𝑘 − deg 𝑖 deg 𝑗 𝑛 = 𝑘 𝐴 𝑖𝑘 𝐴 𝑗𝑘 − 1 𝑛 𝑘 𝐴 𝑖𝑘 𝑙 𝐴 𝑗𝑙 = 𝑘 𝐴 𝑖𝑘 𝐴 𝑗𝑘 − 1 𝑛 𝑛< 𝐴 𝑖 >𝑛 < 𝐴 𝑗 > = 𝑘 𝐴 𝑖𝑘 𝐴 𝑗𝑘 − 𝑘=1 𝑛 < 𝐴 𝑖 >< 𝐴 𝑗 > = 𝑘 [ 𝐴 𝑖𝑘 𝐴 𝑗𝑘 − < 𝐴 𝑖 >< 𝐴 𝑗 >] = 𝑘 [ 𝐴 𝑖𝑘 − < 𝐴 𝑖 >][ 𝐴 𝑗𝑘 − < 𝐴 𝑗 >] Since the mean of the 𝑖 𝑡ℎ row is < 𝐴 𝑖 > = 𝑘 𝐴 𝑖𝑘 𝑛 Since 𝑘 1 =𝑛 𝑎𝑠 < 𝐴 𝑖 >< 𝐴 𝑗 > are constants with respect to k FOIL and simplify using 𝑘 <𝐴 𝑖 > 𝐴 𝑗𝑘 = 𝑛<𝐴 𝑖 > <𝐴 𝑗 > = 𝑘 <𝐴 𝑗 > 𝐴 𝑖𝑘

Pearson Correlation Coeff (3) 4. Pearson Correlation Coefficient (continued) is the actually number of common neighbors minus the expected common neighbors that will be normalized: 𝒄𝒐𝒗 𝑨 𝒊 , 𝑨 𝒋 = 𝒌 𝑨 𝒊𝒌 𝑨 𝒋𝒌 − 𝐝𝐞𝐠 𝒊 𝐝𝐞𝐠 𝒋 𝒏 = 𝒌 [ 𝑨 𝒊𝒌 − < 𝑨 𝒊 >][ 𝑨 𝒋𝒌 − < 𝑨 𝒋 >] 𝑘 𝐴 𝑖𝑘 𝐴 𝑗𝑘 − deg 𝑖 deg 𝑗 𝑛 = 0 if the # of common neighbors is what is expected by chance 𝑘 𝐴 𝑖𝑘 𝐴 𝑗𝑘 − deg 𝑖 deg 𝑗 𝑛 < 0 if the # of common neighbors is less than what is expected by chance 𝑘 𝐴 𝑖𝑘 𝐴 𝑗𝑘 − deg 𝑖 deg 𝑗 𝑛 > 0 if the # of common neighbors is more than what is expected by chance

Pearson Correlation Coeff (4) 4. Pearson Correlation Coefficient (continued) is the actually number of common neighbors minus the expected common neighbors normalized (by row 𝐴 𝑖 and 𝐴 𝑗 ): 𝑟 𝑖𝑗 = 𝑐𝑜𝑣( 𝐴 𝑖 , 𝐴 𝑗 ) 𝑣𝑎𝑟( 𝐴 𝑖 ) 2 = 𝑐𝑜𝑣( 𝐴 𝑖 , 𝐴 𝑗 ) 𝑣𝑎𝑟 𝐴 𝑖 𝑣𝑎𝑟( 𝐴 𝑖 ) = 𝑘 [ 𝐴 𝑖𝑘 − < 𝐴 𝑖 >][ 𝐴 𝑗𝑘 − < 𝐴 𝑗 >] 𝑘 [ 𝐴 𝑖𝑘 − < 𝐴 𝑖 >] 2 𝑘 [ 𝐴 𝑗𝑘 − < 𝐴 𝑗 >] 2 𝑟 𝑖𝑗 = 0 if the # of common neighbors is what is expected by chance −1< 𝑟 𝑖𝑗 < 0 if the # of common neighbors is less than what is expected by chance 0< 𝑟 𝑖𝑗 < 1 if the # of common neighbors is more than what is expected by chance 𝑟 𝑖𝑗 shows more/less vertex similarity in a network compared to vertices expected to be neighbors at random

More measures There are more measures for structural equivalences, the Pearson Correlation Coefficient is commonly used Another one is Euclidean distance which is the cardinality of the symmetric difference between the neighbors of i and neighbors of j 𝑑 𝑖𝑗 = 𝑘 ( 𝐴 𝑖𝑘 − 𝐴 𝑗𝑘 )( 𝐴 𝑗𝑘 − 𝐴 𝑖𝑘 ) = 𝑘 ( 𝐴 𝑖𝑘 − 𝐴 𝑗𝑘 ) 2 All these are measures of structural equivalence of a network: they measure different ways of counting common neighbors

Overview updated: Structural similarity 1. Count of common neighbors: 𝑛 𝑖𝑗 =|𝑁[𝑖]∩𝑁[𝑗]| = 𝑘 𝐴 𝑖𝑘 𝐴 𝑘𝑗 = 𝐴 2 𝑖𝑗 2. Normalized common neighbors (divide by the): number of vertices: 𝑛 𝑖𝑗 |𝑉(𝐺)| = 𝐴 2 𝑖𝑗 |𝑉(𝐺)| max count of common neighbors possible 𝑛 𝑖𝑗 𝑉 𝐺 −2 = 𝑛 𝑖𝑗 𝑉 𝐺 −2 exact count of common neighbors (Jaccard similarity) 𝐽 𝑖𝑗 = 𝑛 𝑖𝑗 |𝑁 𝑖 ∪ 𝑁 𝑗 | = 𝐴 2 𝑖𝑗 |𝑁 𝑖 ∪ 𝑁 𝑗 | 3. Cosine similarity: 𝑛 𝑖𝑗 deg 𝑖 deg 𝑗 = 𝐴 2 𝑖𝑗 deg 𝑖 deg 𝑗 4. Pearson Correlation Coefficient (exists in NetworkX) : 𝑟 𝑖𝑗 = 𝑐𝑜𝑣( 𝐴 𝑖 , 𝐴 𝑗 ) 𝑣𝑎𝑟( 𝐴 𝑖 ) 2 = 𝑐𝑜𝑣( 𝐴 𝑖 , 𝐴 𝑗 ) 𝑣𝑎𝑟 𝐴 𝑖 𝑣𝑎𝑟( 𝐴 𝑖 ) = 𝑘 [ 𝐴 𝑖𝑘 − < 𝐴 𝑖 >][ 𝐴 𝑗𝑘 − < 𝐴 𝑗 >] 𝑘 [ 𝐴 𝑖𝑘 − < 𝐴 𝑖 >] 2 𝑘 [ 𝐴 𝑗𝑘 − < 𝐴 𝑗 >] 2

Regular Equivalence

Structural vs. Regular Equivalent nodes Defn: Two vertices are structurally equivalent if they share many neighbors. Defn: Two vertices are regularly equivalent if they have many neighbors that are also equivalent. 𝑎 𝑏 Such as two math students 𝑎 and 𝑏 that know the same professors 𝑎 𝑏 Such as two Deans 𝑎 and 𝑏 that have similar ties: provost, president, department chairs (some could actually be common)

Defn of regular Equivalence Defn. Two nodes are regularly equivalent if they are equally related to equivalent nodes (i.e. their neighbors need to be equivalent as well). We capture equivalence through color pattern adjacencies. To have similar roles, these nodes need different colors because of their adjacencies White and Reitz, 1983; Everette and Borgatti, 1991

Structural vs. Regular equivalent color classes Nodes with connectivity to the same nodes (common information about the network) vs. Nodes with similar roles in the network (similar positions in the network). Leonid E. Zhukov

Regular Equivalence (1) Regular equivalences: nodes don’t have to be adjacent to be regular equivalent REGE – an algorithm developed in 1970 to discover regular equivalences, “but the operations are very involved and it is not easy to interpret” (Newman) Newer methods: METHOD 1: develop σ 𝑖𝑗 that is high if the neighbors k and 𝑙 of i and j are (regularly) equivalent σ 𝑖𝑗 = α 𝑘𝑙 𝐴 𝑖𝑘 𝐴 𝑗𝑙 σ 𝑘𝑙 A recurrence relation in terms of σ, much like eigenvector. σ 𝑖𝑗 counts how many common neighbors are there, times their neighbors’ regular similarity σ 𝑘𝑙

Regular Equivalence (2) σ 𝑖𝑗 = α 𝑘𝑙 𝐴 𝑖𝑘 𝐴 𝑗𝑙 σ 𝑘𝑙 Is equivalent to σ = α 𝐴σ𝐴 , where α is the leading eigenvalue of A, and σ is the eigenvector. But: It may not give high self-similarity σ 𝑖𝑖 as it heavily depends on the similarity σ 𝑘𝑙 of neighbors of i It doesn’t give high similarity to pairs of vertices with lots of common neighbors (because it looks at values of 𝑘 and 𝑙, including 𝑘 ≠𝑙) Needs to be altered, and we’ll see two methods next.

Regular Equivalence (3) MEHTOD 1: σ 𝑖𝑗 = α 𝑘𝑙 𝐴 𝑖𝑘 𝐴 𝑗𝑙 σ 𝑘𝑙 + δ 𝑖𝑗 Is equivalent to σ = α 𝐴σ𝐴+𝐼, with σ (𝑡=0) = 0 , δ 𝑖𝑗 = 1, if i=j 0, if i ≠j. Calculate by starting with the initial values: σ (𝑡=1) =α 𝐴 0 𝐴+𝐼=𝐼, σ (𝑡=2) =α 𝐴𝐼𝐴+𝐼 = α 𝐴 2 +𝐼, σ (𝑡=3) =α𝐴 α 𝐴 2 +𝐼 𝐴+𝐼= α 𝐴 4 +α 𝐴 2 +𝐼, Note: σ 𝑖𝑗 is a weighted some of the even powers of A (walks of even length), but we want all the lengths…

Regular Equivalence (4) METHOD 2: define σ 𝑖𝑗 to be high if vertex j has a neighbor k that is similar to j σ 𝑖𝑗 = α 𝑘 𝐴 𝑖𝑘 σ 𝑘𝑗 + δ 𝑖𝑗 Is equivalent to σ = α 𝐴σ+𝐼, with σ (𝑡=0) = 0 with: σ (𝑡=1) =α 𝐴 0 +𝐼=𝐼, σ (𝑡=2) =α 𝐴𝐼+𝐼 = α 𝐴+𝐼, σ (𝑡=3) =α𝐴 α 𝐴+𝐼 +𝐼= α 2 𝐴 2 +α𝐴+𝐼, σ 𝑖𝑗 is a weighted ( α ? ) count of all walks between i and j, (with α < 1 so longer paths weigh less) Leicht, Holme, and Newman, 2006

Regular Equivalence (5) METHOD 2: define σ 𝑖𝑗 to be high if vertex j has a neighbor k that is similar to j σ 𝑖𝑗 = α 𝑘 𝐴 𝑖𝑘 σ 𝑘𝑗 + δ 𝑖𝑗 Is equivalent to σ = α 𝐴σ+𝐼, with σ (𝑡=0) = 0 This looks a lot like Katz centrality (recall 𝑥 𝑖 =α 𝑗 𝐴 𝑖𝑗 𝑥 𝑗 + β, where β is a constant initial weight given to each vertex so that its out degree matters) Katz centrality of vertex i is the sum of σ 𝑖𝑗 , ∀𝑗∈𝑁(𝑖)

Regular Equivalence (6) The regular equivalence σ 𝑖𝑗 tends to be high for vertices of high degree (more chances of their neighbors being similar) However, some low degree vertices could be similar as well, so the formula can be modified if desired: σ 𝑖𝑗 = α 𝐝𝐞𝐠 𝒊 𝑘 𝐴 𝑖𝑘 σ 𝑘𝑗 + δ 𝑖𝑗 which is equivalent to σ = α 𝐷 −1 𝐴σ+𝐼,

The connection between structural and regular equivalence

Equivalence (1) Recall that the regular equivalence σ 𝑖𝑗 is a weighted count of all walks between i and j, i.e: σ (𝑡=3) =α𝐴 α 𝐴+𝐼 +𝐼= α 2 𝐴 2 +α𝐴+𝐼, Recall that the structural equivalence 𝑟 𝑖𝑗 attempts to count the # of common neighbors of i and j in different ways What is the correlation between the number of common neighbors of 4 and 5, and paths of length 2 between 4 and 5?

So the regular equivalence σ 𝑖𝑗 is a generalization of Recall that the regular equivalence σ 𝑖𝑗 is a weighted count of all walks between i and j, i.e: σ (𝑡=3) =α𝐴 α 𝐴+𝐼 +𝐼= α 2 𝐴 2 +α𝐴+𝐼, Recall that the structural equivalence 𝑟 𝑖𝑗 attempts to count the # of common neighbors in different ways So the regular equivalence σ 𝑖𝑗 is a generalization of structural equivalence 𝑟 𝑖𝑗 ( 𝑟 𝑖𝑗 counts just paths of length 2)