1. Node Similarity Ralucca Gera,
Applied Mathematics Dept. Naval Postgraduate School Monterey, California

2. We talked about global properties We talked about local properties:
Motivation
We talked about global properties
Average degree, average clustering, ave path length
We talked about local properties:
Some node centralities
Next we’ll look at pairwise properties of nodes:
Node equivalence and similarity,
Correlation between pairs of nodes
Why care? How do items get suggested to users?
Based on past history of the user
Based on the similarity of the user with the other people

3. Similarity In complex network, one can measure similarity
Between vertices
Between networks
We consider the similarity between vertices in the same network. Why and how can this similarity be quantified?
Differentiating vertices helps tease apart the types and relationships of vertices
Useful for “click here for pages/movies/books similar to this one”
We determine similarities based on the network structure
There are two types of similarities:
Structural equivalence (main one is the Pearson Corr. Coeff)
Regular equivalence

4. Structural vs. Regular Equivalent nodes
Defn: Two vertices are structurally equivalent if they share many neighbors.
Defn: Two vertices are regularly equivalent if they have many neighbors that are also equivalent. 𝑎 𝑏
Such as two math students 𝑎 and 𝑏
that know the same professors 𝑎 𝑏
Such as two Deans 𝑎 and 𝑏 that have similar ties:
provost, president, department chairs (some could actually be common)

5. Structural Equivalence

6. Structural Equivalent nodes
Two vertices are structurally equivalent if they share many neighbors
How can we measure this similarity? 𝑎 𝑏

7. Structural similarity 𝑛 𝑖𝑗
1. Count of common neighbors 𝑛 𝑖𝑗 :
𝑛 𝑖𝑗 =|𝑁[𝑖]∩𝑁[𝑗]| = 𝑘 a 𝑖𝑘 a 𝑘𝑗 = a 2 𝑖𝑗
2. Normalized common neighbors (divide by a constant):
by the number of vertices: 𝑛 𝑖𝑗 |𝑉(𝐺)|
by the max number of common neighbors 𝑛 𝑖𝑗 𝑉 𝐺 −2
Jaccard similarity is the quotient of number of common to the union of all neighbors:
𝐽 𝑖𝑗 = |𝑁 𝑖 ∩𝑁 𝑗 | |𝑁 𝑖 ∪ 𝑁 𝑗 | = 𝑛 𝑖𝑗 |𝑁 𝑖 ∪ 𝑁 𝑗 |
Fast to compute (thus popular) as it just counts neighbors

8. Structural similarity (2)
3. Cosine similarity (divide by variable):
Let x be the row corresponding to vertex i in A, and
Let y be the column corresponding to vertex j in A:
cos α= 𝒙 ∙𝒚 𝒙 ∙|𝒚| = 𝑘 a 𝑖𝑘 a 𝑘𝑗 𝑘 a 𝑖𝑘 2 ∙ 𝑘 a 𝑗𝑘 = ? ? simplifies to
= 𝑛 𝑖𝑗 deg 𝑖 deg 𝑗 = 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑜𝑚𝑚𝑜𝑛 𝑛𝑒𝑖𝑔ℎ𝑏𝑜𝑟𝑠 deg 𝑖 deg 𝑗
Convention: If deg i = 0 (denominator is 0), then cos 𝛼 = 0 (orthogonal vectors). Range: 0≤ cos α≤1

9. Cosine similarity is an example of a technique used in
information retrieval,
text analysis, or
any comparison of 𝑥 to 𝑦, where each of 𝑥 and 𝑦 can be vectorized based on their components
For example: To find the closest document(s)/website(s) to another document or a query…

10. Technique used in text analysis
A collection of n documents ( 𝐷 1 , 𝐷 2 ,…, 𝐷 𝑛 ) can be represented in the vector space model by a term-document matrix.
A list of t terms of interest: 𝑇 1 , 𝑇 2 , …, 𝑇 𝑡
An entry in the matrix corresponds to the “weight” of a term in the document; zero means the term has no significance in the document or it simply doesn’t exist in the document.
T1 T2 … Tt
D1 w11 w21 … wt1
D2 w12 w22 … wt2
: : : :
Dn w1n w2n … wtn
Source: Raymond J. Mooney, University of Texas at Austin

11. Graphic Representation
Example: Given terms T2, T2, T2
Documents D1, D1 and Query Q:
D1 = 2T1 + 3T2 + 5T3
D2 = 3T1 + 7T2 + T3
Q = 0T1 + 0T2 + 2T3 T3 T1 T2
D1 = 2T1+ 3T2 + 5T3
D2 = 3T1 + 7T2 + T3
Q = 0T1 + 0T2 + 2T3 7 3 2 5
Is D1 or D2 more similar to Q?
How to measure the degree of similarity? Distance? Angle? Projection?
Source: Raymond J. Mooney, University of Texas at Austin

12. Cosine Similarity Measure2 t3 t1 t2 D1 D2 Q 1
Cosine similarity measures the cosine of the angle between two vectors.
Inner product normalized by the vector lengths.
CosSim(dj, q) =
D1 = 2T1 + 3T2 + 5T3 CosSim(D1 , Q) = 10 / (4+9+25)(0+0+4) = 0.81
D2 = 3T1 + 7T2 + 1T3 CosSim(D2 , Q) = 2 / (9+49+1)(0+0+4) = 0.13
Q = 0T1 + 0T2 + 2T3
D1 is 6 times better than D2 using cosine similarity
Source: Raymond J. Mooney, University of Texas at Austin

13. Illustration of 3 Nearest Neighbor for Text
It works the same with vectors
of the adjacency matrix, where
the terms are just the neighbors, and similarity is how many common neighbors there are
query
Source: Raymond J. Mooney, University of Texas at Austin

14. So far: Structural similarity 𝑛 𝑖𝑗
1. Count of common neighbors:
𝑛 𝑖𝑗 =|𝑁[𝑖]∩𝑁[𝑗]| = 𝑘 a 𝑖𝑘 a 𝑘𝑗 = a 2 𝑖𝑗
2. Normalized common neighbors (divide by the):
number of vertices: 𝑛 𝑖𝑗 |𝑉(𝐺)| = a 2 𝑖𝑗 |𝑉(𝐺)|
max count of common neighbors possible 𝑛 𝑖𝑗 𝑉 𝐺 −2 = a 2 𝑖𝑗 𝑉 𝐺 −2
exact count of common neighbors (Jaccard similarity)
𝐽 𝑖𝑗 = 𝑛 𝑖𝑗 |𝑁 𝑖 ∪ 𝑁 𝑗 | = a 2 𝑖𝑗 |𝑁 𝑖 ∪ 𝑁 𝑗 |
3. Cosine similarity: 𝑛 𝑖𝑗 deg 𝑖 deg 𝑗 = a 2 𝑖𝑗 deg 𝑖 deg 𝑗
A new one!

15. (Degree) Pearson Correlation Coeff (1)
4. Degree Pearson Correlation Coefficient: compare to the expected value of common neighbors (i.e. to a network in which vertices choose their neighbors at random). Here is the derivation:
Let i and j be two vertices of deg i and deg j
The probability of j to choose at random one of i’s neighbors is 𝑑𝑒𝑔 𝑖 𝑛−1 (or 𝑑𝑒𝑔 𝑖 𝑛 if loops in G)
Thus the probability that all of j’s neighbors are neighbors of i is 𝑑𝑒𝑔 𝑖 ∙ deg 𝑗 𝑛−1

16. Pearson Correlation Coeff (2)
4. Degree Pearson Correlation Coefficient (continued) is the number of common neighbors minus the expected common neighbors (that we will normalize):
𝑘 a 𝑖𝑘 a 𝑗𝑘 − deg 𝑖 deg 𝑗 𝑛 = 𝑘 a 𝑖𝑘 a 𝑗𝑘 − 1 𝑛 𝑘 a 𝑖𝑘 𝑙 a 𝑗𝑙 = 𝑘 a 𝑖𝑘 a 𝑗𝑘 − 1 𝑛 𝑛< a 𝑖 >𝑛 < a 𝑗 >
= 𝑘 a 𝑖𝑘 a 𝑗𝑘 − 𝑘=1 𝑛 < a 𝑖 >< a 𝑗 >
= 𝑘 [ a 𝑖𝑘 a 𝑗𝑘 − < a 𝑖 >< a 𝑗 >]
= 𝑘 [ a 𝑖𝑘 − < a 𝑖 >][ a 𝑗𝑘 − < a 𝑗 >]
Since the mean of the 𝑖 𝑡ℎ row is < a 𝑖 > = 𝑘 a 𝑖𝑘 𝑛
Since 𝑘 1 =𝑛 𝑎𝑠
< a 𝑖 >< a 𝑗 > are constants with respect to k
FOIL and simplify using 𝑘 <a 𝑖 > a 𝑗𝑘 = 𝑛<a 𝑖 > <a 𝑗 > = 𝑘 <a 𝑗 > a 𝑖𝑘

17. Pearson Correlation Coeff (3)
4. Degree Pearson Correlation Coefficient (continued) is the actually number of common neighbors minus the expected common neighbors (from previous page):
𝒄𝒐𝒗 𝒂 𝒊 , 𝒂 𝒋 = 𝒌 𝒂 𝒊𝒌 𝒂 𝒋𝒌 − 𝐝𝐞𝐠 𝒊 𝐝𝐞𝐠 𝒋 𝒏 = 𝒌 [ 𝒂 𝒊𝒌 − < 𝒂 𝒊 >][ 𝒂 𝒋𝒌 − < 𝒂 𝒋 >]
that we now normalize:
𝑘 a 𝑖𝑘 a 𝑗𝑘 − deg 𝑖 deg 𝑗 𝑛 = 0 if the # of common neighbors is what is expected by chance
𝑘 a 𝑖𝑘 a 𝑗𝑘 − deg 𝑖 deg 𝑗 𝑛 < 0 if the # of common neighbors is less than what is expected by chance
𝑘 a 𝑖𝑘 a 𝑗𝑘 − deg 𝑖 deg 𝑗 𝑛 > 0 if the # of common neighbors is more than what is expected by chance

18. Pearson Correlation Coeff (4)
4. Degree Pearson Correlation Coefficient (continued) is the actually number of common neighbors minus the expected common neighbors (normalized by row a 𝑖 and a 𝑗 ):
𝑟 𝑖𝑗 = 𝑐𝑜𝑣( a 𝑖 , a 𝑗 ) 𝑣𝑎𝑟( a 𝑖 ) 2 = 𝑐𝑜𝑣( a 𝑖 , a 𝑗 ) 𝑣𝑎𝑟 a 𝑖 𝑣𝑎𝑟( a 𝑖 ) = 𝑘 [ a 𝑖𝑘 − < a 𝑖 >][ a 𝑗𝑘 − < a 𝑗 >] 𝑘 [ a 𝑖𝑘 − < a 𝑖 >] 𝑘 [ a 𝑗𝑘 − < a 𝑗 >] 2
𝑟 𝑖𝑗 = 0 if the # of common neighbors is what is expected by chance
−1< 𝑟 𝑖𝑗 < 0 if the # of common neighbors is less than what is expected by chance
0< 𝑟 𝑖𝑗 < 1 if the # of common neighbors is more than what is expected by chance
𝑟 𝑖𝑗 shows more/less neighbor similarity in a network, compared to vertices expected to be neighbors at random

19. More measures
There are more measures for structural equivalences, the Pearson Correlation Coefficient is commonly used
Another one is Euclidean distance which is the cardinality of the symmetric difference between the neighbors of i and neighbors of j
𝑑 𝑖𝑗 = 𝑘 ( a 𝑖𝑘 − a 𝑗𝑘 )( a 𝑗𝑘 − a 𝑖𝑘 ) = 𝑘 ( a 𝑖𝑘 − a 𝑗𝑘 ) 2
All these are measures of structural equivalence of a network: they measure different ways of counting common neighbors

20. Overview updated: Structural similarity
1. Count of common neighbors:
𝑛 𝑖𝑗 =|𝑁[𝑖]∩𝑁[𝑗]| = 𝑘 a 𝑖𝑘 a 𝑘𝑗 = a 2 𝑖𝑗
2. Normalized common neighbors (divide by the):
number of vertices: 𝑛 𝑖𝑗 |𝑉(𝐺)| = a 2 𝑖𝑗 |𝑉(𝐺)|
max count of common neighbors possible 𝑛 𝑖𝑗 𝑉 𝐺 −2 = 𝑛 𝑖𝑗 𝑉 𝐺 −2
exact count of common neighbors (Jaccard similarity)
𝐽 𝑖𝑗 = 𝑛 𝑖𝑗 |𝑁 𝑖 ∪ 𝑁 𝑗 | = a 2 𝑖𝑗 |𝑁 𝑖 ∪ 𝑁 𝑗 |
3. Cosine similarity: 𝑛 𝑖𝑗 deg 𝑖 deg 𝑗 = a 2 𝑖𝑗 deg 𝑖 deg 𝑗
4. Degree Pearson Correlation Coefficient (coded in NetworkX) :
𝑟 𝑖𝑗 = 𝑐𝑜𝑣( a 𝑖 , a 𝑗 ) 𝑣𝑎𝑟( a 𝑖 ) 2 = 𝑐𝑜𝑣( a 𝑖 , a 𝑗 ) 𝑣𝑎𝑟 a 𝑖 𝑣𝑎𝑟( a 𝑖 ) = 𝑘 [ a 𝑖𝑘 − < a 𝑖 >][ a 𝑗𝑘 − < a 𝑗 >] 𝑘 [ a 𝑖𝑘 − < a 𝑖 >] 𝑘 [ a 𝑗𝑘 − < a 𝑗 >] 2

21. Regular Equivalence

22. Structural vs. Regular Equivalent nodes
Defn: Two vertices are structurally equivalent if they share many neighbors.
Defn: Two vertices are regularly equivalent if they have many neighbors that are also equivalent. 𝑎 𝑏
Such as two math students 𝑎 and 𝑏
that know the same professors 𝑎 𝑏
Such as two Deans 𝑎 and 𝑏 that have similar ties:
provost, president, department chairs (some could actually be common)

23. Defn of regular Equivalence
Defn. Two nodes are regularly equivalent if they are equally related to equivalent nodes
(i.e. their neighbors need to be equivalent as well).
We capture equivalence through color pattern adjacencies.
To have similar roles, these nodes need different colors because of their adjacencies
White and Reitz, 1983; Everette and Borgatti, 1991

24. Structural vs. Regular equivalent color classes
Structural: Nodes with connectivity to the same nodes (common information about the network)
vs. Regular: Nodes with similar roles in the network (similar positions in the network).
Leonid E. Zhukov

25. Regular Equivalence (1)
Regular equivalences: nodes don’t have to be adjacent to be regular equivalent
REGE – an algorithm developed in 1970 to discover regular equivalences, “but the operations are very involved and it is not easy to interpret” (Newman)
Newer methods: METHOD 1: develop σ 𝑖𝑗 that is high if the neighbors k and 𝑙 of i and j are (regularly) equivalent
σ 𝑖𝑗 = α 𝑘𝑙 a 𝑖𝑘 a 𝑗𝑙 σ 𝑘𝑙
A recurrence relation in terms of σ, much like eigenvector.
σ 𝑖𝑗 counts how many common neighbors are there, times their neighbors’ regular similarity σ 𝑘𝑙

26. Regular Equivalence (2)
σ 𝑖𝑗 = α 𝑘𝑙 a 𝑖𝑘 a 𝑗𝑙 σ 𝑘𝑙
Is equivalent to σ = α 𝐴σ𝐴 ,
where α is the leading eigenvalue of A, and σ is the eigenvector.
But:
It may not give high self-similarity σ 𝑖𝑖 as it heavily depends on the similarity σ 𝑘𝑙 of neighbors of i
It doesn’t give high similarity to pairs of vertices with lots of common neighbors (because it looks at values of 𝑘 and 𝑙, including 𝑘 ≠𝑙)
Needs to be altered, and we’ll see two methods next.

27. Regular Equivalence (3)
MEHTOD 1: σ 𝑖𝑗 = α 𝑘𝑙 a 𝑖𝑘 a 𝑗𝑙 σ 𝑘𝑙 + δ 𝑖𝑗
Is equivalent to
σ = α 𝐴σ𝐴+𝐼, with σ (𝑡=0) = 0 , δ 𝑖𝑗 = 1, if i=j 0, if i ≠j.
Calculate by starting with the initial values:
σ (𝑡=1) =α 𝐴 0 𝐴+𝐼=𝐼,
σ (𝑡=2) =α 𝐴𝐼𝐴+𝐼 = α 𝐴 2 +𝐼,
σ (𝑡=3) =α𝐴 α 𝐴 2 +𝐼 𝐴+𝐼= α 𝐴 4 +α 𝐴 2 +𝐼,
Note: σ 𝑖𝑗 is a weighted some of the even powers of A (walks of even length), but we want all the lengths…

28. Regular Equivalence (4)
METHOD 2: define σ 𝑖𝑗 to be high if vertex j has a neighbor k that is similar to j
σ 𝑖𝑗 = α 𝑘 a 𝑖𝑘 σ 𝑘𝑗 + δ 𝑖𝑗
Is equivalent to
σ = α 𝐴σ+𝐼, with σ (𝑡=0) = 0
with:
σ (𝑡=1) =α 𝐴 0 +𝐼=𝐼,
σ (𝑡=2) =α 𝐴𝐼+𝐼 = α 𝐴+𝐼,
σ (𝑡=3) =α𝐴 α 𝐴+𝐼 +𝐼= α 2 𝐴 2 +α𝐴+𝐼,
σ 𝑖𝑗 is a weighted ( α ? ) count of all walks between i and j, (with α < 1 so longer paths weigh less)
Leicht, Elizabeth A., Petter Holme, and Mark EJ Newman. "Vertex similarity in networks." Physical Review E 73.2 (2006):

29. Regular Equivalence (5)
METHOD 2: define σ 𝑖𝑗 to be high if vertex j has a neighbor k that is similar to j
σ 𝑖𝑗 = α 𝑘 a 𝑖𝑘 σ 𝑘𝑗 + δ 𝑖𝑗
Is equivalent to
σ = α 𝐴σ+𝐼, with σ (𝑡=0) = 0
This looks a lot like Katz centrality (recall 𝑥 𝑖 =α 𝑗 a 𝑖𝑗 𝑥 𝑗 + β, where β is a constant initial weight given to each vertex so that its out degree matters) Katz centrality of vertex i is the sum of σ 𝑖𝑗 , ∀𝑗∈𝑁(𝑖)

30. Regular Equivalence (6)
The regular equivalence σ 𝑖𝑗 tends to be high for vertices of high degree (more chances of their neighbors being similar)
However, some low degree vertices could be similar as well, so the formula can be modified similar to PageRank if desired:
σ 𝑖𝑗 = α 𝐝𝐞𝐠 𝒊 𝑘 a 𝑖𝑘 σ 𝑘𝑗 + δ 𝑖𝑗
which is equivalent to
σ = α 𝐷 −1 𝐴σ+𝐼,

31. The connection between
structural and regular equivalence

32. Equivalence (1)
Recall that the regular equivalence σ 𝑖𝑗 is a weighted count of all walks between i and j, i.e:
σ (𝑡=3) =α𝐴 α 𝐴+𝐼 +𝐼= α 2 𝐴 2 +α𝐴+𝐼,
Recall that the structural equivalence 𝑟 𝑖𝑗 attempts to count the # of common neighbors of i and j in different ways
What is the correlation between the number of common neighbors of 4 and 5, and paths of length 2 between 4 and 5?

33. So the regular equivalence σ 𝑖𝑗 is a generalization of
Recall that the regular equivalence σ 𝑖𝑗 is a weighted count of all walks between i and j, i.e:
σ (𝑡=3) =α𝐴 α 𝐴+𝐼 +𝐼= α 2 𝐴 2 +α𝐴+𝐼,
Recall that the structural equivalence 𝑟 𝑖𝑗 attempts to count the # of common neighbors in different ways
So the regular equivalence σ 𝑖𝑗
is a generalization of
structural equivalence 𝑟 𝑖𝑗
( 𝑟 𝑖𝑗 counts just paths of length 2)

34. References
Newman, Mark. Networks: An introduction. Oxford university press, 2010.
Everett, Martin G., and Steve Borgatti. "Role colouring a graph." Mathematical Social Sciences 21.2 (1991):
White, Douglas R., and Karl P. Reitz. "Graph and semigroup homomorphisms on networks of relations." Social Networks 5.2 (1983):
Leicht, Elizabeth A., Petter Holme, and Mark EJ Newman. "Vertex similarity in networks." Physical Review E 73.2 (2006):