section 7.12: similarity - facultyfaculty.nps.edu/rgera/ma4404/winter2017/15-nodesimilarity.pdf ·...
TRANSCRIPT
![Page 1: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/1.jpg)
Section 7.12: Similarity
By: Ralucca Gera, NPS
![Page 2: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/2.jpg)
Motivation
• We talked about global properties– Average degree, average clustering, ave path length
• We talked about local properties:– Some node centralities
• Next we’ll look at pairwise properties of nodes:– Node equivalence and similarity, – Correlation between pairs of nodes– Why care? How do items get suggested to users?
• Based on past history of the user• Based on the similarity of the user with the other people
2
![Page 3: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/3.jpg)
Similarity
• In complex network, one can measure similarity– Between vertices– Between networks
• We consider the similarity between vertices in the same network. How can this similarity be quantified?
• Differentiating vertices helps tease apart the types and relationships of vertices– Useful for “click here for pages/movies/books similar to this
one”• We determine similarities based on the network structure• There are two types of similarities:
– Structural equivalence (main one is the Pearson Corr. Coeff)– Regular equivalence
3
![Page 4: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/4.jpg)
Structural vs. Regular Equivalent nodes
• Defn: Two vertices are structurally equivalent if they share many neighbors.
• Defn: Two vertices are regularly equivalent if they have many neighbors that are also equivalent.
4
Such as two math students andthat know the same professors
Such as two Deans and that have similar ties:provost, president, department chairs (some could actually be common)
![Page 5: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/5.jpg)
Structural Equivalence
![Page 6: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/6.jpg)
Structural Equivalent nodes
• Two vertices are structurally equivalent if they share many neighbors
• How can we measure this similarity?
6
![Page 7: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/7.jpg)
Structural similarity
1. Count of common neighbors=
2. Normalized common neighbors (divide by constant):
– by the number of vertices: | |
– by the max number of common neighbors
– Jaccard similarity is the quotient of number of common to the union of all neighbors:
Fast to compute (thus popular) as it just counts neighbors7
![Page 8: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/8.jpg)
Structural similarity (2)
3. Cosine similarity (divide by variable):Let x be the row corresponding to vertex i in A, andLet y be the column corresponding to vertex j in A:
· ·| |
∑
∑ · ∑= ??
simplifies to
=
=
Convention: If deg i = 0 (denominator is 0), then cos = 0.Range:
8
![Page 9: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/9.jpg)
Cosine similarity
• Cosine similarity is an example of a technique used in – information retrieval, – text analysis, or – any comparison of to , where each of and
can be vectorized based on their components • For example: To find the closest
document(s)/website(s) to another document or a query…
9
![Page 10: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/10.jpg)
10
Technique used in text analysis
• A collection of n documents ( , , … , ) can be represented in the vector space model by a term-document matrix.
• A list of t terms of interest: , , … ,• An entry in the matrix corresponds to the “weight” of a term in
the document; zero means the term has no significance in the document or it simply doesn’t exist in the document.
T1 T2 …. TtD1 w11 w21 … wt1D2 w12 w22 … wt2: : : :: : : :Dn w1n w2n … wtn
Source: Raymond J. MooneyUniversity of Texas at Austin
![Page 11: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/11.jpg)
11
Graphic Representation
Example:D1 = 2T1 + 3T2 + 5T3
D2 = 3T1 + 7T2 + T3
Q = 0T1 + 0T2 + 2T3
T3
T1
T2
D1 = 2T1+ 3T2 + 5T3
D2 = 3T1 + 7T2 + T3
Q = 0T1 + 0T2 + 2T3
7
32
5
• Is D1 or D2 more similar to Q?• How to measure the degree of
similarity? Distance? Angle? Projection?
Source: Raymond J. MooneyUniversity of Texas at Austin
![Page 12: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/12.jpg)
12
Cosine Similarity Measure
• Cosine similarity measures the cosine of the angle between two vectors.
• Inner product normalized by the vector lengths.
D1 = 2T1 + 3T2 + 5T3 CosSim(D1 , Q) = 10 / (4+9+25)(0+0+4) = 0.81D2 = 3T1 + 7T2 + 1T3 CosSim(D2 , Q) = 2 / (9+49+1)(0+0+4) = 0.13Q = 0T1 + 0T2 + 2T3
t3
t1
t2
D1
D2
Q
D1 is 6 times better than D2 using cosine similarity
t
i
t
i
t
i
ww
wwqdqd
iqij
iqij
j
j
1 1
22
1
)(
CosSim(dj, q) =
Source: Raymond J. MooneyUniversity of Texas at Austin
![Page 13: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/13.jpg)
13
Illustration of 3 Nearest Neighbor for Text
Source: Raymond J. MooneyUniversity of Texas at Austin
query
It works the same with vectorsof the adjacency matrix, wherethe terms are just the neighbors, and similarity is how many common neighbors there are
![Page 14: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/14.jpg)
So far: Structural similarity
1. Count of common neighbors:=
2. Normalized common neighbors (divide by the):
number of vertices: | | | |
max count of common neighbors possible
exact count of common neighbors (Jaccard similarity)
| ∪ | | ∪ |
3. Cosine similarity:
![Page 15: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/15.jpg)
Pearson Correlation Coeff (1)
4. Pearson Correlation Coefficient (compare to the expected value of common neighbors in a network in which vertices choose their neighbors at random):• Let i and j be two vertices of deg i and deg j• The probability of j to choose at random one of
i’s neighbors is (or if loops in G)
• Thus the probability that all of j’s neighbors are neighbors of i is ·
15
![Page 16: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/16.jpg)
Pearson Correlation Coeff (2)
4. Pearson Correlation Coefficient (continued) is the actual number of common neighbors minus the expected common neighbors that will be normalized:
=
=
== ] = ]
16
Since the mean of the row is < > = ∑
FOIL and simplify using ∑ ∑
Since ∑ 1 are
constants with respect to k
![Page 17: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/17.jpg)
Pearson Correlation Coeff (3)
4. Pearson Correlation Coefficient (continued) is the actually number of common neighbors minus the expected common neighbors that will be normalized:
, ∑ =∑ ]
• ∑ = 0 if the # of common neighbors is what is expected by chance
• ∑ < 0 if the # of common neighbors is lessthan what is expected by chance
• ∑ > 0 if the # of common neighbors is more than what is expected by chance
17
![Page 18: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/18.jpg)
Pearson Correlation Coeff (4)
4. Pearson Correlation Coefficient (continued) is the actually number of common neighbors minus the expected common neighbors normalized (by row and ):
=,
,
= ∑ ]
∑ ∑
• = 0 if the # of common neighbors is what is expected by chance• 1 < 0 if the # of common neighbors is less than what is
expected by chance• 0 < 1 if the # of common neighbors is more than what is
expected by chance
shows more/less vertex similarity in a network compared to vertices expected to be neighbors at random
![Page 19: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/19.jpg)
More measures
• There are more measures for structural equivalences, the Pearson Correlation Coefficient is commonly used
• Another one is Euclidean distance which is the cardinality of the symmetric difference between the neighbors of i and neighbors of j
All these are measures of structural equivalence of a network: they measure different ways of counting common neighbors 19
![Page 20: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/20.jpg)
Overview updated: Structural similarity
1. Count of common neighbors:| ∩ | = ∑
2. Normalized common neighbors (divide by the):
number of vertices: | | | |
max count of common neighbors possible
exact count of common neighbors (Jaccard similarity)
| ∪ | | ∪ |
3. Cosine similarity:
4. Pearson Correlation Coefficient (exists in NetworkX) :
= ,
,
= ∑ ]
∑ ∑
![Page 21: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/21.jpg)
Regular Equivalence
![Page 22: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/22.jpg)
Structural vs. Regular Equivalent nodes
• Defn: Two vertices are structurally equivalent if they share many neighbors.
• Defn: Two vertices are regularly equivalent if they have many neighbors that are also equivalent.
22
Such as two math students andthat know the same professors
Such as two Deans and that have similar ties:provost, president, department chairs (some could actually be common)
![Page 23: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/23.jpg)
Defn of regular Equivalence
23
Defn. Two nodes are regularly equivalent if they are equally related to equivalent nodes
(i.e. their neighbors need to be equivalent as well).
We capture equivalence through color pattern adjacencies.
White and Reitz, 1983; Everette and Borgatti, 1991
To have similar roles, these nodes need different colors because of their adjacencies
![Page 24: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/24.jpg)
Structural vs. Regular equivalent color classes
24Leonid E. Zhukov
Nodes with connectivity to the same nodes (common information about the network) vs. Nodes with similar roles in the network (similar positions in the network).
![Page 25: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/25.jpg)
Regular Equivalence (1)
• Regular equivalences: nodes don’t have to be adjacent to be regular equivalent
• REGE – an algorithm developed in 1970 to discover regular equivalences, “but the operations are very involved and it is not easy to interpret” (Newman)
• Newer methods: METHOD 1: develop that is high if the neighbors kand of i and j are (regularly) equivalent
A recurrence relation in terms of , much like eigenvector.25
σ counts how many common neighbors are there, times their neighbors’ regular similarity σ
![Page 26: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/26.jpg)
Regular Equivalence (2)
Is equivalent to
where α is the leading eigenvalue of A, and σ is the eigenvector.
But:• It may not give high self-similarity as it heavily
depends on the similarity of neighbors of i• It doesn’t give high similarity to pairs of vertices with
lots of common neighbors (because it looks at values of and , including )
Needs to be altered, and we’ll see two methods next. 26
![Page 27: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/27.jpg)
Regular Equivalence (3)
+ δIs equivalent to
with , δ =
• Calculate by starting with the initial values:,
Note: is a weighted some of the even powers of A (walks of even length), but we want all the lengths
![Page 28: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/28.jpg)
Regular Equivalence (4)
METHOD 2
+ δIs equivalent to
with with:
σ α 0 ,σ α α ,σ α α α α ,
is a weighted ( ?) count of all walks between iand j, (with < 1 so longer paths weigh less)
28Leicht, Holme, and Newman, 2006
![Page 29: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/29.jpg)
Regular Equivalence (5)
METHOD 2
+ δIs equivalent to
with • This looks a lot like Katz centrality
(recall α ∑ + β, where β is a constant initial weight given to each vertex so that its out degree matters)
Katz centrality of vertex i is the sum of ,
29
![Page 30: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/30.jpg)
Regular Equivalence (6)
• The regular equivalence tends to be high for vertices of high degree (more chances of their neighbors being similar)
• However, some low degree vertices could be similar as well, so the formula can be modified if desired:
+ δ
which is equivalent to
30
![Page 31: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/31.jpg)
The connection betweenstructural and regular equivalence
![Page 32: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/32.jpg)
Equivalence (1)
• R is a weighted count of all walks between i and j, i.e:
σ α α α α ,
• Recall that the structural equivalence attempts to count the # of common neighbors of i and j in different ways
• What is the correlation between the number of common neighbors of 4 and 5, and paths of length 2 between 4 and 5? 32
![Page 33: Section 7.12: Similarity - Facultyfaculty.nps.edu/rgera/MA4404/Winter2017/15-nodeSimilarity.pdf · • We talked about global properties –Average degree, average clustering, avepath](https://reader035.vdocuments.us/reader035/viewer/2022062505/5eb9ab6d8628e1538853e198/html5/thumbnails/33.jpg)
Equivalence (2)
• R is a weighted count of all walks between i and j, i.e:
σ α α α α ,
• Recall that the structural equivalence attempts to count the # of common neighbors in different ways
So the regular equivalence is a generalization of structural equivalence ( counts just paths of length 2)
3