network analysis with networkx : fundamentals of network theory-1

Kyunghoon Kim

Network Analysis with networkX

Fundamentals of network theory-1

2014. 05. 14.

UNIST Mathematical Sciences

Kyunghoon Kim ( [email protected] )

5/14/2014 Fundamentals of network theory-1 1

mailto:[email protected]

Kyunghoon Kim

● A NETWORK is, in its simplest form, a collection of points joined together in pairs by lines.

What is a Network?

Newman, Mark. Networks: an introduction. OUP Oxford, 2009.


Kyunghoon Kim

● A NETWORK is, in its simplest form, a collection of points joined together in pairs by lines.

● Network is general, but powerful means of representing patterns of connections or interactions between the parts of a system.

What is a Network?



Kyunghoon Kim

What is a Network?

Shipping (sea) networksTelecommunications networks

Air traffic networkData networks

Source: Britain From Above (http://www.bbc.co.uk/britainfromabove)


http://www.bbc.co.uk/britainfromabove

Kyunghoon Kim

● Network thinking can be applied almost anywhere!

What is a Network?


Kyunghoon Kim

● Network thinking can be applied almost anywhere!

Example of Network Thinking

http://journals.uic.edu/ojs/index.php/fm/article/view/941/863

My data sources were publicly released information reported in major newspapers such as the New York Times, Wall Street Journal, Washington Post, and the Los Angeles Times. As I monitored the investigation, it was apparent that the investigators would not be releasing all pertinent network/relationship information and actually may be releasing misinformation to fool the enemy. I soon realized that the data was not going to be as complete and accurate as I had grown accustomed to in mapping and measuring organizational networks.



Kyunghoon Kim


http://www.orgnet.com/hijackers.html



Kyunghoon Kim



Kyunghoon Kim


Six (6) shortcuts were added to the network temporarily in order to collaborate and coordinate.These shortcuts reduced the average path length in the network by over 40% thus improving the information flow in the network - see Table 1.

When the network is brought closer together by these shortcuts, all of the pilots ended up in a small clique - the perfect structure to efficiently coordinate tasks and activities.

There is a constant dynamicbetween keeping the network hidden and actively using it to accomplish objectives (Baker and Faulkner, 1993).


http://journals.uic.edu/ojs/index.php/fm/article/view/941/863%23tab1

Kyunghoon Kim



Kyunghoon Kim


Saddam Hussein


Kyunghoon Kim



An Introduction to Network Theory | Kyle Findlay | SAMRA 2010 | 15Where’s it applied?

Epidemiology (i.e. spread of diseases)e.g. spread of foot & mouth disease in the UK in 2001 over 75 days

URL: http://www.youtube.com/watch?v=PufTeIBNRJ4

http://www.youtube.com/watch?v=PufTeIBNRJ4

An Introduction to Network Theory | Kyle Findlay | SAMRA 2010 | 16

Physicse.g. particle interactions, the structure of the universe

Where’s it applied?

URL: http://www.youtube.com/watch?v=8C_dnP2fvxk

http://www.youtube.com/watch?v=8C_dnP2fvxk


Engineeringe.g. creation of robust infrastructure (e.g. electricity, telecoms), rust formation (natural growth processes similar to diffusion limited aggregation)


URL: http://www.youtube.com/watch?v=lRZ2iEHFgGo URL: http://www.youtube.com/watch?v=AEoP-XtJueo

http://www.youtube.com/watch?v=lRZ2iEHFgGo

http://www.youtube.com/watch?v=AEoP-XtJueo


Technologye.g. mapping the online world, making networks resilient in the face of cyber-terrorism, optimising cellular networks, controlling air traffic


URL: http://www.youtube.com/watch?v=l-RoDv7c5ok URL: http://www.youtube.com/watch?v=o4g930pm8Ms

Vid not working

http://www.youtube.com/watch?v=l-RoDv7c5ok

http://www.youtube.com/watch?v=o4g930pm8Ms

An Introduction to Network Theory | Kyle Findlay | SAMRA 2010 | 19Where’s it applied?

Biologye.g. fish swimming in schools, ant colonies, birds flying in formation, crickets chirping in unison, giant honeybees shimmering

URL: http://www.youtube.com/watch?v=Sp8tLPDMUygURL: http://www.youtube.com/watch?v=YadP3w7vkJA

http://www.youtube.com/watch?v=Sp8tLPDMUyg

http://www.youtube.com/watch?v=YadP3w7vkJA


Medicinee.g. cell formation, nervous system, neural networks


Source: The Human Brain Book by Rita Carter


And, most interestingly…societye.g. interactions between people (e.g. Facebook; group behaviour)


URL: http://www.youtube.com/watch?v=9n9irapdON4 URL: http://www.youtube.com/watch?v=sD2yosZ9qDw

http://www.youtube.com/watch?v=9n9irapdON4

http://www.youtube.com/watch?v=sD2yosZ9qDw

Kyunghoon Kim

● A network – also called a graph in the mathematical literature – is, as we have said, a collection of vertices joined by edges.

Networks and their Representation

Area Object Relation

Graph Theory Vertices Edges

Computer Science Nodes Links

Physics Sites Bonds

Sociology Actors Ties


Kyunghoon Kim


Figure 6.1: Two small networks. (a) A simple graph, i.e., one having no multiedges or self-edges.(b) A network with both multiedges and self-edges.



Kyunghoon Kim


𝐴𝐴𝑖𝑖𝑖𝑖 = �10

𝑖𝑖𝑖𝑖 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡 𝑖𝑖𝑖𝑖 𝑎𝑎𝑎𝑎 𝑡𝑡𝑒𝑒𝑒𝑒𝑡𝑡 𝑏𝑏𝑡𝑡𝑡𝑡𝑏𝑏𝑡𝑡𝑡𝑡𝑎𝑎 𝑣𝑣𝑡𝑡𝑡𝑡𝑡𝑡𝑖𝑖𝑣𝑣𝑡𝑡𝑖𝑖 𝑖𝑖 𝑎𝑎𝑎𝑎𝑒𝑒 𝑗𝑗,𝑜𝑜𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑏𝑏𝑖𝑖𝑖𝑖𝑡𝑡.


Kyunghoon Kim

● import networkx as nxG = nx.Graph()G.add_node(1)G.add_nodes_from([1,2,3,4,5])G.nodes()

● >>> [1, 2, 3, 4, 5]



Kyunghoon Kim

● G.add_edge(1,2)G.add_edges_from([(1,2),(2,4),(2,5),(3,4)])G.edges()

● >>> [(1, 2), (2, 4), (2, 5), (3, 4)]



Kyunghoon Kim

● nx.to_numpy_matrix(G)

● matrix([[ 0., 1., 0., 0., 0.],

[ 1., 0., 0., 1., 1.],

[ 0., 0., 0., 1., 0.],

[ 0., 1., 1., 0., 0.],

[ 0., 1., 0., 0., 0.]])

nx.draw(G)



Kyunghoon Kim



Kyunghoon Kim


= 𝐴𝐴 =


Kyunghoon Kim



Kyunghoon Kim

Degree


Kyunghoon Kim

Degree

𝑘𝑘𝑖𝑖 = �𝑖𝑖=1

𝑛𝑛

𝐴𝐴𝑖𝑖𝑖𝑖

𝑘𝑘2 = �𝑖𝑖=1

5

𝐴𝐴1𝑖𝑖 = 3


Kyunghoon Kim

Degree


Kyunghoon Kim

Degree


𝑛𝑛


2𝑚𝑚 = �𝑖𝑖=1

𝑛𝑛

𝑘𝑘𝑖𝑖


Kyunghoon Kim

Total Degree m


𝑛𝑛


2𝑚𝑚 = �𝑖𝑖=1

𝑛𝑛

𝑘𝑘𝑖𝑖

𝑚𝑚 =12�𝑖𝑖=1

𝑛𝑛

𝑘𝑘𝑖𝑖 =12�

𝑖𝑖𝑖𝑖



Kyunghoon Kim

Total Degree m


𝑛𝑛


2𝑚𝑚 = �𝑖𝑖=1

𝑛𝑛

𝑘𝑘𝑖𝑖

𝑚𝑚 =12�𝑖𝑖=1

𝑛𝑛

𝑘𝑘𝑖𝑖 =12�

𝑖𝑖𝑖𝑖

𝐴𝐴𝑖𝑖𝑖𝑖𝑎𝑎


Kyunghoon Kim

Mean Degree c


𝑛𝑛


2𝑚𝑚 = �𝑖𝑖=1

𝑛𝑛

𝑘𝑘𝑖𝑖

𝑣𝑣 =1𝑎𝑎�𝑖𝑖=1

𝑛𝑛

𝑘𝑘𝑖𝑖 =2𝑚𝑚𝑎𝑎


Kyunghoon Kim

Mean Degree c

𝑣𝑣 =1𝑎𝑎�𝑖𝑖=1

𝑛𝑛

𝑘𝑘𝑖𝑖 =2𝑚𝑚𝑎𝑎

>>> G.degree()

{1: 1, 2: 3, 3: 1, 4: 2, 5: 1}

>>> G.degree(2)

3

>>> Degree = G.degree().values()

[1, 3, 1, 2, 1]

>>> sum(Degree)/len(G.nodes())

1.6000000000000001


Kyunghoon Kim

Density or connectance of a graph

𝑎𝑎2 =

12𝑎𝑎(𝑎𝑎 − 1)

in a simple graph(i.e., one with no multiedgesor self-edges)


Kyunghoon Kim


𝜌𝜌 =𝑚𝑚𝑎𝑎2

=2𝑚𝑚

𝑎𝑎(𝑎𝑎 − 1)

=𝑣𝑣

𝑎𝑎 − 1

𝑎𝑎2 =

12𝑎𝑎(𝑎𝑎 − 1)


Kyunghoon Kim


𝜌𝜌 =𝑚𝑚𝑎𝑎2

=2𝑚𝑚

𝑎𝑎(𝑎𝑎 − 1)

=𝑣𝑣

𝑎𝑎 − 1

>>> total = 0.5*5*(5-1)

>>> 4/total

0.4

>>> nx.density(G)

0.4


Kyunghoon Kim

● The in-degree is the # of ingoing edges connected to a vertex.

● The out-degree is the # of outgoing edges connected to a vertex.

● 𝐴𝐴𝑖𝑖𝑖𝑖 = 1 if there is an edge from 𝑗𝑗 to 𝑖𝑖.

In-degree and out-degree

𝑘𝑘𝑖𝑖𝑖𝑖𝑛𝑛 = �𝑖𝑖=1

𝑛𝑛

𝐴𝐴𝑖𝑖𝑖𝑖 𝑘𝑘𝑖𝑖𝑜𝑜𝑜𝑜𝑜𝑜 = �𝑖𝑖=1

𝑛𝑛



Kyunghoon Kim

● A path in a network is any sequence of vertices such that every consecutive pair of vertices in the sequence is connected by an edge in the network.

Paths


Kyunghoon Kim

● A path in a network is any sequence of vertices such that every consecutive pair of vertices in the sequence is connected by an edge in the network.

● A path is a route across the network that runs from vertex to vertex along the edges of the network.

Paths


Kyunghoon Kim

● The length of a path in a network is the # of edges traversed along the path (not the # of vertices).

Length of Paths


Kyunghoon Kim


● 𝐴𝐴𝑖𝑖𝑖𝑖 = 1 if there is an edge from 𝑗𝑗 to 𝑖𝑖,and 0 otherwise.

Length of Paths


Kyunghoon Kim



Length of Paths

𝑁𝑁𝑖𝑖𝑖𝑖(2) = �

𝑘𝑘=1

𝑛𝑛

𝐴𝐴𝑖𝑖𝑘𝑘𝐴𝐴𝑘𝑘𝑖𝑖 = 𝐀𝐀2 𝑖𝑖𝑖𝑖


Kyunghoon Kim



i ← k , k ←j

Length of Paths


𝑘𝑘=1

𝑛𝑛



Kyunghoon Kim



Length of Paths


𝑘𝑘=1

𝑛𝑛



𝑘𝑘,𝑙𝑙=1

𝐴𝐴𝑖𝑖𝑘𝑘𝐴𝐴𝑘𝑘𝑙𝑙𝐴𝐴𝑙𝑙𝑖𝑖 = 𝐀𝐀3 𝑖𝑖𝑖𝑖


Kyunghoon Kim

● import numpy as npmatrix = nx.to_numpy_matrix(G)matrix*np.transpose(matrix)

matrix([[ 1., 0., 0., 1., 1.],

[ 0., 3., 1., 0., 0.],

[ 0., 1., 1., 0., 0.],

[ 1., 0., 0., 2., 1.],

[ 1., 0., 0., 1., 1.]])

Length of Paths


𝑘𝑘=1

𝑛𝑛



Kyunghoon Kim

Loops

𝑁𝑁𝑖𝑖𝑖𝑖(2) =


Kyunghoon Kim

● A shortest path is a path between two vertices such that no shorter path exists:

Shortest path (Geodesic path)



Kyunghoon Kim

● A shortest path is a path between two vertices such that no shorter path exists:

● Shortest paths are necessarily self-avoiding.

● Shortest paths are not necessarily unique.

Shortest path (Geodesic path)



Kyunghoon Kim

● The shortest distance between vertices 𝑖𝑖 and 𝑗𝑗is the smallest value of 𝑡𝑡 such that 𝐀𝐀𝑟𝑟 𝑖𝑖𝑖𝑖 > 0.

Shortest distance (Geodesic distance)


Kyunghoon Kim

● The shortest distance between vertices 𝑖𝑖 and 𝑗𝑗is the smallest value of 𝑡𝑡 such that 𝐀𝐀𝑟𝑟 𝑖𝑖𝑖𝑖 > 0.

● However, in practical case we use Dijkstra’salgorithm. We will study it later.

Shortest distance (Geodesic distance)


Kyunghoon Kim

● The diameter of a graph is the length of the longest geodesic path between any pair of vertices in the network for which a path actually exists.

Diameter of a graph


Kyunghoon Kim

● Which are the most importantor central vertices in a network?

Measures and Metrics


Kyunghoon Kim

● Which are the most importantor central vertices in a network?

● Degree centrality,Closeness centrality,Betweenness centrality,Eigenvector centrality,Pagerank,Similarity…

Measures and Metrics


Kyunghoon Kim

● Degree is sometimes called degree centrality in the social networks literature.

● In directed networks, vertices have both an in-degree and an out-degree, and both may be useful as measures of centrality in the appropriate circumstances.

Degree Centrality


Kyunghoon Kim

Degree Centrality


Kyunghoon Kim

Clustering Coefficient (local)

● For a vertex 𝑖𝑖, we define :

𝐶𝐶𝑖𝑖 =(𝑎𝑎𝑛𝑛𝑚𝑚𝑏𝑏𝑡𝑡𝑡𝑡 𝑜𝑜𝑖𝑖 𝑝𝑝𝑎𝑎𝑖𝑖𝑡𝑡𝑖𝑖 𝑜𝑜𝑖𝑖 𝑎𝑎𝑡𝑡𝑖𝑖𝑒𝑒𝑡𝑏𝑏𝑜𝑜𝑡𝑡𝑖𝑖 𝑜𝑜𝑖𝑖 𝑖𝑖 𝑡𝑡𝑡𝑎𝑎𝑡𝑡 𝑎𝑎𝑡𝑡𝑡𝑡 𝑣𝑣𝑜𝑜𝑎𝑎𝑎𝑎𝑡𝑡𝑣𝑣𝑡𝑡𝑡𝑡𝑒𝑒)

(𝑎𝑎𝑛𝑛𝑚𝑚𝑏𝑏𝑡𝑡𝑡𝑡 𝑜𝑜𝑖𝑖 𝑝𝑝𝑎𝑎𝑖𝑖𝑡𝑡𝑖𝑖 𝑜𝑜𝑖𝑖 𝑎𝑎𝑡𝑡𝑖𝑖𝑒𝑒𝑡𝑏𝑏𝑜𝑜𝑡𝑡𝑖𝑖 𝑜𝑜𝑖𝑖 𝑖𝑖)


Kyunghoon Kim


● >>> nx.clustering(G)

● {1: 0.0, 2: 0.3333, 3: 0.6666, 4: 1.0, 5: 1.0}

𝐶𝐶2 =242

=13

= 0.3333


Kyunghoon Kim



“Unclustered” network “Clustered” networkNone of Ego’s friends know each other* All of Ego’s friends know each other

An Introduction to Network Theory | Kyle Findlay | SAMRA 2010

Kyunghoon Kim



– A real-world example: CEOs of Fortune 500 companies• Which companies share directors? Clusters are colour-coded

An Introduction to Network Theory | Kyle Findlay | SAMRA 2010

Kyunghoon Kim

● This measures the mean distance from a vertex to other vertices.

● Suppose 𝑒𝑒𝑖𝑖𝑖𝑖 is the length of a geodesic path from 𝑖𝑖 to 𝑗𝑗, meaning the # of edges along the path.

Closeness Centrality


Kyunghoon Kim

● This measures the mean distance from a vertex to other vertices.

● Suppose 𝑒𝑒𝑖𝑖𝑖𝑖 is the length of a geodesic path from 𝑖𝑖 to 𝑗𝑗, meaning the # of edges along the path.

● The mean shortest distance from 𝑖𝑖 to 𝑗𝑗, averaged over all vertices 𝑗𝑗 in the network, is


𝑙𝑙𝑖𝑖 =1𝑎𝑎�

𝑖𝑖

𝑒𝑒𝑖𝑖𝑖𝑖


Kyunghoon Kim

● Exclude for 𝑗𝑗 = 𝑖𝑖,


𝑙𝑙𝑖𝑖 =1

𝑎𝑎 − 1 �𝑖𝑖(≠𝑖𝑖)


𝑙𝑙𝑖𝑖 =1𝑎𝑎�

𝑖𝑖



Kyunghoon Kim

● This quantity takes low values for vertices that are separated from others by only a short shortest distance on average.

● which is the opposite of other measures.

● So commonly we take inverse :


𝐶𝐶𝑖𝑖 =1𝑙𝑙𝑖𝑖

=𝑎𝑎

∑𝑖𝑖 𝑒𝑒𝑖𝑖𝑖𝑖


Kyunghoon Kim

>>> nx.closeness_centrality(G)

{1: 0.5714, 2: 1.0, 3: 0.8, 4: 0.6666, 5: 0.6666}


𝐶𝐶𝑖𝑖 =1𝑙𝑙𝑖𝑖

=𝑎𝑎

∑𝑖𝑖 𝑒𝑒𝑖𝑖𝑖𝑖

𝐶𝐶1 =4

∑𝑖𝑖 𝑒𝑒1𝑖𝑖=

41 + 2 + 2 + 2

= 0.5714


Kyunghoon Kim

● This measures the extent to which a vertex lies on paths between other vertices.

Betweenness Centrality


Kyunghoon Kim

● This measures the extent to which a vertex lies on paths between other vertices.

● For example, assume that every pair of vertices exchanges a message with equal probability per unit time and that messages always take the shortest path.



Kyunghoon Kim

● For example, assume that every pair of vertices exchanges a message with equal probability per unit time and that messages always take the shortest path.

● Question : if we wait a suitably long time until many messages have passed between each pair of vertices, how many messages, on average, will have passed through each vertex en route to their destination?



Kyunghoon Kim

● Question : if we wait a suitably long time until many messages have passed between each pair of vertices, how many messages, on average, will have passed through each vertex en route to their destination?

● Answer : since messages are passing down each shortest path at the same rate, the number passing through each vertex is simply proportional to the # of shortest paths the vertex lies on.



Kyunghoon Kim

● Let 𝑎𝑎𝑠𝑠𝑜𝑜𝑖𝑖 be 1 if vertex 𝑖𝑖 lies on the shortest path from 𝑖𝑖 to 𝑡𝑡 and 0 if it does not or if there is no such path.

● The betweenness centrality 𝑥𝑥𝑖𝑖 is given by


𝑥𝑥𝑖𝑖 = �𝑠𝑠𝑜𝑜

𝑎𝑎𝑠𝑠𝑜𝑜𝑖𝑖


Kyunghoon Kim

● Note that the shortest paths between a pair of vertices need not be vertex independent, meaning they may pass through some of the same vertices.

● Define 𝑒𝑒𝑠𝑠𝑜𝑜 to be the total # of shortest paths from 𝑖𝑖 to 𝑡𝑡 .

where 𝑛𝑛𝑠𝑠𝑠𝑠𝑖𝑖

𝑔𝑔𝑠𝑠𝑠𝑠= 0 if both 𝑎𝑎𝑠𝑠𝑜𝑜𝑖𝑖 and 𝑒𝑒𝑠𝑠𝑜𝑜 are zero.




𝑒𝑒𝑠𝑠𝑜𝑜


Kyunghoon Kim

>>> nx.betweenness_centrality(G)

{1: 0.0, 2: 0.5833, 3: 0.0833, 4: 0.0, 5: 0.0}





𝑥𝑥1 = �𝑠𝑠𝑜𝑜

𝑎𝑎𝑠𝑠𝑜𝑜1



Kyunghoon Kim


{1: 0.0, 2: 0.5833, 3: 0.0833, 4: 0.0, 5: 0.0}g = nx.Graph()

g.add_edges_from([(1,2),(2,3),(2,4),(2,5),(3,4),(3,5)])end = 6

for i in range(1,end):

for j in range(1,end):

#print 'from', i, 'to', j

print([p for p in nx.all_shortest_paths(g,i,j) if len(p)>2])

#print nx.dijkstra_path(g,i,j)

#print '=============='

nx.betweenness_centrality(g, normalized=False)

plt.cla()

nx.draw(g)




𝑒𝑒𝑠𝑠𝑜𝑜= 3 +

12


Kyunghoon Kim


{1: 0.0, 2: 0.5833, 3: 0.0833, 4: 0.0, 5: 0.0}

[[1, 2, 3]]

[[1, 2, 4]]

[[1, 2, 5]]

[[3, 2, 1]]

[[4, 2, 1]]

[[4, 2, 5], [4, 3, 5]]

[[5, 2, 1]]

[[5, 2, 4], [5, 3, 4]]




𝑒𝑒𝑠𝑠𝑜𝑜= 3 +

12


Kyunghoon Kim


{1: 0.0, 2: 0.5833, 3: 0.0833, 4: 0.0, 5: 0.0}

To normalize,

we use the following equation,2

(𝑛𝑛−1)(𝑛𝑛−2)

Then,


𝑥𝑥2 =2

4 × 3�𝑠𝑠𝑜𝑜


𝑒𝑒𝑠𝑠𝑜𝑜=

16

3 +12

= 0.5833


Kyunghoon Kim

Appendix

● G.add_edges_from([(1,3),(3,6),(3,7),(3,8),(6,8),(7,8),(8,9)])


Kyunghoon Kim

● nx.draw_spring(G)

● nx.to_numpy_matrix(G)

Appendix


Kyunghoon Kim

● Newman, Mark. Networks: an introduction. OUP Oxford, 2009.

● An Introduction to Network Theory | Kyle Findlay | SAMRA 2010

● http://journals.uic.edu/ojs/index.php/fm/article/view/941/863

● http://www.orgnet.com/hijackers.html

Reference




network analysis with networkx : fundamentals of network theory-1

Education