medusa – new model of internet topology using k-shell decomposition shai carmi shlomo havlin...
Post on 19-Dec-2015
219 views
TRANSCRIPT
![Page 1: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/1.jpg)
MEDUSA – New Model of Internet Topology Using
k-shell DecompositionShai Carmi
Shlomo Havlin
Bloomington 05/24/2005
![Page 2: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/2.jpg)
Who we are
Talk prepared by Shai Carmi. Graduate student in the
Department of Physics, Bar-Ilan University, Israel.
Supervised by Prof. Shlomo Havlin, who gives the talk.
![Page 3: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/3.jpg)
Who we are
Collaborators – Prof. Scott Kirkpatrick, Hebrew
University of Jerusalem, Israel. Dr. Yuval Shavitt, Tel-Aviv
University, Israel. Eran Shir, Ph.D. Student, Tel-
Aviv University, Israel.
Scott
![Page 4: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/4.jpg)
Measuring the Internet Previous efforts to measure the Internet have
used : One machine + Traceroute to many
destinations. Many machines, specially deployed to
traceroute to many destinations Sites restricted to academic or gov’t labs, on
network backbone General perception was that Law of
Diminishing Returns has set in.
![Page 5: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/5.jpg)
Measuring the Internet
DIMES : Distributed Internet MEasurement and Simulations (http://www.netdimes.org), seems to have made a breakthrough.
Don’t manage machines, offer a very lightweight, limited purpose client, and collect its measurements centrally.
100 – 1000 clients via word-of-mouth (Sep04 to Apr05). >5000 clients now, achieved via Science article, slashdot. 82 countries represented. 2-3 M measurements per day.
![Page 6: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/6.jpg)
The network we analyze
We consider the Internet at the level of its autonomous systems (ASes), with roughly 20,000 nodes and 70,000 links.
We use data gathered between March and June 2005. In the future, can study network dynamics, using intervals of months, weeks or even days.
![Page 7: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/7.jpg)
k-shell method
Use recursive pruning to peel network layers. To remove the 1-shell, keep removing all nodes
with one link (degree=1) until only nodes with degree 2 or more remain.
To remove the 2-shell, keep removing nodes with 2 links, until all degrees are >= 3.
Keep going until all nodes are removed.
![Page 8: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/8.jpg)
k-shell method - example
Original Graph
![Page 9: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/9.jpg)
k-shell method - example
Pruning Degree 1
![Page 10: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/10.jpg)
k-shell method - example
Keep Pruning Degree 1
![Page 11: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/11.jpg)
k-shell method - example
Keep Pruning Degree 1
![Page 12: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/12.jpg)
k-shell method - example
Pruning Degree 2
![Page 13: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/13.jpg)
k-shell method - example
Keep Pruning Degree 2
![Page 14: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/14.jpg)
k-shell method - example
Pruning Degree 3
![Page 15: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/15.jpg)
k-shell method
Definitions : k-Core – union of all
shells with indices >= k.
k-Crust – union of all shells withindices <= k.
![Page 16: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/16.jpg)
Applications
Can use k-shell method to analyze the AS network.
For example, color each node by its shell index to visualize the network.
Next, plot quantities as a function of the shell index.
Gain understanding of the network structure. More useful indicator then the degree.
![Page 17: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/17.jpg)
AS graph colored by shells
![Page 18: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/18.jpg)
Identification of a nucleus
k-shell method enables us to identify the heart, or nucleus of the network as nodes in the last core.
No parameters need to be fixed. (Topology dependent only).
Stable over time. Significant ASes (tier-1) were verified to be in
the nucleus. Most quantities show singular behavior at the
last shell. Some examples -
![Page 19: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/19.jpg)
Number of nodes and degrees in the shells
Slope=
~2.6
![Page 20: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/20.jpg)
Centrality vs. shell
![Page 21: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/21.jpg)
Where links go
![Page 22: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/22.jpg)
Distances (vs. crusts)
Distances measured between all pairs in the largest cluster of the crust
![Page 23: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/23.jpg)
Number of site-distinct paths in the nucleus
At least 41 distinct paths between each pair
41 is the k-shell index of the nucleus
The nucleus is k-connected!
![Page 24: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/24.jpg)
Beyond the nucleus
It is left to understand the role of the other nodes in the network.
We look at the connectivity properties of the crusts.
Incorporate this with observations from previously shown plots.
![Page 25: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/25.jpg)
Clusters in the crusts
Percolation Threshold
![Page 26: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/26.jpg)
Structure of the AS network
Nodes outside of the nucleus can be categorized into –
The fractal part – nodes in the largest cluster of the one-before-last crust – contains ~70% of the nodes in the network.
The rest of the nodes become then the isolated part.
![Page 27: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/27.jpg)
Properties of the fractal part
Connected (by construction), so that routing is possible without traversing and congesting the nucleus.
Connections to the nucleus decrease path lengths significantly.
Show fractal properties and power-laws.
![Page 28: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/28.jpg)
Properties of the fractal part
Fractal dimension calculated using the ‘box cover method’ (SHM 2005).
Crossover behavior between non-fractal and completely fractal – at the percolation critical point.
Percolation theory arguments predict
2 lN
![Page 29: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/29.jpg)
Properties of the fractal part
Percolation theory
prediction – slope = 2.5
The 6-crust is renormalized with box of
size 4
![Page 30: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/30.jpg)
Properties of the isolated part Contains ~30% of the ASes, not reachable
without the nucleus. Low degree nodes, high clustering. Many small clusters. Contributions found in up to k=10 shell. Many nodes are connected directly to highly
connected nodes in the nucleus.
![Page 31: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/31.jpg)
AS network model We summarize – the AS network is composed of 3
main sub-components :
1. Nucleus – Nodes in last shell.
2. Fractal Part – Nodes in the largest cluster of the one-before-last crust.
3. Isolated Part – Nodes in all-but-largest clusters of the one-before-last crust.
We name this model Medusa because of its jellyfish like structure.
Some similarities to Faloutsous’ Jellyfish model but important differences.
![Page 32: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/32.jpg)
Medusa model of the AS network
Our view of the Internet
![Page 33: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/33.jpg)
The End.
Thank you for your attention.
![Page 34: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/34.jpg)
Comments
Some properties (such as percolation) are found in the “Random-Scale-Free-Model”
Internet might not be so special. To have more insight must investigate
navigation with commercial restrictions –Many properties change.
![Page 35: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/35.jpg)
Clustering coefficient vs. shell
![Page 36: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/36.jpg)
Nearest neighbor degree vs. shell
![Page 37: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/37.jpg)
Derivation of the fractal dimension At the threshold, almost all the high degree nodes
are removed, such that the network becomes similar to a random (Erdos-Renyi) network.
Percolation in random networks is equivalent to percolation in an infinite dimensional lattice, in which we know the fractal dimension of the largest component is 4, .
For infinite dimensional lattices, . Thus we conclude, ,or the ''shortest-path''
fractal dimension is 2.
2lM
2rl 4rM
![Page 38: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/38.jpg)
Faloutsos’ Internet jellyfish model The Jellyfish Model : Identify core of network as maximal clique.
( not a very robust or reproducible approach) Shells around network labeled by hop count
from core (a small world) Find large portion of peripheral sites connect to
core.
![Page 39: MEDUSA – New Model of Internet Topology Using k-shell Decomposition Shai Carmi Shlomo Havlin Bloomington 05/24/2005](https://reader035.vdocuments.us/reader035/viewer/2022062407/56649d385503460f94a121d5/html5/thumbnails/39.jpg)
Faloutsos’ Internet jellyfish model
Faloutsos’ view of the internet