small-world overlay p2p network

Small-world Overlay P2P Small-world Overlay P2P NetworkNetwork

Small-world Overlay P2P Small-world Overlay P2P NetworkNetwork

John C.S. LuiJohn C.S. Lui

Outline• Describe our motivations on small world overlay P2P network.

• Introduce the background information of P2P network and small world network.

• Propose our Small-world Overlay Protocol (SWOP).

• Explain our Flash Crowd Handling Protocol

• Illustrate the experimental results

Motivation• Fundamental Improvements in P2P network– Improve the performance of Object Lookup in P2P Network

– Solve high traffic loading of a “popular” and “dynamic” object, i.e. under a Flash Crowd Scenario

• Small world is applied to achieve above criteria

Background--Structured P2P Network--

• P2P network contains nodes (computers), which are acting as server as well as client.

• Structured P2P with two extra characteristics:– Decentralized – Structured

• To achieve this, consistent Distributed Hash Table (DHT) has been used.

• Two implementation issues for maintaining a logical structure:– Unique key assignment scheme

using DHT– Characteristic routing table

aims at reducing distance by at least half in each forwarding

Background--Structured P2P Network--

Objects

Nodes

x.x.x.c

x.x.x.d

x.x.x.a

x.x.x.e

x.x.x.b

224

Item 1

Item 2Item 3

1. Unique key assignment

100010000

217+100217+10

212-1

600012000

216+1

Background--Structured P2P Network--

Objects

Nodes

224

1000 10000

217+100 217+10

212-1

Item 1Item 2

Item 3

2. Characteristic Routing

600012000

216+1

Node 100 sends a message to node 217+100

Routing Table:…>217 : 217+10….

Routing Table:>21 : 217+100….

Background--Small world paradigm--

• Small world networks represent two major properties:1. Two randomly chosen

nodes are connected by short avg. distance

2. Nodes are joined together in groups. The effect is called high clustering.

Main Ideas• Apply Small world’s small average distance to structure P2P routing in order to improve the performance of object lookup

• Apply Small world’s high clustering coefficient to provide large traffic resolving solution

Small world Overlay Protocol (SWOP)

-- Overview --

DHT’s network

Small World Overlay Layer

Topology Control

Object Lookup

•Join Cluster (JCP)•Leave Cluster (LCP)

•Object Lookup (OLP)

Small world Overlay Protocol (SWOP)

-- Overview --

• Terminologies– Type of links

•Short links•Long links

– Types of nodes•Head nodes•Inner nodes

1

15

8

23

7

65

4

9

1413

12

11

10

23

161718

1920

21

22

2425

262728

2930 31 0

Join Cluster Protocol• Compute “distance” between predecessor and successor nodes

• Retrieve “group size” from nodes

• Select a group to join

• Update links information

1

15

8

23

7

65

4

9

1413

12

11

10

23

1617

1819

20

21

22

2425

262728

2930 31 0

D1 = 3D2 = 1G1 = 2G2 = 2

Leave Cluster Protocol• Contact short links neighbors for leaving

• If it is the “Head” node, hand over the short links neighbor and long links neighbors to next “Head”, and generate necessary new long links neighbors

P.S. There exists boundary case, like only one node in a cluster. The solution is written in the thesis.

Leave Cluster Protocol

• Node 9 leaves• Node 9 announces a new head 10 to 11

• Node 10 gets the long links and short links

• Link from node 4 to node 9 fade out when this link being used by a lookup request.

1

15

8

23

7

6

54

9

1413

12

11

10

23

1617

1819

20

21

22

2425

2627

2829

30 31 0

Object Lookup Protocol• Phase 1

– Query short link neighbors if there exists one which contains desired object.

– If result is positive, the object lookup request ends. Otherwise, phase 2 begins.

• Phase 2– By using the “head” ‘s long link neighbor, forward the object lookup request to another cluster.

– Phase 1 continues by that node receives the object lookup request.

Object Lookup Example (1)

Node 0 requests object 29, managed by node 31

1

15

8

23

7

65

4

9

1413

12

11

10

23

1617

1819

20

21

22

2425

262728

2930 31 0

Object Lookup Example (2)

Node 0 requests object 16, managed by node 17

1

15

8

23

7

65

4

9

1413

12

11

10

23

1617

1819

20

21

22

2425

262728

2930 31 0

Performance analysis mathematical

• Worst case average link traversals–E[X] ≤ (1 + log2(m/2)) (8ln(3m)/k)• E[X] represents worst case expected number of link traversals

• m represents the number of clusters in the SWOP network

• k represents the number of long links

• The proof is conducted by randomized algorithm and it leaves in the thesis

Performance analysis simulation result

• Performance Metrics– Probability density function of lookup hop count

• Simulation Setup– We added nodes one by one and reform the topology according the construction protocol.

– Total about 1k-5k nodes were generated in the system.

– Each node generated data lookup request randomly certain times.

Performance analysis simulation result

Internet Flash Crowd• Unpredictable huge amount of request for a popular object is generated towards the object owner

• This overwhelms the network and the CPU resources of the owner.

• e.g. CNN news server during 911

Types for Internet Flash Crowd

• Static Flash Crowd– The popular object involved will remain unchanged after its first appearance, e.g. new movie.

• Dynamic Flash Crowd– The popular object involved will change after its first appearance, e.g. news

Requirements for handling Internet Flash

Crowd• Loading Distributed

– Reduce the bottleneck by caching and replication

• Demand driven– Replication scheme has to be demand driven, otherwise, it will be a flooding scheme.

• Dynamic compatibility– Consider how to handle dynamic objects which are changed by original source

Protocols for handlingFlash Crowd

Algorithm1. Each node periodically records the

access rates of objects stored.2. If the access rate of an object is

greater than certain threshold, the owner of this object contacts the “head” of its cluster.

3. “head” spread this object to all its long link neighbors

Protocols for handlingFlash Crowd

4. Each long link neighbor “caches” this object so that each neighbor can be acted as an image source of this object.

5. Each long link neighbor keeps track of the rate of newly added object.

Dynamic cases – Refresh Message Aims for reminding neighbor nodes

to get the latest updated object.

Static Flash Crowd Example

1

15

8

23

7

6

5

4

9

1413

12

11

10

23

1617

1819

20

21

22

2425

26

27

2829

3031 0

Assume, in node 4, an item’s hitting rate exceeds threshold

Dynamic Flash Crowd Example

1

15

8

23

7

6

5

4

9

1413

12

11

10

23

1617

1819

20

21

22

2425

26

27

2829

3031 0

Original ver.New ver.Refresh msg.

Experiment results• Study the performance on handling flash crowd scenario

• Performance metrics:– Number of successful request

• In each interval, the nodes try to lookup the popular object. The metric counts the number of nodes can retrieve in this interval.

– Number of messages produced (traffic burden)• Total number of messages in the system.

• Settings:– Static flash crowd with fixed rate req/s.– Dynamic flash crowd with fixed rate req/s with

simulation time 25, 50 and 75 for object version update

– Dynamic flash crowd varying with simulation time 25, 50 and 75 for object version update

– Static flash crowd with fixed rate req/s

Experiment resultsEvaluation procedure

We added 2000 nodes in the system using SWOP to form small world.

One object was randomly chosen and was acted as the popular object.

Each node generated that popular object’s lookup request traffic with rate .

Each node has fixed service rate and a queue for handling the item request.

Experimental resultsStatic flash crowd

Experimental resultsDynamic flash crowd

Experiment resultsVariation of object request

rate

Experiment resultsOperation cost

Replicating Region

Summary• Build a protocol that applies Small world features on P2P network

• Improve the Object Lookup performance with the support of mathematical analysis.

• Propose an algorithm to handle massive traffics produced by “popular” and “dynamic” objects.

Thank youThank youThank youThank you

Questions & AnswersQuestions & Answers

Small world Overlay Protocol

Stabilize algorithm• Method 1:

– Each node periodically probes their neighbors. Once a timeout event occurs (node failure) for a corresponding probing, the routing information is updated.

• Method 2:– Each node does not perform a probing until an item lookup event occurs. When the lookup event fails, meaning there exists a node failure, the routing information is updated according to the failed node.

Experiment resultsVariation on number of long link

neighbors k

Experiment resultsVariation of queue size

Mathematical Analysis• Apply Markov Model• Define each finite state as number of cached clusters

• Retrieve the transition probabilities

• Define Troop State• Compute the troop state probability• Calculate the expected time to troop state