land: locality aware networks for distributed hash tables dahlia malkhi the hebrew university of...
Post on 21-Dec-2015
215 views
TRANSCRIPT
![Page 1: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/1.jpg)
LAND:Locality Aware Networks for Distributed Hash Tables
Dahlia MalkhiThe Hebrew University of Jerusalem
Joint work with:Ittai Abraham and Oren Dobzinski
![Page 2: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/2.jpg)
2
Motivation
Today’s Internet:– Many lightweight clients (web browsers)– Relatively simple servers (HTTP)
Client-Server paradigm is suited for a world of thin clients which do not have a lot of bandwidth and computational power.
Tomorrow’s Internet ?– Most devices will have enough bandwidth and
CPU to become both a client and a server (peer).– Users will have an active network presence.
![Page 3: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/3.jpg)
3
Challenges
New Distributed Storage and Retrieval Services will face many challenges:
Scalability – number of users constantly increases. Dynamism – unlike today’s web servers, peers will be
constantly joining and leaving. Congestion – Access to data is non-uniform and may
cause hot spots. Fault tolerance – availability and reliability of data. Efficiency – systems should be resource efficient and
provide the best performance (more on this later).
![Page 4: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/4.jpg)
4
Overlay networks and distributed data structures
Hash tables: store and lookup by object id
Quorum systems: global search Prefix lookup SQL Google ?
![Page 5: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/5.jpg)
6
Viceroy Overview
Viceroy, the first constant-degree distributed hash table [Malkhi, Naor, Ratacjzak, PODC 02]
LAND, the first peer-to-peer network and lookup algorithm that has worst case constant distortion [Abraham, Malkhi, Dobzinski SODA 2004]
A generic overlay network approach with implicit load balancing [Abraham, Awerbuch, Azar, Bartal, Malkhi, Pavlov, IPDPS 03]
A publish-subscribe mechanisms for scale-free graphs based on probabilistic quorums [Abraham, Malkhi, DISC 03]
Small-world DHTs on planar metrics [Abraham, Malkhi, 2003] An optimal asynchronous resource discovery scheme [Abraham,
Dolev, PODC 03] Investigation of user privacy and anonymity [Bickson, Malkhi, 2003] An efficient, localized scheme for estimating the number of
nodes in a dynamic network [Horowitz, Malkhi, IPL 2003]
![Page 6: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/6.jpg)
7
Overlay Networks for Finding Nearest Copy of Data
Nodes construct an overlay layer that allows to use new network architectures and services.
A Content Addressable Network allows to route to the target node by examining the object’s id.
If multiple copies of the same object exist then the closest copy should be accessed.
Complexity measures for Overlay Networks:
– Number of hops from source node to target node. – Degree of the overlay network. – Amount of additional memory needed per object. – Adaptability: number of nodes that change their state each
time a peer joins/leaves the system.– Load on nodes related to the locating task
![Page 7: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/7.jpg)
Previous worksNode-degree
Dilation Congestion Topology
Chord(MIT)
log(n) log(n) log(n)/n Hypercube
Tapestry(Berkeley)
log(n) log(n) log(n)/n Hypercube
CAN(ICSI)
d dn(1/d) dn(1/d)/n d-dim. tourus
Smallworlds
3 log2(n) log2(n)/n Cube connected cycle
Viceroy 3 log(n) log(n)/n Butterfly
![Page 8: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/8.jpg)
9
Locality
Suppose a new DHT ensures each object will be found in 4 hops.
So a lookup could begin in Boston and go to Brazil, New Zealand, France and finally New York.
For some applications this is not a desired outcome.
![Page 9: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/9.jpg)
10
Network models
The Internet– Fully connected weighted graph– Weight = ping latency (?)
Internet with geometric coordinates– Distance = geographic distance
Mobile network– Geometric space with limited transmission range
Arbitrary graphs
![Page 10: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/10.jpg)
11
Distortion
Let c(s, t) be the distance from s to t Let s=x1 x2 …xk=t the route from s to t
Distortion is the ratio betweenc(x1, x2) + … + c(xk-1, xk)andc(s, t)
![Page 11: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/11.jpg)
12
The Model Cost function c that forms a metric
– c(x,y) ≥ 0 (positive), c(x,x)=0 (reflexive), c(x,y)=c(y,x) (symmetric)
– Triangle inequality: c(x,y) + c(y,z) ≥ c(x,z)
Minimal distance between peers is 1. N(x,r) denote the set of nodes at distance <r
from x. Growth Bounded Metric:
Actually, assume uniform density first
r2r rxNrxN ,2,
![Page 12: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/12.jpg)
13
LAND origins and related work
Based on the scheme of Plaxton, Rajaraman, Richa. “Accessing nearby copies of replicated objects in a distributed environment.” Theory of Computing Systems, 1999.
PRR ensures that the expected distortion is constant.
Tapestry and Pastry DHT’s are both based on the basic static PRR scheme. They enhance PRR by handling dynamic changes in the network.
More..
![Page 13: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/13.jpg)
14
LAND architecture
A set of objects A. Objects can be stored on any
node.– Multiple nodes can keep a replica
of each object. Uniformly distributed hash
function h(A). Uniformly selected node
identifiers Nodes keep
transient routing information about objects.
Resides hereH
ashe
d-ho
me
here
?
![Page 14: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/14.jpg)
15
Identifiers and links
Each node has n=log(N) identifier 2-bit digits Node a1a2…an has n links
– Link k `fixes’ k’th digit– Connects to closest node with identifier
a1a2…ak-1[0-3]*
Link k property:– Found with probably 1/4k
– Expected to be found within a ball with 4k nodes– Goes to distance 2k
![Page 15: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/15.jpg)
16
Publish and lookup
Prefix-routing: fix one digit at a time Publish object A at node t:
– Leave reference “A;t” at each node en-route
Lookup A: route until reference to A found
Route distance: At most network diameter = 21 + 22 + … + 2n
![Page 16: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/16.jpg)
17
Example of Publish:
t w1 w2
w3
node t publishes Objh(Obj)=101t=***w1=1**w2=10*w3=101
![Page 17: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/17.jpg)
18
Problem 1:
Link distance could be more than expected
Solution: Emulate a shadow node– A shadow node that fixes digit k has links
fixing digit k+1
– If a1a2..ak-1d* not found at distance 2k then look for a1a2..ak-1d[0-3]* and within 2(k+1)
![Page 18: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/18.jpg)
19
Expected O(n) number of shadow nodes
– Probability of emulating link k at most (1-1/4k) 4^k < e-1
– Bk+i – expected number of shadow nodes with prefix k+i
– E[Bk+i] = E[Bk+i|Bk+i-1] = 2e-1E[Bk+i-1] = (2/e) i
– Total expected number of shadow nodes starting with link k is constant
![Page 19: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/19.jpg)
20
Problem 2:
Unbounded distortion; if target is very close, price is too high
Solution: `publish’ links– Step k in a publish route places reference
in appropriate nodes in bigger neighborhood
![Page 20: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/20.jpg)
21
Publish links
Denote 2k ≥ c(s,t)– k is the first such index
Step k of lookup route is at most 2(k+1) away (denote xk)
Step k of publish route is at most 2(k+1) away (denote wk)
c(xk , wk) ≤ 3*2(k+1)
Publish a k’th step to all nodes within distance 2(k+1)+2 with identifer matching k-prefix , so xk will contain a reference to wk
![Page 21: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/21.jpg)
22
Example of Publish:
t w1 w2
w3
node t publishes Objh(Obj)=101t=***w1=1**w2=10*w3=101
For example:w1 publishes to nodes with id=1* within a distance proportional to distance(t, w1)
![Page 22: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/22.jpg)
23
x0
Lookup Algorithm
t w1 w2
w3
x x1 x2 x3
x4
![Page 23: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/23.jpg)
24
Distortion
Route from s to xk
Plus distance from xk to wk
Plus route from wk to t
constant factor over distance from s to t
Distortion can be made close to 1 by increasing range of publish links
![Page 24: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/24.jpg)
25
Summary of LAND properties
Guaranteed (small) constant distortion Expected logarithmic node-degree Simple analysis Amenable to dynamic deployment
![Page 25: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/25.jpg)
26
Dynamic maintenance
Find closest node to x:– From any node y, let Sn = {y}– Recursively, set Sk-1 = closest node among
incoming prefix-(k-1) links into Sk
– Closest is S1
Correctness:– Let s be closest to x; route from s to y– Step k is within 2(k+1) distance– If Sk is closest to x with k-prefix match to Sn is within
2(k+1) distance, then so is Sk-1
![Page 26: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/26.jpg)
27
Neighbor finding
From closest, route k steps, then back-track incoming links two steps, to find prefix-(k-2) links
Correctness:– Node k is 2(k+1) distance away– Incoming links are 2k+2 away, covering 2(k-2)
away
![Page 27: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/27.jpg)
28
Back to growth-bound model N(x,r) denote the set of nodes at distance <r
from x. Growth Bounded Metric:
r2r
rxNrxN ,2,
![Page 28: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/28.jpg)
29
Setting node Identifier and Level.
Set a radix B Let M such that BM = N. Id has M digits of radix B. Denote A i (x) as the ball around x with α
Bi nodes (e-α B<1)– A i (x) has constant number of expected
nodes with any specific length-i identifier.
![Page 29: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/29.jpg)
30
Network links
Each node has M initial routers Router in level k has links to routers in level
k+1 :– router u with id a1,a2,…,ak,ak+1…aM and level k,
maintains three types of links:
1. Neighbor – for each digit b in [0…B-1] a link to the closest node with id
beginning with a1,a2,a3,…,ak,b and level k+1 inside the ball A k+1 (u).
2. Publish - a link to all nodes with id beginning with a1,a2,a3,…,ak and
level k+1 inside the ball A k+5 (u).
![Page 30: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/30.jpg)
31
Example of Publish:
t w1 w2
w3
Super-node t publishes Objh(Obj)=abct=***w1=a**w2=ab*w3=abc
For example:w1 publishes to routers with level 2 and id=ab* inside theball A6(w1)
![Page 31: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/31.jpg)
32
Enforcing Locality withShadow Nodes Recall: Neighbor – for each digit b in B a link to the
closest node with id a1,a2,a3,…,ak,b and level k+1. Want all neighbor links of a level k node u to be
inside Ak+1(u). For any b, if no b’th neighbor is in Ak+1(u) then u
emulates a shsadow node v with id a1,a2,a3,…,ak,b and level k+1.
Node u establishes all of this shadow router’s network links. Including v’s neighbor links.
Recursively this process continues until all shadow nodes have all their links either close enough or emulated.
![Page 32: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/32.jpg)
33
Variations and extensions
Two-tier architecture, constant expected node degree
Content Addressable Networks Fault Tolerance
![Page 33: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/33.jpg)
34
Two-hop stretch-3 DHT
Each node v has identifier h(v) – h() has sqrt(N) different values
Node v has links to:– log(N)*sqrt(N) closest nodes, so one of each value w.h.p– All nodes u with h(u)=h(v)
Routing from s to t in two hops:– Find node w with h(w)=h(t)– Find t
Stretch:– c(s, w) + c(w, t) ≤ c(s, t) + 2c(s, t)
![Page 34: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/34.jpg)
35
Analysis:Balls
Recall Ai(x) is the smallest ball around x with α Bi M nodes (e-αB<1).
Suppose y in Ai(x) then:
– Ai(y) in Ai+1(x)
– Ai(x) in Ai+1(y)y
xAi(x)
Ai+1(x)
Ai(y)
Ai+1(y)
![Page 35: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/35.jpg)
36
Proof of Ai(y) in Ai+1(x)
Ai(y) is less than N(2ai(x),x) because it contains the ball Ai(x)
N(2ai(x),y) is less than N(4ai(x),x) by simple distances
N(4ai(x),x) is less than Ai+1(x) due to the growth restriction and the way we chose B
Proving Ai(x) in Ai+1(y) is very similar
![Page 36: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/36.jpg)
37
Growth of Balls
Recall Ai(x) is the smallest ball around x with α Bi nodes (e-αB<1).
Let ai(x) denote the radius of Ai(x)
ai+1(x) ≤ maxgrow ai(x)
ai+1(x) ≥ mingrow ai(x)
Ai(x)2log
2log
Bmingrow
Bmaxgrow
Ai+1(x)
ai+1(x) ai(x)
![Page 37: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/37.jpg)
38
Analysis:Distortion The initial node is x looking for Obj. x0=s is the closest super node. w0=t is the closest super node holding Obj. w0,w1,w2,.. is the sequence of nodes used to publish
OBj. x0,x1,x2,.. is the set of nodes fixing the bits to reach
h(Obj), node xi has level i. Xk is the first node that has a reference to OBj
published by node wk-1. Need to find bound on path x,x0,x1,x2,…,xk,wk-1,wk-2,
…,w2,w1,w0=t compared to c(x,t).
![Page 38: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/38.jpg)
39
Analysis:Distortion
For every i:– xi in Ai+1(s) and wi in Ai+1(t)
– The path from s=x0 to xi is at most
– Similarly for the path from t=w0 to wi
If t in Ak(s) then xk contains a reference to Obj
Distortion is
)(1
21 sa
mingrow
mingrowi
22
31
41 mingrow
mingrow
maxgrowmingrow
![Page 39: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/39.jpg)
40
Analysis:Expected Degree
Expected number of virtual nodes emulated by a node is constant.
Expected number of publish links is constant. Expected degree of regular nodes is
constant. Expected degree of super-nodes is
logarithmic.
![Page 40: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/40.jpg)
41
Expected number of emulated nodes is
The probability that a random node will be a neighbor link is 1/(Bl+1M).
The probability that a neighbor link will be found inside Al+1(u) is
1
1 Be
1
1
11 1 1
kB M
ke
B M
![Page 41: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/41.jpg)
42
Expected number of emulated nodes is
Let bl+i be the number of virtual nodes of level l+i. So bl=1.
E(bl+i | bl+i-1) = bl+i-1 B e-α.
Thus E(bl+i) = E(bl+i-1) B e-α.
By induction: E(bl+i) = (B e-α)i
Number of virtual nodes:
BeBebE
i
i
lMiil 1
1
00
1
1 Be
![Page 42: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/42.jpg)
53
![Page 43: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/43.jpg)
A Generic Scheme for Building Overlay Networks in Adversarial Scenarios
Ittai Abraham (HUJI), Baruch Awerbuch (JHU),Yossi Azar (TAU), Yair Bartal (HUJI),Dahlia Malkhi (HUJI), Elan Pavlov (HUJI)
![Page 44: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/44.jpg)
55
Dynamic Model Suppose the set of node in the network is
dynamically evolving. Peers in the DHT are constantly leaving and
joining the system. Join: a new node wants to join the system, it
initially has access to an existing node. Leave: a node departures from the system,
this departure can either be graceful (performing any necessary cleanup operation) or sudden.
Low degree overlay network helps reduce overhead in the event of join/leave.
This process may cause network imbalance.
![Page 45: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/45.jpg)
56
Coping with Imbalance Solution 1 [FS 01, FSGKS 02]: Assume population is
always in between n and ½n (censorship). Solution 2 [Chord 01, CAN 01]: Execute periodic global
overhaul operations for rebalancing. Problem: global operations are costly and may totally
shut down the service. Impractical for large systems. Solution 3 [Pastry 01, Tapestry 01]: Assume population
change is a random process that maintains the initial randomness.
Problems:– DHT systems may have hot spots, and many nodes entering
may use the same access node.– Failures tend to be correlated.– A malicious adversary may try to disrupt the network by causing
imbalance.
![Page 46: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/46.jpg)
57
Load Balancing Against an Adversary In this work we allow an adversary to
adaptively choose:– The order of join and leave events.– For leave events: which node to remove.– For join events: what is the access node of the
newly added node. Against such an adversary we employ load
balancing upon arrival and departure. After each event the overlay network
executes protocols for rebalancing the network.
![Page 47: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/47.jpg)
58
Problem Statement
Devise an overlay network and join leave protocols, with the following properties:– Efficient decentralized routing.– Low cost for rebalancing join and leave
events against an adversary.
![Page 48: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/48.jpg)
59
Generic Solution for Child-Neighbor Commutative Families
Consider a set of graphs G1, G2, G3, … With mapping pi from Gi+1 on to Gi. Denote child function ci(u)={v| pi(v)=u} Denote neighbor function n(u)={v|
(u,v)E} Child-Neighbor commutative property:
For every u: n(c(u))=c(n(u))
![Page 49: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/49.jpg)
60
The Hypercube as an example The hypercube Gi :has 2^i nodes each node’s
id is a binary string of length i. A node in Gi :has links to the i nodes that
have only one bit different in their id. The child function is c(x)={x0,x1} Example of Child-Neighbor commutative
property for node 10 in G2. n(10)={11,00}, c(n(10))={111,110,001,000} c(10)={100,101},
n(c(10))={000,110,101,001,110,100}
![Page 50: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/50.jpg)
61
The Dynamic Graph For this talk we focus on the Dynamic
Hypercube (in paper de Bruijn, Butterfly). Start with two nodes with id 1, 0. Split: change node with id x into nodes x0, x1. Merge: change two twin nodes x0, x1 into node
with id x. Not all of the nodes n(x) exist in the network. Edges: A node x connects to all the nodes
whose id is a prefix of n(x) or n(x) is a prefix of their id.
For example: Node 110 would link to all nodes in the dynamic graph whose id is a prefix of {010, 100, 111} for instance: 01, 100101, 1000, and 111.
![Page 51: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/51.jpg)
62
Example of split Operations on the Dynamic Hypercube
Start with 0,1. Split 1 to 10,11 Split 10 to 100,101 Split 0 to 00,01 Split 11 to 110,111
1010
11
101
10000
01 110
111
![Page 52: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/52.jpg)
63
Tree View of Dynamic Graphs
Leafs of the tree represent current nodes
Inner nodes in the tree represent nodes that were split
000
111
110
101
00
110 111
001
100
011
010
000 001 010 011 100 101
00
Example: merge of 000, 001 into 00
![Page 53: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/53.jpg)
64
Dynamic Graphs
Goal: efficient routing (logarithmic in number if nodes).
Level of a node – number of bit in identifier (distance from the root).
Global gap – difference between smallest level and biggest level.
Local gap of a node– difference in the levels of the nodes neighbors.
We show that with a logarithmic global gap efficient (logarithmic) routing is possible.
Constant local gap implies a logarithmic global gap.
![Page 54: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/54.jpg)
65
Deterministic Balancing Strategy
Goal: maintain a local gap of 1. Node addition: starting from the access node
using the network links, reach a node with the smallest level and split it.
Node removal: starting form the removed node, reach a node with the biggest level and merge it with its twin.
If the local gap was 1, it will remain 1. Since the global gap is log n, requires to
examine log n nodes at most.
![Page 55: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/55.jpg)
66
Randomized Balancing Strategy Goal: maintain a global gap of O(1) w.h.p. Node addition: Choose log n different locations reach
them using the network routing, and split the node with the smallest level.
Node removal: Choose log n different locations and merge the node with the biggest level.
This process is similar to throwing balls into bins, by choosing log n different bins, adding into the least full bin, and removing from the most full.
Requires to examine log n nodes per operation. Analysis: M. Mitzenmacher, A. Richa and R.
Sitaraman. The power of two random choices: a survey of techniques and results.
![Page 56: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/56.jpg)
67
In the paper
Dynamic de Bruijn, and butterfly networks with logarithmic routing and constant degree.
Logarithmic number of messages sent and constant number of nodes that change state per leave/join event.
Generalize to any Child-Neighbor commutative family of graphs (like grids).
![Page 57: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/57.jpg)
68
Related Work M. Naor and U. Weilder present a generic
way to emulate continuous graphs by discrete graphs [MW - SPAA 03].
Use rapidly mixing random walks on a dynamic de Bruijn network in order to establish probabilistic quorums in dynamic settings [A,Malkhi – DISC 03].
Independently, other de Bruijn based P2P networks: [MW], [D2B: Fraigniaud, Gauron], [Koorde: Kaashoek, Karger]
![Page 58: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/58.jpg)
69
Open Questions Deal with multiple, parallel joins and
leaves without waiting till the network rebalances. Use online analysis to find the rate in which the network remains balanced.
Find ways to avoid network partitions. Find other graphs that are applicable to
the generic construction (random graphs).
![Page 60: LAND: Locality Aware Networks for Distributed Hash Tables Dahlia Malkhi The Hebrew University of Jerusalem Joint work with: Ittai Abraham and Oren Dobzinski](https://reader035.vdocuments.us/reader035/viewer/2022062516/56649d5e5503460f94a3d90b/html5/thumbnails/60.jpg)
71
x0
Lookup Algorithm
t w1 w2
w3
x s=x1 x2 x3
x4