![Page 1: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/1.jpg)
A Sybil-Proof Distributed Hash Table
Chris Lesniewski-Laas M. Frans Kaashoek MIT
28 April 2010 NSDI
http://pdos.csail.mit.edu/whanau/slides.pptx
![Page 2: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/2.jpg)
Distributed Hash Table
• Interface: PUT(key, value), GET(key)→value • Route to peer responsible for key
GET( sip://alice@foo )
PUT( sip://alice@foo, 18.26.4.9 )
![Page 3: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/3.jpg)
The Sybil aBack on open DHTs
• Create many pseudonyms (Sybils), join DHT • Sybils join the DHT as usual, disrupt rouFng
Brute‐force aBack Clustering aBack
![Page 4: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/4.jpg)
Sybil state of the art 20
01
2002
2003
2004
2005
2006
2007
2008
2009
2010
P2P mania!
Chord, Pastry, Tapestry, CAN
The Sybil ABack [Douceur], Security ConsideraFons [Sit, Morris]
Restricted tables [Castro et al]
BFT [Rodrigues, Liskov]
SPROUT, Turtle, Bootstrap graphs
Puzzles [Borisov]
CAPTCHA [Rowaihy et al]
SybilLimit [Yu et al]
SybilInfer, SumUp, DSybil
(This work)
P2P mania!
![Page 5: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/5.jpg)
ContribuFon
• Whānau: an efficient Sybil‐proof DHT protocol – GET cost: O(1) messages, one RTT latency
– Cost to build rouFng tables: O(√N log N) storage/bandwidth per node (for N keys)
– Oblivious to number of Sybils!
• Proof of correctness • PlanetLab implementaFon
• Large‐scale simulaFons vs. powerful aBack
![Page 6: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/6.jpg)
Division of labor
• ApplicaFon provides integrity • Whānau provides availability
• E.g., applicaFon signs values using private key • Proc GET(key):
UnFl valid value found: Try value = LOOKUP(key) Repeat
![Page 7: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/7.jpg)
Approach
• Use a social network to limit Sybils – Addresses brute‐force aBack
• New technique: layered iden4fiers – Addresses clustering aBacks
![Page 8: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/8.jpg)
• SETUP: periodically build tables using social links • LOOKUP: use tables to route efficiently
Two main phases
SETUP LOOKUP
Social Network RouFng Tables
key
value
key value
PUT(key, value)
PUT Queue
![Page 9: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/9.jpg)
Social links created
![Page 10: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/10.jpg)
Social links maintained over Internet
![Page 11: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/11.jpg)
Sybil region
Social network Honest region
…
ABack edges
![Page 12: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/12.jpg)
Random walks c.f. SybilLimit [Yu et al 2008]
![Page 13: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/13.jpg)
Building tables using random walks c.f. SybilLimit [Yu et al 2008]
What have we accomplished?
• Small fracFon (e.g. < 50%) of bad nodes in rouFng tables
• Bad fracFon is independent of number of Sybil nodes
![Page 14: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/14.jpg)
SETUP LOOKUP
Social Network RouFng Tables
key
value
key value
PUT(key, value)
PUT Queue
![Page 15: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/15.jpg)
RouFng table structure
• O(√n) fingers and O(√n) keys stored per node • Fingers have random IDs, cover all keys WHP • Lookup: query closest finger to target key
Finger tables: (ID, address)
Key tables: (key,value)
Keynes
Aardvark Zyzzyva
Kelvin
![Page 16: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/16.jpg)
From social network to rouFng tables
• Finger table: randomly sample O(√n) nodes • Most samples are honest
ID IP address
![Page 17: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/17.jpg)
A B
C
D
E
F
G
H
I
J
K
L M N O
P
Q
R
S
T
U
V
W
X
Y
Z
Honest nodes pick IDs uniformly
Plenty of fingers near key
![Page 18: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/18.jpg)
A B
C
D
E
F
G
H
I
J
K
L M N O
P
Q
R
S
T
U
V
W
X
Y
Z
Sybil ID clustering aBack
[HypotheFcal scenario: 50% Sybil IDs, 50% honest IDs]
Many bad fingers near key
![Page 19: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/19.jpg)
A B
C
D
E
F
G
H
I
J
K
L M N O
P
Q
R
S
T
U
V
W
X
Y
Z A
B C
D
E
F
G
H
I
J
K
L M N O
P
Q
R
S
T
U
V
W
X
Y
Z
Honest layered IDs mimic Sybil IDs Layer 0 Layer 1
![Page 20: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/20.jpg)
Every range is balanced in some layer
A B
C
D
E
F
G
H
I
J
K
L M N O
P
Q
R
S
T
U
V
W
X
Y
Z A
B C
D
E
F
G
H
I
J
K
L M N O
P
Q
R
S
T
U
V
W
X
Y
Z
Layer 0 Layer 1
![Page 21: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/21.jpg)
A B
C
D
E
F
G
H
I
J
K
L M N O
P
Q
R
S
T
U
V
W
X
Y Z
A B
C
D
E
F
G
H
I
J
K
L M N O
P
Q
R
S
T
U
V
W
X
Y Z
Two layers is not quite enough Layer 0 Layer 1
RaFo = 1 honest : 10 Sybils
RaFo = 10 honest : 100 Sybils
![Page 22: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/22.jpg)
Log n parallel layers is enough
• log n layered IDs for each node • Lookup steps:
1. Pick a random layer
2. Pick a finger to query 3. GOTO 1 unFl success or Fmeout
A B C
D E F G H I J
K L M N O P
Q R S T U V W
X Y Z
A B C
D E F G H I J
K L M N O P
Q R S T U V W
X Y Z
A B C
D E F G H I J
K L M N O P
Q R S T U V W
X Y Z
A B C
D E F G H I J
K L M N O P
Q R S T U V W
X Y Z
Layer 0 Layer 1 Layer 2 Layer L
…
![Page 23: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/23.jpg)
Main theorem: secure DHT rouFng
If we run Whānau’s SETUP using:
1. A social network with walk length = O(log n) and number of aBack edges = O(n/log n)
2. RouFng tables of size Ω(√N log N) per node
Then, for any input key and all but εn nodes:
• Each lookup aBempt (i.e., coin flip) succeeds with probability Ω(1)
• Thus GET(key) uses O(1) messages (expected)
![Page 24: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/24.jpg)
EvaluaFon: Hypotheses
1. Random walk technique yields good samples
2. Lookups succeed under clustering aBacks
3. Layered idenFfiers are necessary for security
4. Performance scales the same as a one‐hop DHT
5. Whānau handles network failures and churn
![Page 25: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/25.jpg)
Method
• Efficient message‐based simulator – Social network data spidered from Flickr, Youtube, DBLP, and LiveJournal (n=5.2M)
– Clustering aBack, varying number of aBack edges
• PlanetLab implementaFon
![Page 26: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/26.jpg)
Escape probability
0
0.2
0.4
0.6
0.8
1
0 10 20 30 40 50 60 70 80
Random walk length
2M aBack edges
200K aBack edges
20K aBack edges
[Flickr social network: n ≈ 1.6M, average degree ≈ 9.5]
![Page 27: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/27.jpg)
Walk length tradeoff
0
0.2
0.4
0.6
0.8
1
0 10 20 30 40 50 60 70 80
Random walk length
2M aBack edges
200K aBack edges
20K aBack edges
Clumpiness
[Flickr social network: n ≈ 1.6M, average degree ≈ 9.5]
![Page 28: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/28.jpg)
Whānau delivers high availability
0
10
20
30
40
100 1000 10000 100000 1000000
Med
ian lookup
messages
Table size
2M aBack edges (>n)
200K aBack edges
20K aBack edges
No aBacker
[Flickr social network: n ≈ 1.6M, 3√n ≈ 4000]
3√n
![Page 29: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/29.jpg)
Everything rests on the model…
…
![Page 30: A Sybil-Proof Distributed Hash Table · Sybil state of the art 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 P2P mania! Chord, Pastry, Tapestry, CAN](https://reader034.vdocuments.us/reader034/viewer/2022051920/600cb19291d50473623d5952/html5/thumbnails/30.jpg)
ContribuFons
• Whānau: an efficient Sybil‐proof DHT – Use a social network to filter good nodes – Resist up to O(n/log n) aBack edges – Table size per node: O(√N log N)
– Messages to route: O(1)
• Introduced layers to combat clustering aBacks