GIA: Making Gnutella-like P2P GIA: Making Gnutella-like P2P Systems ScalableSystems Scalable
Yatin ChawatheYatin ChawatheSylvia Ratnasamy, Scott Shenker, Sylvia Ratnasamy, Scott Shenker, Nick Lanham, Lee BreslauNick Lanham, Lee Breslau
The Peer-to-peer PhenomenonThe Peer-to-peer Phenomenon
Internet-scale distributed system– Distributed file-sharing applications
– E.g., Napster, Gnutella, KaZaA
File sharing is the dominant P2P app
Mass-market– Mostly music, some video, software
The ProblemThe Problem
Potentially millions of users– Wide range of heterogeneity
– Large transient user population
Existing search solutions cannot scale– Flooding-based solutions limit capacity
– Distributed Hash Tables (DHTs) not necessarily appropriate
Our Solution: GIAOur Solution: GIA
Scalable Gnutella-like P2P system
Design principles:– Explicitly account for node heterogeneity
– Query load proportional to node capacity
Results:– Gia outperforms Gnutella by 3–5 orders of
magnitude
OutlineOutline
Existing approaches
GIA: Scalable Gnutella
Results: Simulations & Experiments
Conclusion
GnutellaGnutella
Distributed search and download
Unstructured: ad-hoc topology– Peers connect to random nodes
Random search– Flood queries across network
Scaling problems– As network grows, search
overhead increases
P1
P2
P4
P3
who has“madonna”
P 4 has “
madonna-
american-life.mp3”P5
P6
P2 has “madonna-ray-of-light.mp3”
Distributed Hash Tables (DHTs)Distributed Hash Tables (DHTs)
Structured solution– Given a filename, find its location
Can DHTs do file sharing?– Probably, but with lots of extra work:
Caching, keyword searching
Do we need DHTs?– Not necessarily: Great at finding rare files,
but most queries are for popular files
Note: Not questioning the utility of DHTs in general, merely for mass-market file sharing
Other SolutionsOther Solutions
Supernodes [KaZaA]– Classify nodes as low- or high-capacity– Only pushes the problem to a bigger scale
Random Walks [Lv et al]– Forwarding is blind– Queries can get stuck in overloaded nodes
Biased Random Walks [Adamic et al]– Right idea, but exacerbates overloaded-node
problem
OutlineOutline
Existing approaches
GIA: Scalable Gnutella
Results: Simulations & Experiments
Conclusion
GIA: 10,000-foot viewGIA: 10,000-foot view
Unstructured, but take node capacity into account– High-capacity nodes have room for more
queries: so, send most queries to them
Will work only if high-capacity nodes:– Have correspondingly more answers, and
– Are easily reachable from other nodes
Make high-capacity nodes easily reachable– Dynamic topology adaptation
Make high-capacity nodes have more answers– One-hop replication
Search efficiently– Biased random walks
Prevent overloaded nodes– Active flow control
Make high-capacity nodes easily reachable– Dynamic topology adaptation
Make high-capacity nodes have more answers– One-hop replication
Search efficiently– Biased random walks
Prevent overloaded nodes– Active flow control
GIA DesignGIA Design
Query
Dynamic Topology AdaptationDynamic Topology Adaptation
Make high-capacity nodes have high degree (i.e., more neighbors)
Per-node level of satisfaction, S:– 0 no neighbors, 1 enough neighbors
– Function of: Node’s capacity ● Neighbors’ capacities
Neighbors’ degrees ● Their age
– When S << 1, look for neighbors aggressively
Active Flow ControlActive Flow Control
Accept queries based on capacity– Actively allocation “tokens” to neighbors
– Send query to neighbor only if we have received token from it
Incentives for advertising true capacity– High capacity neighbors get more tokens
– Allocate tokens with weighted fair queuing
Practical ConsiderationsPractical Considerations
Query resilience: node death– Periodic keep-alive messages
– Query responses are implicit keep-alives
Determining node capacity– Function of bandwidth and “age” of node
Finding rare items– Bifurcate the random walk every 10 hops…
OutlineOutline
Existing approaches
GIA: Scalable Gnutella
Results: Simulations & Experiments
Conclusion
Simulation ResultsSimulation Results
Compare four systems– FLOOD: TTL-scoped, random topologies
– RWRT: Random walks, random topologies
– SUPER: Supernode-based search
– GIA: search using GIA protocol suite
Metric:– Collapse point: aggregate throughput that
the system can sustain
QuestionsQuestions
What is the relative performance of the four algorithms?
Which of the GIA components matters the most?
How does the system behave in the face of transient nodes?
System PerformanceSystem Performance
0.00001
0.001
0.1
10
1000
0.01 0.1 1Replication Rate (percentage)
Co
lla
ps
e P
oin
t (q
ps
/no
de
)
GIA: N=10,000
SUPER: N=10,000
RW RT: N=10,000
FLOOD: N=10,000
GIA outperforms SUPER, RWRT & FLOOD by many orders of magnitude in terms of aggregate query load
Factor AnalysisFactor Analysis
Algorithm Collapse point
RWRT 0.0005
RWRT+OHR 0.005RWRT+BIAS 0.0015
RWRT+TADAPT 0.001RWRT+FLWCTL 0.0006
Algorithm Collapse point
GIA 7GIA – OHR 0.004GIA – BIAS 6
GIA – TADAPT 0.2
GIA – FLWCTL 2
No single component is useful by itself; the combinationof them all is what makes GIA scalable
Transient BehaviorTransient Behavior
0.001
0.01
0.1
1
10
100
1000
10 100 1000 10000Per-node max-lifetime (seconds)
Co
llap
se
po
int
(qp
s/n
od
e)
replication rate = 1.0%
replication rate = 0.5%
replication rate = 0.1%
Static SUPER
Static RWRT (1% repl)
Even under heavy churn, GIA outperforms the otheralgorithms by many orders of magnitude
DeploymentDeployment
Prototype client implementation using C++
Deployed on PlanetLab:– 100 machines spread across 4 continents
Measured the progress of topology adaptation…
Progress of Topology AdaptationProgress of Topology Adaptation
0
10
20
30
40
50
0 20 40 60 80 100Time (seconds)
Nu
mb
er
of
ne
igh
bo
rs
C=1xC=10xC=100xC=1000x
Nodes quickly discover each other and soon reach their target “satisfaction level”
OutlineOutline
Existing approaches
GIA: Scalable Gnutella
Results: Simulations & Experiments
Conclusion
SummarySummary
GIA: scalable Gnutella– 3–5 orders of magnitude improvement in
system capacity
Unstructured approach is good enough!– DHTs may be overkill– Incremental changes to deployed systems
Status: Prototype implementation deployed on PlanetLab