Download - Vitus Masters Defense
![Page 1: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/1.jpg)
PlanetenWachHundNetz
Instrumentation Infrastructure for PlanetLab
Vitus Lorenz-Meyer
![Page 2: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/2.jpg)
Peer-to-Peer
• Distributed on open internet• All participants both receive & provide
services to/from others• Not centrally administered• Membership changes over time (churn)• Example: file sharing (napster, gnutella,…)
Any node can publish a named file Any node can obtain file from another node who
has it. Range of strategies to find nodes containing
desired contentVitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso2
Problem
![Page 3: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/3.jpg)
Rel. Work
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso3
The Problem
• P2P systems hard to tune, requires understanding of complex behavior Requires instrumentation & analysis
• Many P2P systems constructed without scalable instrumentation infrastructure Frequently done in ad-hoc manner
Data transmitted to single collection & analysis node Inadequate for understanding behavior of large systems
of many (hundreds to MILLIONS) of nodes• My work: development of a flexible tool to
enable scalable instrumentation algorithms, data structures
P2P
![Page 4: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/4.jpg)
Related work (1 of 2: cousins)
• Distributed Database Mngmt. Systems Select data at sources Optimize joins (run near to sources…) Commercially used in non-p2p configurations P2P (research): PIER, Sophia
• Sensor Networks Unmanaged radio-connected nodes
provide “network” of surveilance SQL; Compiled into a 3-step process Software communicates through same mechanism IrisNet, TAG
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso4
![Page 5: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/5.jpg)
Related work (2of2: Siblings)
• Aggregation Overlays Information collection subsystem Nodes provide information tuples
Internal aggregation language Computed using parallel prefix of pre-defined
assoc/comm ops Astrolabe, SDIMS, SOMO
• Google’s MapReduce Data selection & aggregation in distributed
system User provides “map” and “reduce” program
Not fully p2p (resource mgmt. overlay)
![Page 6: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/6.jpg)
High-level Approach
• User specifiable programs like MapReduce • Split data collection into 3 ‘phases’
1. Generate values on all nodes2. Pairwise aggregation throughout system3. Evaluate results
• emit measured vals (val,num=1)• Aggregate: (val1+val2,num1+num2)• Evaluate (avg) (val / num)
Easy to use: user provides 3 programs (scripts)
Impl.Rel. Work
![Page 7: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/7.jpg)
Illustration of Binary Aggregation Impl.Rel. Work
![Page 8: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/8.jpg)
Why is this hard in P2P?
• Problem: membership churn• Nodes continuously enter & leave
system• Nobody in charge (p2p)
• Nobody knows membership list!
Exposes following challenges1. Finding all participating nodes2. Constructing an (appx) balanced tree
Impl.Rel. Work
![Page 9: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/9.jpg)
Building Structure Upon Anarchy: Key Based
Routing
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso9
GoalAppr.
02160
2158
2159
2159 + 2158
![Page 10: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/10.jpg)
“Chord” Routing
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso10
GoalAppr.
![Page 11: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/11.jpg)
Chord lookup
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso11
GoalAppr.
![Page 12: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/12.jpg)
Building a tree upon KBR
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso12
GoalAppr.
a aa b
b
d ea b d
e
g if hd ea b
f
g
h
i
![Page 13: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/13.jpg)
Building a tree: FTT & KBT
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso13
GoalAppr.
FTT: finger-based tree• Operation associated with a “target” node• Systems send data to finger closest to target• Ambiguous
Depends on all nodes’ fingertables• Tree useful only for aggregation
• KBT: Maps tree on key-space Operation associated w/ target node System/tree-node mapping:
Node assigned to node w/ nearest key
Non-ambiguous Tree useful for both dissemination &
aggregation Single, global tree
![Page 14: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/14.jpg)
Our Structure
101…
111…101…
111…101… 110…100…
001…
011…001…
011…001… 010…000…
101…
011…
001…
110…
As 001..!
GoalAppr.
• KMR: Subset of KBT, rooted at specific node• One tree / root
Better load-balancing• Tree fully determined by set of active nodes and root
![Page 15: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/15.jpg)
Implementation details
• PWHN-Server layered on FreePastry• PWHN-Client connects to PWHN-Server and
makes query• Callee builds tree making itself root
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso15
Goal
SS S
SS
S
S
S
C
Appr.
![Page 16: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/16.jpg)
Our Goal
• Develop toolkit for data collection/aggregation in P2P networks Useful for PlanetLab-community
• Extend MR’s model to P2P K.I.S.S.
Users provide programs for gen/agg/eval• Use techniques from P2P
Construct aggregation tree upon key-based-routing
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso16
ExampleImpl. details
![Page 17: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/17.jpg)
Example (1)
• First implementation: Script version, flat, to test approach
• Example 1: Overall average system load Gen emits (1,<1load>,<5load>,<15load>) for each server Agg adds all numbers Eval divides last 3 numbers by first to get average
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso17
EvaluationGoal
![Page 18: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/18.jpg)
Example (2)
• PWHN client (Java) Can start and stop
server Used for specifying
all programs and parameters (Servers, username for flat, method)
Front-end for connecting to servers and making query
Allows saving and graphically representing result
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso18
EvaluationGoal
![Page 19: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/19.jpg)
Example (3)
• Graphing of queried results
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso19
EvaluationGoal
Bar Chart Color bubbles on world map
![Page 20: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/20.jpg)
Example (4)
Graphing of tree
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso20
EvaluationGoal
Graphing of paths of the query
![Page 21: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/21.jpg)
Evaluation
• Minimize disruption Minimize incoming
bytes to client
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso21
SynopsisExamples
•More efficientLower average fan-in of aggregation tree
![Page 22: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/22.jpg)
Evaluation: Fern
• Global Update latency histogram
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso22
SynopsisExamples
10 clients 701 clients
![Page 23: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/23.jpg)
Summary
• PWHN - Instrumentation toolkit Extends MR’s model to P2P Uses P2P techniques (DHTs) Combines FTT and KBT to be more efficient Conclusion: Useful tool that is more efficient
than to build infrastructure into software What did I do?
Survey of systems that provide aggregation in dynamic networks
Classification and naming of aggregation trees upon DHTs
Design and implementation of my own tool (KMR/PWHN)
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso23
Examples
![Page 24: Vitus Masters Defense](https://reader033.vdocuments.us/reader033/viewer/2022061104/541445138d7f72fe698b45b9/html5/thumbnails/24.jpg)
Questions
Vitus Lorenz-Meyer: Thesis defense University of Texas @ El Paso24
Synopsis