distributed search with rendezvous search systems

Distributed Search withRendezvous Search SystemsChristof Leng

Dipl.-Inform. Christof LengDatabases & Distributed SystemsDepartment of Computer ScienceTechnische Universität Darmstadthttp://[email protected]

Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"2

Distributed Search: A Traditional View

| 27.01.12 |

Implement distributed key-value lookup e.g., a distributed hash table (DHT) scalable & simple

Build everything else on top of it keyword search, XPath, range queries map every operation to a number of lookups

Downside: implement every query language yourself query processing not portable between DHTs unforeseen side-effects & interactions

Experts needed for every application application domain P2P networking information retrieval

Application

Developer

P2P Networking Expert

Information Retrieval

Expert

… need sophisticated queries …


Rendezvous Search

“Every query will meet each data item somewhere in the network” don’t worry how it gets there black box approach

Implement the match functionality as if it was local re-use existing libraries

Advantages: separation of concerns flexible scalable (O(√n) search cost) robust (O(√n) replicas)

| 27.01.12 |

Application

Developer

Rendezvous Search System

Information Retrieval

Library

… need sophisticated queries …

glue code


Example Application

| 27.01.12 |

P2P WikiLucene

Keyword Search

Rendezvous Middleware

Overlay Network


Rendezvous Search Genealogy

| 27.01.12 |


Data Center Solutions

| 27.01.12 |


Grid

Pro: simple efficient flexible

Con: not fault-tolerant not size-adaptive not load-adaptive two-dimensional

structure

Reference:L. A. Barroso, J. Dean, and U. Hölzle. Web Search for a Planet: The Google Cluster Architecture. IEEE Micro, 23(2):22–28, 2003.

| 27.01.12 |

Q Q Q Q Q

Data

Query

D

D

D

D

D

D

QRRendezvous Node

Q Q Q Q Q

Data

Query

D

D

D

D

QRRendezvous Node

Q Q Q


ROAR – Rendezvous on a Ring

Pro: simple mapping to 1

dim. size-adaptive load-adaptive fault-tolerant

Con: centralizedor inefficient

Reference:C. Raiciu, F. Huici, M. Handley, and D. S. Rosenblum. ROAR: Increasing the Flexibility and Performance of Distributed Search. In Proceedings of SIGCOMM’09, pages 291–302, 2009. ACM.

| 27.01.12 |


Structured Peer-to-Peer

| 27.01.12 |


Bit Zipper

Pro: z-order mapping DHT-based =

decentralized fast tree-based

replication

Con: relies on DHT’s

reliability no replica maintenance

Reference:W. W. Terpstra, S. Behnel, L. Fiege, J. Kangasharju, and A. Buchmann.Bit Zipper Rendezvous—Optimal Data Placement for General P2P Queries.In EDBT’04 Workshop on Peer-to-Peer Computing and Databases, 2004.

| 27.01.12 |


Deetoo

Pro: 2 rings for projection includes replica

maintenance

Con: maintenance overhead

for 2 rings no responsibility ranges

(probabilistic approach) additional overhead

Reference:T. W. Choi and P. O. Boykin. Deetoo: Scalable unstructured Search built on a Structured Overlay. In HOTP2P - International Workshop on Hot Topics in Peer-to-Peer Systems, Los Alamitos, CA, USA, 2010. IEEE Computer Society.

| 27.01.12 |


Unstructured Peer-to-Peer

| 27.01.12 |


Ferreira et al. – Random Walks

| 27.01.12 |

Pro: very resilient topology proven correctness

Con: probabilistic additional overhead slow & fragile random

walks no topology protocol

Reference:R. A. Ferreira, M. K. Ramanathan, A. Awan, A. Grama, and S. Jagannathan. Search with Probabilistic Guarantees in Unstructured Peer-to-Peer Networks. In Proceedings of P2P’05, pages 165–172, Washington, DC, USA, 2005. IEEE Computer Society.


BubbleStorm

| 27.01.12 |

Pro: resilient & fast size-adaptive load-adaptive topology & replica

maintenance included

Con: probabilistic additional overhead

Reference:W. W. Terpstra, J. Kangasharju, C. Leng, and A. P. Buchmann. BubbleStorm: Resilient, Probabilistic, and Exhaustive Peer-to-Peer Search. In Proceedings of SIGCOMM’07, pages 49–60, New York, NY, USA, 2007. ACM.


Hautakorpi & Schultz

| 27.01.12 |

Pro: unstructured solution on

top of DHT incremental deployment

possible

Con: probabilistic additional overhead fragile

Reference:J. Hautakorpi and G. Schultz. A Feasibility Study of an Arbitrary Search in Structured Peer-to-Peer Networks. In Proceedings of 19th International Conference on Computer Communications and Networks (ICCCN), pages 1–8. IEEE, Aug. 2010.


Semi-Structured Peer-to-Peer

| 27.01.12 |


SplitQuest

| 27.01.12 |

Pro: combines unstructured

resilience with structured efficiency

Con: probabilistic lacks analysis and

prototype evaluation

Reference:P. Lopes and R. A. Ferreira. SplitQuest: Controlled and Exhaustive Search in Peer-to-Peer Networks. In Proceedings of the 9th International Conference on Peer-to-Peer Systems, IPTPS’10, Berkeley, CA, USA, 2010. USENIX Association.


Comparison

Data Center Structured

Name Grid ROAR Bit Zipper Deetoo

Pro simple adaptive DHT add-on maintenance

Con not adaptive centralized maintenance unclear probabilistic

| 27.01.12 |

Unstructured Semi-Structured

Name Ferreira BubbleStorm Hautakorpi SplitQuest

Pro simple resilient & adaptive

increm. DHT add-on efficient & resilient

Con fragile & slow probabilistic probabilistic immature


Q Q Q Q Q

Data

Query

D

D

D

D

QRRendezvous Node

Q Q Q

Rendezvous Search & Cloud Computing

| 27.01.12 |

similar problems similar requirements less research history

could benefit from size adaptivity traffic adaptivity fault tolerance algorithm diversity

Sharding

Key-Value Store Scalability

Replication

D

D

D

Q Q Q M Q QQ


Conclusion

| 27.01.12 |

Rendezvous Search is a powerful abstraction scalable flexible robust

A wide range of solutions is out there data center unstructured P2P structured P2P semi-structured P2P

Ever needed decentralized but powerful search in a project? don’t try to reinvent the wheel!


Thank you!

?Questionshttp://www.bubblestorm.nethttp://www.dvs.tu-darmstadt.de

| 27.01.12 |

distributed search with rendezvous search systems

Documents

itkeyword search

web search

peer computing

general p2p queries

distributed keyvalue

data item

treebased replicationcon

range queriesmap