1 distributed monitoring of peer-to-peer systems by serge abiteboul, bogdan marinoiu docflow...

1

Distributed Monitoring of

Peer-to-Peer SystemsBy

Serge Abiteboul, Bogdan Marinoiu

Docflow meeting, Bordeaux

2

2

Outline The Monitoring Problem & Approach

A language for specifying monitoring tasks: P2PML

P2PMonitor System ActiveXML Stream Algebra Architecture of P2PMonitor Monitoring Plan Generation & Query Rewriting

Focus on Filtering

Reusing running tasks

Work in progress

3

3

The Monitoring Problem P2P systems are:

a popular support for content sharing communities, distributed applications

highly dynamic (intense communications, content changing rapidly, peers come/leave)

and

difficult to observe

Observation is important ! error management & diagnosis

statistics gathering & optimization issues : the « busiest » peer in a network

business applications : billing & quality of service

Web surveillance

4

4

Is it possible to observe & analyse a P2P system ?

Difficult (if not impossible) in a centralized way

Yes, in a distributed manner

5

5

XM

L

strea

m

Approach

Detect events at the (monitored) peer level

Data changes, Web service calls -> alerters

Each event is represented as an XML document

XML Stream

(distributed) XML stream processing system

XML Streams are published

6

6



P2PMonitor System ActiveXML Stream Algebra Architecture of P2PMonitor Monitoring Plan Generation & Query Rewriting

Focus on Filtering


Work in progress

7

7

P2PML statement structureXQuery FLWR flavourFor – maps streams to XML variables

Let – assigns new XML variables

Where – imposes conditions on events (filtering and join criteria)

Return – generates reports / restructures XML

By – specifies publication means : in channels for inside system publication e-mails, Web pages, RSS feeds for outside system publication

8

8

P2PML statement examplefor $c on local: outCOM

let $timeCall := $c.call.time

and $duration := $c.response.time - $timeCall

where

$c.call.method = “GetTemp” and $duration > 10

and $c.call.site = "http://meteofrance.fr"

return <longGetTemp>

<callTime>{$timeCall}</callTime>

<duration>{$duration}</duration>

</longGetTemp>

by channel “QoS:Alerts”

9

9



P2PMonitor System ActiveXML Stream Algebra Monitoring Plans Architecture of P2PMonitor

Focus on Filtering


Work in progress

10

10

ActiveXML Stream Algebra

is the support for monitoring plan representation and the basis for the its optimization :

Distribute the work among the peers

Try to place computation close to data if possible

Try to reduce redundancy

11

11

Scenario

P2PMLQueries

XML streams

12

12

Monitoring Plans

13

13

Architecture of P2PMonitor(1)

Subscription Manager

Alerters (WS Alerter, Database Alerter, RSS Alerter)

Stream Processors Without « storage »: Filter, Restructure, Union With « storage »: Join, Group-By, Duplicate Removal

Publishers E-mail, WebPage, RSS Channel Publisher : a user or another peer may subscribe to it

14

14

Architecture of P2P Monitor(2)

15

15




Focus on Filtering


Work in progress

16

16

Focus on Filtering

Filtering is a crucial operator in stream processing !

E.g., Many users might be interested in events coming from the same source / alerter : bottleneck hazard

Our approach to the problem : two-stage filtering

Reasons:

Attributes of XML document’s root reflect the most important properties of an event

The event’s details can be given intentionnally (ActiveXML style)

17

17

Two-stage filteringA subscription is viewed as a conjunction of simple conditions (e.g., « attribute » = « value ») and « more difficult » XPath queries

1st data structure regroups the (ordered) simple conditions of all the subscriptions by commonalities (Atomic Event Set structure)

2nd data structure regroups XPath queries of all the subscriptions (path –based indexing YFilter style – using NFA)

On a XML document:

1st stage: read the root, evaluate AES, detect the « difficult » XPath queries that remain to be evaluated

2nd stage (if needed): adapt the second structure and evaluate the set of XPath queries on the body of the XML document (if necessary execute Web service calls).

The output is the set of the subscriptions « hit » by the XML document

18

18




Focus on Filtering

Reusing streams / running tasks

Work in progress

19

19


Optimization by trying to avoid redundancy

Before building new operators (and streams), try to discover useful ones

Stream representation in XML:

Stream Definition Database – description of available streamsDistributed, not centralized (avoid bottlenecks)Implemented using KadoP – index and repository system over

a DHT

20

20

Stream replication and equivalence(1)Streams can be replicated between peers

With two similar operators on two replicas of the same stream, we obtain two equivalent streams

Replication can be represented in the Stream Definition Database

Stream Equivalence is difficult to detect

21

21

Algorithm for discovering useful streams It uses XPath queries on the Stream Definition Database:

E.g. for identifying the output stream of alerter inCOM :

/Stream[@PeerId=$p1][Operator/inCOM] ->(S1, P1)

It goes bottom-up

on the query tree

E.g.,

JoinP(σF(inCOM@P1),

outCOM@P2)

(S5, P1)

22

22




Focus on Filtering

Reusing streams / running tasks

Work in progress

23

23

Work in progress (1)Link with Incremental View Maintenance

Defining a monitoring task by a tree-pattern query on an active document with streams - powerful way of expressing complex monitoring tasks (difficult to express directly in P2PML)

24

24

Work in progress (2)Introducing explictly the « time » in P2PML -possible impact on P2PMonitor performance (reactivity) and resource consumption (needed storage)

E.g.

for $e1 on P1:inCOM, $e2 on P2:outCOMwhere $e1.timeEvent > $e2.timeEvent +25

…

Queries on traces obtained by P2PMonitor – diagnosis, detecting patterns of evolution for the monitored system

E.g. Trace = I1,I2…In - instances of a document

For each new order detected in instance Ik, there is a payment present in one of the following instances

25

25

Thank you very much!

1 distributed monitoring of peer-to-peer systems by serge abiteboul, bogdan marinoiu docflow...

Documents