enabling online analysis of notification services sebastian salvucci - tandil, aug2006 enabling...

Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006

Enabling Online Analysis of Notification ServicesSebastian Salvucci

Advisor: Mariano CiliaCo-Advisor: Ricardo Orosco

Facultad de Ciencias ExactasUniversidad Nacional del Centrode la Provincia de Buenos AiresTandil, August 2006


proposed approach

intro/motivation

architecture

implementation

conclusions/outlook


Pub/Sub Systems intro


NS characteristics intro

• Matching: is the process carried out by event brokers to check messages against subscriptions.• Routing: is in charge of forwarding delivering the messages to the subscribers. Different algorithms (flooding, simple routing, identity routing, covering, merging)• Other features: persistence, priorities, transactions, selectors, caching, expiration, filters, message types, …

Behavior (performance influenced by…)

• Research Projects: REBECA, Siena, Hermes, Jedi, Gryphon, Srcibe• Commercial Products: Tibco Rendezvous, WebSphereMQ, SonicMQ, Fiorano• JMS Standard API specification: Open Source Implementations (OpenJMS, JBossMQ, ActiveMQ, Joram)

Implementations (different features)

Adressing Models• channel-based• subject/topic-based• content-based


Centralized/Distributed intro

Centralized Distributed event broker network


Traditional Analysis Aproaches

Black-box analysis: data are observed and registeredat the clients, when messages are published and/or received

Tracing, log files: add extra source code finding first the right places and then modifying the code to write the relevant data into a text file.

intro

• enormous ad-hoc text files• source code abstraction level

• thick granularity does not allow to determine behavior details.


Motivation/Goal

Motivation

Goal

intro

• Today is difficult to analyze notification services focalizing individuals aspects of the behavior

• Few partials and proprietary solutions

• Should help to improve NS behavior and performance (research/development/tuning)

• Create an NS-independent analysis framework based on observations, metrics definitions and visual representations of them.


proposed approach

intro

architecture

implementation

conclusions/outlook


Proposed Approach proposed approach


Observations proposed approach


Observations (2) proposed approach


Following messages proposed approach


Metrics from observations proposed approach


Metrics Analysis

Online analysis

Post-mortem analysis

Offline analysis: replay of executionsStatic analysis: traditional queries, data mining, etc.

metric values external visualizationtechniques can be configured to represent metrics in different ways

proposed approach


proposed approach

intro/motivation

architecture

implementation

conclusions/outlook


Proposed Architecturearchitecture


Considerations

Visualization • Reusable and configurable visualization techniques• Metrics independence• Multi users (client)

Metrics Composition• Data in movement• Generic data transformation • High-level metrics definition• Post-mortem analysis• Global Time

Behavior Observation• Notification service independence• Influence over the NS under observation• Sending and generation mechanisms performance

Communication• Push-based communication• Many-to-many• Firewalls, security restrictions

Management• Configuration• Components orchestration

architecture


Behavior Observation Layer

• Adopting AOP to avoidintrusive modification ofsource code.

• AOP lets separate and modularize observation logic.

architecture


Behavior Observation Layer (2)

Implementation Alternatives

-static weaving

-dynamic weaving

• Enable/disable observations generation according metrics composition needs

• Sending frequency: (online, offline, time intervals, fixed amount, package size)

architecture


Metrics Composition Layerarchitecture


Compositors

time interval

observations

metric values

Generic transformation over data streams (configurable)

architecture

Implementation alternative:Stream Process Engines


Metrics Definition: Data Warehouse Metaphor

• High-level and intuitive modelling (from static analysis)• Generic and abstract data transformation• Different measures from the same observations• Decoupling metric composition output from visualization• Metadata for stored data

architecture


Metrics Composition

Configuring compositors from metrics definition

architecture


From Data to Visualizationarchitecture


Visualization

Visualization components

Visualization panel (web browser)

Implementation Alternatives

• Applets• DHTML/AJAX• SVG• Flash• VRML/X3D

architecture


Communication

centralized approach pub/sub approach

architecture


Components Orchestration

Test Run Definition• load generation scenario• metrics• visualization techniques

Test Run Recording1. Enable observations2. Deploy compositors (subs)3. Init visualization components (subs)4. Subscribe repository (all)5. Start Load generation

Test Run Reproduction1. Init visualization panel2. Stored data publication

architecture


proposed approach

intro

architecture

implementation

conclusions/outlook


Initial decisisons

HTTP-based publish/subscribe framework

Pushlets Characteristics• subject based• runs in a servlet container• very thin Java client• JavaScript client for browsers• modes stream/polling

Pushlets Limitations• scalability with mode stream• message type

HTTP streamingInstead of closing the HTTP connection after fetching an HTML page, the connection remains open while fresh data is pushed to the client

Communication Data representation

• XML data representation for observations and metric values

XML repository structure

Repository

• Adopting eXist XML native database (XQuery, XUpdate support)

implementation


Reusing Aspects

method signature & parameters

implementation


Compositors: Aggregator filter + primitives

Aggregator Filter

Primitives based filters

Enhancing Filtering in Notication Services

• Pushlet adapters• XPath primitive• Compositors creation (from metrics)• Compositors deploy/undeploy (pub/sub)

Implementations

implementation


Composing metrics: data transformation implementation


Visualization Panel

•Metrics and components from test run definition

implementation


Visualization Components

• Visualization components description.• Mapping between metrics and components

implementation


Components Orchestration implementation

Test Run Recording Test Run Reproduction


proposed approach

intro

architecture

implementation

conclusions/outlook


Conclusions

The utilization of three different kinds of components to observe the behavior(aspects), compose streams of observations (metrics) and graphically represent them (visualization techniques) as well as their loosely-coupled communication through a pub/sub service offer a high flexible framework to enable online analysis of notification services.

conclusions


Conclusions (2)

The use of a repository to store the data produced by aspects and metrics composition supports post-mortem (offline and static) analysis using the same architecture that was developed for online analysis.

conclusions


Conclusions (3)

• Independence of notification service implementations (by associating semantics of observations to behavior description instead source code)

• Not intrusive modification of the source code (by using AOP to generate observations)

• Generic and intuitive modeling of computations over data streams• Metrics decoupled from visualization • Metadata associated to stored executions (by data cube description)

conclusions


Outlook

• Declarative description of observations and automatically generate the aspects involved letting choose different AOP alternatives.

• By seeking and improving the data warehouse metaphor could be possible to associate a complete and unique multidimensional analysis to data streams and historical data.

• Graphical wizards and interfaces upon XML descriptions towards a higher abstraction level for users, offering a complete analysis-oriented application (bookmarks, notes, etc..)

• Other domains (alternative data sources): software analysis, business activity monitoring, applications that rely on pub/sub systems, etc..

conclusions


Muchas Gracias!

questions / comments / feedback

enabling online analysis of notification services sebastian salvucci - tandil, aug2006 enabling...

Documents