sybase bam overview
TRANSCRIPT
Sybase Confidential 2
Agenda
• Technology Background • Analytic Model
• Architecture • Main Features • Demo
Sybase Confidential 3
Background - Overview
• Business Activity Monitoring (BAM) • Complex Event Processing / Event Stream Processing
• Two approach of CEP/ESP • Real Time Business Intelligence
• Two approach of RTBI
Sybase Confidential 4
Background - BAM
"Business activity monitoring" (BAM) is Gartner's term defining how we can provide real-time access to critical business performance indicators to improve the speed and effectiveness of business operations. Unlike traditional real-time monitoring, BAM draws its information from multiple application systems and other internal and external sources, enabling a broader and richer view of business activities.
Sybase Confidential 5
Background – ESP/CEP
“Event Stream Processing” (ESP) is software technology that allows applications to monitor streams of event data, analyze those events, and act upon opportunities and threats in real time. ESP systems often utilize, or include, event databases and event visualization tools, event-driven middleware, and event processing languages “Complex Event Processing” (CEP) is a key element of ESP that provides language elements that allows applications to express the complex patterns among events it's looking for. CEP provides constructs that include event correlation, event abstraction, event hierarchies, and the ability to express relationships between events such as causality, membership, and timing.
Sybase Confidential 6
Two approach of CEP/ESP – SQL Based Approach I
Some people coming from RDBMS development have extended SQL to provide CEP/ESP. • The SQL processing in traditional RDBMS is “data is static and query is dynamic”.
• The SQL processing in CEP/ESP is “data is dynamic and query is static”.
• Because the event data may be overflow, it is necessary to introduce “time window” to SQL
Sybase Confidential 7
Two approach of CEP/ESP – SQL Based Approach II
SELECT I1.SourceIP As SourceIP, I1.AttackKind As AttackKind, V1.Virus As Virus FROM InIDSAlerts As I1 KEEP 30 SECONDS, InVirusAlerts As V1 KEEP 30 SECONDS WHERE I1.SourceIP=V1.SourceIP
Join
Projection
. . .
. . .
Scan Scan
. . .
I1 KEEP 30 SEC I2 KEEP 30 SEC
I1 I2
Sybase Confidential 8
Two approach of CEP/ESP – Rule Based Approach I
Some people come from integration development have extend rule engine to provide CEP/ESP. Sybase BAM chooses this approach. The key of this approach is to add complex state management and corresponding operator to the traditional rule engine that can support complex event pattern and event correlation.
Sybase Confidential 10
Background – RT BI
“Real time business intelligence” (RT BI) is the process of delivering information about business operations without any latency. While traditional business intelligence presents historical information to users for analysis, real time business intelligence compares current business events with historical patterns to detect problems or opportunities automatically.
Sybase Confidential 11
Two approach of RT BI
• Event driven, Real time Business Intelligence Real time Business Intelligence systems are event driven, and use ESP/CEP techniques to enable events to be analyzed without being first transformed and stored in a date warehouse. This approach is better for BAM. • Real time Data warehouse An alternative approach to event driven architectures is to increase the refresh cycle of an existing data warehouse to update the data more frequently. These real time data warehouse systems can achieve near real time update of data, where the data latency typically is in the rage from minutes to hours out of date. This approach is better for ETL.
Sybase Confidential 12
Analytic Model - Overview
Fields: Abstract states definition. Key, Unbound, Bound, Aggregation
Rules: Intelligence If condition Then action
Actions: Behavior Update, Aggregation, Alert, Timer, SQL, Java Script, Purge
Timers: Scheduler If timer arrive Then action
Binder: Concrete states storage BAMDB, UserDB, RefAM
Sybase Confidential 13
Analytic Model – Processing
Fields Key
Bound Bound Bound
Aggregate
Unbound
Rules Actions • Update
• Aggregate
• SQL
• Alert
• Java Script
• Timer control
• Purge
if…
if…
1. Keys, (some) other field passed into Analytic Model
2. Historical values found based on keys
3. Rules applied to data
4. Actions performed, update data
5. Repeat 3, 4 as needed
6. New values stored
Sybase Confidential 14
Analytic Model – in SOA
Input Fields ----- ----- ----- ----- Output Fields ----- -----
Monitor Service
Fields • Key • Unbound • Bound • Aggregate
Rules / Actions • Update • Aggregate • Send Alert • SQL • Java Script • Timer Control • Purge
Analytic Models / Analytic Objects
Fields Rules / Actions
Fields Rules / Actions
Sybase Confidential 15
Analytic Model - Functionality
• Monitor services interact with multiple Analytic Models, setting key fields to define specific object instance. • Within Analytic Object, multiple rule calls trigger actions that further update object and perform other activities. • Any field set in one Analytic Object is then available to subsequent objects, as determined by the Monitor Service.
• If there is implicate or explicate key fields setting between different Analytic Objects, record the cross correlation of Analytic Objects. • Service output fields may be return result of any field from any Analytic Object.
Sybase Confidential 16
Architecture - Overview
Monitor
Service Editor Monitor Service WSDL
Monitor Command and
Control
Monitor
Analytic Model Editor
Dashboard Business Process
External Client
SOAP, JMS, etc
Monitor Service
BAM-Defined
Database Binding
User Defined Database Binding
SCS Container Analytic Object Access Library
Rules
Timed Event
Daemon
Sybase Confidential 17
Architecture - Components
• BAM Engine § Analytic Object Access Library § BAM Rule Engine § Timed Event Daemon § Monitor Service WSIF Provider
• BAM Tooling § Analytic Model Editor § Monitoring Service Editor
• BAM Web GUI § Monitoring Console § Dashboard
Sybase Confidential 18
Runtime Processing of BAM
Queue
SCS
JMS WSHF
Provider CSB Monitor Service WSIF
Provider
Optimus
Analytic Object Access Library DB
Timed Event Daemon
Sybase Confidential 19
Main Features - Overview
• Complex Event Processing Support • Real Time Business Intelligence Support • Comprehensive Alert Capability • Intuitive Visualization for Monitoring and Analysis • Metadata-Driven Design Tooling
• Service Oriented Architecture Support • High Volume
Sybase Confidential 20
Main Features - Complex Event Processing Support I
• Event-Condition-Action (ECA) model § Event Triggering, Rule Evaluation, Execute Action
• Event Transport/Triggering § JMS, HTTP, Email, File, Timer
• Event Parsing/Transformation § XML, CWF, SOAP
• Event Routing § Body-based, Header-based, Endpoint-based
Sybase Confidential 21
Main Features - Complex Event Processing Support II
• Event States § Stateless, Stateful, Historical
• Event Correlation § Correlate low-level events to high-level event § Key correlation, Cross correlation, History correlation
• Event Reprocess § Take corrective action for closed loop integration
• Complex Event Pattern Support § Based on ECA model + Event States + Event Correlation.
Sybase Confidential 22
Main Features - Real Time Business Intelligence Support I
• Rule-based intelligence § Light-weight BAM Rule Engine (BRE) § Patent-pending Boolean Network Rule Engine (BNRE)
• Analyzing real-time data in the context of historic information § Reference contextual data from ASE, IQ, EII
Sybase Confidential 23
Main Features - Real Time Business Intelligence Support II
• Time windowed aggregation / computation § User-defined computation expression § Extensible Aggregator: Average, Rate, Standard Deviation § Sliding Time Window / Fixed Time Window
• Multi-dimensional analysis support § Based on Event Correlation + Aggregation + Computation
Sybase Confidential 24
Main Features – Comprehensive Alert Capability
• Publish-subscribe model § XML Messages Publish via JMS § Customized Subscription
• Multiple Delivery Target § JMS, JMX, Email
• Alert escalation § Timer, On-demand
• Alert lifecycle § Active, Canceled, Completed, Escalated, Suppressed
Sybase Confidential 25
Main Features - Intuitive Visualization for Monitoring and Analysis
• Dashboard § Visual objects of Key Performance Indicator (KPI) is changed
dynamically as events occur in real time
• Monitoring § Real time event is displayed in tabular forms § Drill-down from high-level event to low-level events
• Alerting § View and resolve alerts
Sybase Confidential 26
Main Features - Metadata-Driven Design Tooling
• Based on Eclipse and EMF (Eclipse Modeling Framework) • Fully integrated and conformed to Sybase WorkSpace
Sybase Confidential 27
Main Features – SOA Support
• BAM is exposed as “Monitoring Service” in Sybase Service Container
Sybase Confidential 28
Main Features - High Volume
• High Performance § BAM engine can process about 2000 messages/sec on a 2
CPU machine
• Linear Scalability § BAM engine is linear scalability § Single BAM DB is linear scalability with CPU number § Multiple BAM DB are linear scalability with machine number
Sybase Confidential 29
Reference
Business Activity Monitoring http://en.wikipedia.org/wiki/Business_activity_monitoring Complex Event Processing http://en.wikipedia.org/wiki/Complex_event_processing Event Stream Processing http://en.wikipedia.org/wiki/Event_Stream_Processing Real-time Business Intelligence http://en.wikipedia.org/wiki/Real_time_business_intelligence BI 2.0: The Next Generation http://www.dmreview.com/article_sub.cfm?articleId=1066763 BAM: Event-Driven Business Intelligence for the Real-Time Enterprise http://www.dmreview.com/article_sub.cfm?articleId=8177 Data Integration—the Foundation of a Robust Enterprise Architecture http://www.informatica.com/company/featured_articles/data_integration_foundation_082004.htm