A Rule-Based Language for Complex Event Processing and Reasoning

Darko Anicic, Paul Fodor, Sebastian Rudolph, Roland Stühmer, Nenad Stojanovic, Rudi StuderThe 4th International Conference on Web Reasoning and Rule Systems, Bressanone/Brixen, Italy

Agenda

Introduction, Motivation

ETALIS Language for Events• Syntax;

• Semantics;

• Experimental Results;

Conclusion.

Complex Event Processing

How to capture events from event sources; and transform, combine, interpret and consume them?

Figure source: Opher Etzion & Tali Yatzkar, IBM Research

• Financial services (high frequency trading);• Sensor networks (wireless and mobile networks);• Real-Time Web (click stream analysis, processing updates from

social Web apps, on-line advertising).

Motivation

Efficient Complex Event Processing (timeliness & large volume of data are dimensions of concern)

J. Agrawal, Y. Diao, D. Gyllstrom, and N. Immerman. Efficient pattern matching over event streams. In SIGMOD (2008).

Y. Mei and S. Madden. Zstream: a cost-based query processor for adaptively detecting composite events. In SIGMOD (2010)

A. J. Demers, J. Gehrke. Cayuga: A general purpose event monitoring system. In CIDR, 2007.

N. H. Gehani, Narain H. Composite event specification in active databases: Model & implementation. In VLDB, 1992.

Current CEP systems, based on DSMS provide on the-fly analysis of data streams, but cannot combine streams with evolving knowledge, and they cannot perform reasoning tasks.

Motivation

Formal & knowledge-based processing of events(detect events, context, situation and reason about them)

F. Bry and M. Eckert. Rule-based composite event queries: The language XChangeEQ and its semantics (2007);

A. Paschke, A. Kozlenkov, and H. Boley. A homogenous reaction rules language for complex event processing (2007);

G. Lausen, B. Ludäscher, W. May. On Active Deductive Databases: The Statelog Approach (1996);

E. Behrends, O. Fritzen, W. May, and F. Schenk. Combining ECA rules with process algebras for the Semantic Web. (2006);

Incremental Reasoning on Streams and Rich Background Knowledge, D. F. Barbieri, D. Braga, S. Ceri, E. Della Valle, and M. Grossniklaus (2010).

Detection of complex events based on an event-driven and event-at-a-time (incremental) evaluation remains a challenge.

Our Approach

Knowledge-based processing of events

Efficient CEPw.r.t timelinessand data volume

ETALISbased on: Event-driven Backward Chaining Rules

• ETALIS is an inference system for Complex Event Processing; • Formal semantics to guarantee well defined behaviour; • Deductive rules to express complex relationships between events,

matching certain temporal, relational or causal conditions;• Reasoning over streaming events w.r.t contextual (background)

knowledge, a current state etc.;• ETALIS suitable for: enrichment of events with background

information; detection of more complex situations and intelligent recommendations in real-time; or for accomplishing complex event classifications, clustering, and filtering.

Agenda

Introduction, Motivation

ETALIS Language for Events• Syntax;

• Semantics;

• Experimental Results;

Conclusion.

ETALIS: Language Syntax

ETALIS Language for Events is formally defined by:

• pr - a predicate name with arity n;• t(i) - denote terms; • t - is a term of type boolean; • q - is a nonnegative rational number;• BIN - is one of the binary operators: SEQ, AND, PAR, OR,

EQUALS, MEETS, STARTS, or FINISHES.

Event rule is defined as a formula of the following shape:

where p is an event pattern containing all variables occurring in

ETALIS: Interval-based Semantics

ETALIS: Declarative Semantics

ETALIS: Declarative Semantics

ETALIS: Declarative Semantics

ETALIS: Operational Semantics (SEQ)

a SEQ b SEQ c → ce1

((a SEQ b) SEQ c) → ce1

3. Binarization

2. Decoupling

a SEQ b → ie1

ie1 SEQ c → ce1

4. Event-driven backward chaining rules

1. Complex pattern (not event-driven rule)

⊗⊗

a b c d

e1 e2 e3⊗

action1

action2

action3

⊗⊗e1 e2

e4

e3

⊗ ⊗⊗

e5

action1

action2

action3

a b c d

Evaluation Tests I

Figure 3: Sequence - (a) Throughput (b) Throughput vs. Predicate Selectivity (c) Throughput vs. Workload Change

Test patterns:

Throughput vs. Stream Size (Sequence)

0

5

10

15

20

25

30

35

25 50 75 100

Event stream size x 1000

Th

rou

gh

pu

t (1

000

x E

ven

ts/S

ec)

Esper 3.3.0 P-SWI P-YAP

Throughput vs. Predicate Selectivity (Sequence)

0

50

100

150

200

250

300

350

400

450

500

10% 50% 100%

Predicate selectivity

Th

rou

gh

pu

t (1

000

x E

ven

ts/S

ec)

Esper 3.3.0 P-SWI P-YAP Throughput vs. Workload Change (Sequence)

0

5

10

15

20

25

30

25 50 75 100

Event stream size x 1000

Th

rou

gh

pu

t (1

000

x E

ven

ts/S

ec)

Esper 3.3.0 P-SWI P-YAP

Intel Core Quad CPU Q9400 2,66GHz, 8GB of RAM, Vista x64; ETALIS on SWI Prolog 5.6.64 and YAP Prolog 5.1.3 vs. Esper 3.3.0

Evaluation Tests II

Figure 4: Negation - (a) Throughput vs. Selectivity (b) Throughput vs. Workload Change (c) Conjunction - Throughput

Throughput vs. Negation Selectivity

0

5

10

15

20

25

30

35

40

45

10% 50% 100%

Selectivity of negated events

Th

rou

gh

pu

t (1

000

x E

ven

ts/S

ec)

Esper 3.3.0 P-SWI P-Yap

Throughput vs. Workload Change (Negation)

0

5

10

15

20

25

30

35

40

45

50

25K 50K 75K 100K

Event stream size

Th

rou

gh

pu

t (1

000

x E

ven

ts/S

ec)

Esper 3.3.0 P-SWI P-Yap

Throughput vs. Workload Change (Conjunction)

0

5

10

15

20

25

30

35

40

25K 50K 75K 100K

Event stream size

Th

rou

gh

pu

t (1

000

x E

ven

ts/S

ec)

Esper 3.3.0 Etalis - SWI Etalis - Yap

Evaluation Tests III

Figure 5: (a) Disjunction-Throughput (b) Transitive Closure (c) Plan Sharing

Throughput vs. Workload Change (Disjunction)

0

5

10

15

20

25

30

25K 50K 75K 100K

Event stream size

Th

rou

gh

pu

t (1

000

x E

ven

ts/S

ec)

Esper 3.3.0 P-SWI P-Yap

Throughput for Transitive Closure

0

20

40

60

80

100

2500 5000 7500 10000

Event stream size

Th

rou

gh

pu

t (1

00 x

Even

ts/S

ec)

Esper 3.3.0 P-SWI P-Yap

Computation Sharing (Sequence)

0

5

10

15

20

25

30

1 8 16

Number of queries

Th

rou

gh

pu

t (1

000

x E

ven

ts/S

ec)

Esper 3.3.0 P-SWI P-Yap

Agenda

Introduction, Motivation

ETALIS Language for Events• Syntax;

• Semantics;

• Experimental Results;

Conclusion.

Conclusion: A Common Framework for Event Processing in ETALIS

18

ETALISA

deductive rule

framework with event-

at-time processing

Integration with the domain

knowledge and

databases

Reasoning over

streaming data to detect

complex situations

ETALIS: A Common Framework for Event Processing

A general and

extensible framework

for CEP

Thank you! Questions…

ETALIS

Open source:

http://code.google.com/p/etalis

See also our live DEMO at:

http://krake26.perimeter.fzi.de:8080/etalis

Darko.Anicic@fzi.de