Results on Out-of-Order Event Processing

Paul Fodor, Darko Anicic, Sebastian Rudolph

PADL 2011, Austin, TX

AIFB

Complex Event Processing

• How to capture events from event sources; and transform, combine, interpret and consume them?

Figure source: Opher Etzion & Tali Yatzkar, IBM Research

• Financial services (high frequency trading)

• Sensor networks

• Real-Time Semantic Web (click stream analysis, processing updates from social Web apps, on-line advertising)

• Extensible formal semantics

• Reasoning over streaming events w.r.t contextual (background) knowledge

• Operators to express complex relationships between events, matching certain temporal, relational or causal conditions

• An inference system for Complex Event Processing

• Data-driven complex event processing

• Combines detection of complex events and reasoning over states

• ETALIS is implemented in Prolog and runs on XSB, YAP, SWI, and SICStus

ETALIS: Complex Event Processing & Stream Reasoning

ETALIS: Language Syntax

ETALIS Language for Events is formally defined by:

• pr - a predicate name with arity n;• t(i) - denote terms; • t - is a term of type boolean; • q - is a nonnegative rational number;• BIN - is one of the binary operators: SEQ, AND, PAR, OR, EQUALS, MEETS, STARTS, or FINISHES.

Event rule is defined as a formula of the following shape:

where p is an event pattern containing all variables occurring in

ETALIS: Interval-based Semantics

ETALIS: Declarative Semantics

a ⊗ b ⊗ c → ce1

((a ⊗ b) ⊗ c) → ce1

Binarizationa ⊗ b → ie1

ie1 ⊗ c → ce1

a :- while_do(a,1).a(1) :- ins(goal(b,a,ie1)).

b :- while_do(b,1).b(1) :- goal(b,a,ie1) del(goal(b,a,ie1)) ie1.⊗ ⊗

ie1 :- while_do(ie1,1).ie1(1) :- ins(goal(c,ie1,ce1)).

c :- while_do(c,1).c(1) :- goal(c,ie1,ce1) del(goal(c,ie1,ce1)) ce1.⊗ ⊗

ce1 :- action1.

Event-driven backward chaining rules

Complex pattern (not event-driven rule)

ETALIS: Operational Semantics

Out-of-order Motivating Example• Hedge fund with independent cooperating agents

– events arrive in an out-of-order

– Received vs. real order of events

– missing complex events due to an out-of-order stream, e.g., ce1

– false positive complex events due to out-of-order events, e.g., ce2

Inference Rule Transformation for Out-of-Order Events

Pruning Outdated Events- Pushed Constraints -

Pruning outdated events- Pattern-Based Garbage Collection -

GC via alarms– partial composed events are saved into memory with time

stamps – the rules are labeled with a rule tag X

• For a given rule the partial goals can be pruned by alarms based on the time stamps

• pruning for:

Evaluation Tests I

Stock market patterns

Evaluation Tests II

Evaluation Tests III

Thank you! Questions…

ETALIS

Open source:

http://code.google.com/p/etalis