esper referenceesper.espertech.com/release-8.3.0/reference-esper/pdf/esper_reference.pdfesper...
Post on 12-Mar-2020
19 Views
Preview:
TRANSCRIPT
-
Esper ReferenceVersion 8.3.0
by EsperTech Inc. [http://www.espertech.com]
Copyright 2006 - 2019 by EsperTech Inc.
http://www.espertech.comhttp://www.espertech.com
-
iii
Preface ......................................................................................................................... xxvii
1. Getting Started ............................................................................................................ 1
1.1. Introduction to Complex Event Processing ........................................................... 1
1.2. Introduction to the Architecture ............................................................................ 1
1.3. Introduction to EPL ............................................................................................. 2
1.4. Compiler Getting-Started ..................................................................................... 3
1.4.1. Compiler - Step One: Setting up the Compiler Classpath ............................ 3
1.4.2. Compiler - Step Two: Provide Information on Input Events .......................... 3
1.4.3. Compiler - Step Three: Compiling EPL ...................................................... 4
1.5. Runtime Getting-Started ...................................................................................... 5
1.5.1. Runtime - Step One: Setting up the Runtime Classpath .............................. 5
1.5.2. Runtime - Step Two: Obtain Runtime ........................................................ 6
1.5.3. Runtime - Step Three: Deploy EPL Compiled Module and Attach a Callback
.......................................................................................................................... 6
1.5.4. Runtime - Step Four: Send Events ............................................................ 7
1.6. Required 3rd Party Libraries ................................................................................ 8
1.6.1. Common Required 3rd Party Libraries ....................................................... 8
1.6.2. Compiler - Required 3rd Party Libraries ..................................................... 8
1.6.3. Runtime - Required 3rd Party Libraries ...................................................... 8
2. Basic Concepts ........................................................................................................... 9
2.1. Introduction ........................................................................................................ 9
2.2. Basic Select ..................................................................................................... 10
2.3. Basic Aggregation ............................................................................................. 11
2.4. Basic Filter ....................................................................................................... 12
2.5. Basic Filter and Aggregation .............................................................................. 13
2.6. Basic Data Window ........................................................................................... 14
2.7. Basic Data Window and Aggregation ................................................................. 15
2.8. Basic Filter, Data Window and Aggregation ........................................................ 17
2.9. Basic Where-Clause ......................................................................................... 17
2.10. Basic Time Window and Aggregation ............................................................... 19
2.11. Basic Partitioned Statement ............................................................................. 20
2.12. Basic Output-Rate-Limited Statement ............................................................... 21
2.13. Basic Partitioned and Output-Rate-Limited Statement ........................................ 22
2.14. Basic Named Windows and Tables .................................................................. 23
2.14.1. Named Windows .................................................................................. 23
2.14.2. Tables ................................................................................................. 25
2.15. Basic Aggregated Statement Types ................................................................. 27
2.15.1. Un-Aggregated and Un-Grouped ........................................................... 27
2.15.2. Fully Aggregated and Un-Grouped ........................................................ 27
2.15.3. Aggregated and Un-Grouped ................................................................ 28
2.15.4. Fully Aggregated and Grouped .............................................................. 28
2.15.5. Aggregated and Grouped ...................................................................... 28
2.16. Basic Match-Recognize Patterns ...................................................................... 29
2.17. Basic EPL Patterns ......................................................................................... 29
-
Esper Reference
iv
2.18. Basic Indexes ................................................................................................. 30
2.18.1. Index Kinds .......................................................................................... 30
2.18.2. Filter Indexes ....................................................................................... 31
2.18.3. Event Indexes ...................................................................................... 35
2.19. Basic Null ....................................................................................................... 35
3. Event Representations ............................................................................................... 37
3.1. Event Underlying Java Objects .......................................................................... 37
3.2. Event Properties ............................................................................................... 38
3.2.1. Escape Characters ................................................................................. 39
3.2.2. Expression as Key or Index Value ........................................................... 40
3.3. Dynamic Event Properties ................................................................................. 41
3.4. Fragment and Fragment Type ........................................................................... 43
3.5. Comparing Event Representations ..................................................................... 43
3.5.1. Incoming Events .................................................................................... 44
3.5.2. Outgoing Events .................................................................................... 45
3.5.3. Schema ................................................................................................. 45
3.5.4. Side-by-Side .......................................................................................... 46
3.6. Support for Generic Tuples ............................................................................... 47
3.7. Updating, Merging and Versioning Events .......................................................... 49
3.8. Coarse-Grained Events ..................................................................................... 49
3.9. Event Objects Instantiated and Populated by Insert Into ...................................... 49
3.10. Event Type Uniqueness .................................................................................. 51
4. Context and Context Partitions ................................................................................. 53
4.1. Introduction ....................................................................................................... 53
4.2. Context Declaration ........................................................................................... 55
4.2.1. Context-Provided Properties ................................................................... 55
4.2.2. Keyed Segmented Context ..................................................................... 56
4.2.3. Hash Segmented Context ....................................................................... 63
4.2.4. Category Segmented Context ................................................................. 66
4.2.5. Non-Overlapping Context ........................................................................ 68
4.2.6. Overlapping Context ............................................................................... 70
4.2.7. Context Conditions ................................................................................. 74
4.3. Context Nesting ................................................................................................ 78
4.3.1. Nested Context Sample Walk-Through .................................................... 79
4.3.2. Built-In Nested Context Properties ........................................................... 81
4.4. Partitioning Without Context Declaration ............................................................. 82
4.5. Output When a Context Partition Ends or Terminates .......................................... 82
4.6. Context and Named Window ............................................................................. 84
4.7. Context and Tables ........................................................................................... 86
4.8. Context and Variables ....................................................................................... 87
4.9. Operations on Specific Context Partitions ........................................................... 87
5. EPL Reference: Clauses ............................................................................................ 89
5.1. EPL Introduction ............................................................................................... 89
5.2. EPL Syntax ...................................................................................................... 90
-
v
5.2.1. Specifying Time Periods ......................................................................... 90
5.2.2. Using Comments .................................................................................... 91
5.2.3. Reserved Keywords ............................................................................... 92
5.2.4. Escaping Strings .................................................................................... 93
5.2.5. Data Types ............................................................................................ 93
5.2.6. Using Constants and Enum Types .......................................................... 96
5.2.7. Annotation ............................................................................................. 97
5.2.8. Expression Alias ................................................................................... 103
5.2.9. Expression Declaration ......................................................................... 104
5.2.10. Script Declaration ............................................................................... 107
5.2.11. Referring to a Context ........................................................................ 107
5.2.12. Composite Keys and Array Values as Keys .......................................... 107
5.3. Choosing Event Properties and Events: The Select Clause ................................ 108
5.3.1. Choosing the Event Itself: Select * ........................................................ 108
5.3.2. Choosing Specific Event Properties ....................................................... 109
5.3.3. Expressions ......................................................................................... 109
5.3.4. Renaming Event Properties ................................................................... 110
5.3.5. Choosing Event Properties and Events in a Join ..................................... 110
5.3.6. Choosing Event Properties and Events From a Pattern ........................... 112
5.3.7. Selecting Insert and Remove Stream Events .......................................... 112
5.3.8. Select Distinct ...................................................................................... 113
5.3.9. Transposing an Expression Result to a Stream ...................................... 114
5.3.10. Selecting EventBean Instead of Underlying Event ................................. 114
5.4. Specifying Event Streams: The From Clause .................................................... 115
5.4.1. Filter-Based Event Streams ................................................................... 116
5.4.2. Pattern-Based Event Streams ............................................................... 119
5.4.3. Specifying Data Windows ..................................................................... 120
5.4.4. Multiple Data Windows ......................................................................... 121
5.4.5. Using the Stream Name ....................................................................... 122
5.5. Specifying Search Conditions: The Where Clause ............................................. 124
5.6. Aggregates and Grouping: The Group-By Clause and the Having Clause ............ 125
5.6.1. Using Aggregate Functions ................................................................... 125
5.6.2. Organizing Statement Results into Groups: The Group-by Clause ............ 127
5.6.3. Using Group-By with Rollup, Cube and Grouping Sets ............................ 130
5.6.4. Specifying Grouping for Each Aggregation Function ................................ 134
5.6.5. Specifying a Filter Expression for Each Aggregation Function .................. 135
5.6.6. Selecting Groups of Events: The Having Clause ..................................... 136
5.6.7. How the Stream Filter, Where, Group By and Having-Clauses Interact ..... 137
5.6.8. Comparing Keyed Segmented Context, the Group By Clause and
#groupwin for Data Windows .......................................................................... 138
5.7. Stabilizing and Controlling Output: The Output Clause ....................................... 139
5.7.1. Output Clause Options ......................................................................... 139
5.7.2. Aggregation, Group By, Having and Output Clause Interaction ................. 145
5.8. Sorting Output: the Order By Clause ................................................................ 146
-
Esper Reference
vi
5.9. Limiting Row Count: the Limit Clause ............................................................... 147
5.10. Merging Streams and Continuous Insertion: The Insert Into Clause ................... 148
5.10.1. Transposing a Property to a Stream .................................................... 150
5.10.2. Merging Streams by Event Type .......................................................... 151
5.10.3. Merging Disparate Types of Events: Variant Streams ............................ 152
5.10.4. Decorated Events ............................................................................... 153
5.10.5. Event as a Property ............................................................................ 153
5.10.6. Instantiating and Populating an Underlying Event Object ....................... 154
5.10.7. Transposing an Expression Result ....................................................... 154
5.10.8. Select-Clause Expression and Inserted-Into Column Event Type ............ 154
5.10.9. Insert Into for Event Types Without Properties ...................................... 155
5.11. Subqueries .................................................................................................... 156
5.11.1. The 'Exists' Keyword ........................................................................... 159
5.11.2. The 'In' and 'Not In' Keywords ............................................................. 160
5.11.3. The 'Any' and 'Some' Keywords .......................................................... 160
5.11.4. The 'All' Keyword ................................................................................ 161
5.11.5. Subquery With Group By Clause ......................................................... 162
5.11.6. Multi-Column Selection ....................................................................... 162
5.11.7. Multi-Row Selection ............................................................................ 163
5.11.8. Hints Related to Subqueries ................................................................ 164
5.12. Joining Event Streams ................................................................................... 165
5.12.1. Introducing Joins ................................................................................ 165
5.12.2. Inner (Default) Joins ........................................................................... 167
5.12.3. Outer, Left and Right Joins ................................................................. 168
5.12.4. Unidirectional Joins ............................................................................. 170
5.12.5. Unidirectional Full Outer Joins ............................................................. 172
5.12.6. Hints Related to Joins ......................................................................... 172
5.13. Accessing Relational Data via SQL ................................................................ 173
5.13.1. Joining SQL Query Results ................................................................. 174
5.13.2. SQL Query and the EPL Where Clause ............................................... 176
5.13.3. Outer Joins With SQL Queries ............................................................ 177
5.13.4. Using Patterns to Request (Poll) Data .................................................. 178
5.13.5. Polling SQL Queries via Iterator .......................................................... 178
5.13.6. JDBC Implementation Overview .......................................................... 178
5.13.7. Oracle Drivers and No-Metadata Workaround ....................................... 179
5.13.8. SQL Input Parameter and Column Output Conversion ........................... 180
5.13.9. SQL Row POJO Conversion ............................................................... 180
5.14. Accessing Non-Relational Data via Method, Script or UDF Invocation ............... 180
5.14.1. Joining Method, Script or UDF Invocation Results ................................. 181
5.14.2. Polling Invocation Results via Iterator ................................................... 182
5.14.3. Providing the Method .......................................................................... 183
5.14.4. Using a Map Return Type ................................................................... 185
5.14.5. Using a Object Array Return Type ....................................................... 186
5.14.6. Using an EventBean Return Type ........................................................ 188
-
vii
5.14.7. Providing the Script ............................................................................ 188
5.14.8. Providing the UDF .............................................................................. 189
5.15. Declaring an Event Type: Create Schema ...................................................... 189
5.15.1. Declare an Event Type by Providing Names and Types ......................... 189
5.15.2. Declare an Event Type by Providing a Class Name .............................. 193
5.15.3. Declare a Variant Stream .................................................................... 194
5.16. Splitting and Duplicating Streams ................................................................... 194
5.16.1. Generating Marker Events for Contained Events ................................... 196
5.17. Variables and Constants ................................................................................ 198
5.17.1. Creating Variables: The Create Variable Clause ................................... 198
5.17.2. Setting Variable Values: The On Set Clause ........................................ 201
5.17.3. Using Variables .................................................................................. 202
5.17.4. Object-Type Variables ......................................................................... 203
5.17.5. Class and Event-Type Variables .......................................................... 203
5.18. Declaring Global Expressions, Aliases and Scripts: Create Expression .............. 204
5.18.1. Global Expression Aliases ................................................................... 205
5.18.2. Global Expression Declarations ........................................................... 205
5.18.3. Global Scripts ..................................................................................... 206
5.19. Contained-Event Selection ............................................................................. 207
5.19.1. Select-Clause in a Contained-Event Selection ...................................... 210
5.19.2. Where Clause in a Contained-Event Selection ...................................... 212
5.19.3. Contained-Event Selection and Joins ................................................... 213
5.19.4. Sentence and Word Example .............................................................. 214
5.19.5. More Examples .................................................................................. 216
5.19.6. Contained Expression Returning an Array of Property Values ................ 217
5.19.7. Contained Expression Returning an Array of EventBean ........................ 217
5.19.8. Generating Marker Events Such as a Begin and End Event ................... 218
5.19.9. Contained-Event Limitations ................................................................ 218
5.20. Updating an Insert Stream: The Update IStream Clause .................................. 219
5.20.1. Immutability and Updates .................................................................... 221
5.21. Controlling Event Delivery : The For Clause .................................................... 221
6. EPL Reference: Named Windows and Tables .......................................................... 225
6.1. Overview ........................................................................................................ 225
6.1.1. Named Window Overview ..................................................................... 225
6.1.2. Table Overview .................................................................................... 226
6.1.3. Comparing Named Windows and Tables ............................................... 226
6.2. Named Window Usage .................................................................................... 228
6.2.1. Creating Named Windows: The Create Window clause ........................... 228
6.2.2. Inserting Into Named Windows .............................................................. 231
6.2.3. Selecting From Named Windows ........................................................... 234
6.3. Table Usage ................................................................................................... 235
6.3.1. Creating Tables: The Create Table Clause ............................................. 235
6.3.2. Aggregating Into Table Rows: The Into Table Clause .............................. 238
6.3.3. Table Column Keyed-Access Expressions ............................................. 242
-
Esper Reference
viii
6.3.4. Inserting Into Tables ............................................................................. 244
6.3.5. Selecting From Tables .......................................................................... 245
6.4. Triggered Select: The On Select Clause ........................................................... 246
6.4.1. Notes on On-Select With Named Windows ............................................ 248
6.4.2. Notes on On-Select With Tables ........................................................... 249
6.4.3. On-Select Compared to Join ................................................................. 249
6.5. Triggered Select+Delete: The On Select Delete Clause ..................................... 249
6.6. Updating Data: The On Update Clause ............................................................ 250
6.6.1. Notes on On-Update With Named Windows ........................................... 254
6.6.2. Notes on On-Update With Tables .......................................................... 254
6.7. Deleting Data: The On Delete Clause ............................................................... 254
6.7.1. Using Patterns in the On-Delete Clause ................................................. 255
6.7.2. Silent Delete With Named Windows ...................................................... 256
6.7.3. Notes on On-Delete With Named Windows ............................................ 256
6.7.4. Notes on On-Update With Tables .......................................................... 256
6.8. Triggered Upsert Using the On-Merge Clause ................................................... 256
6.8.1. On-Merge Insert ................................................................................... 257
6.8.2. On-Merge Update, Delete, Select (Upsert) ............................................. 257
6.8.3. Notes on On-Merge With Named Windows ............................................ 262
6.8.4. Notes on On-Merge With Tables ........................................................... 262
6.9. Explicitly Indexing Named Windows and Tables ................................................ 262
6.10. Using Fire-and-Forget Queries With Named Windows and Tables ..................... 265
6.10.1. Inserting Data ..................................................................................... 265
6.10.2. Updating Data .................................................................................... 267
6.10.3. Deleting Data ..................................................................................... 267
6.11. Events as Property ........................................................................................ 268
7. EPL Reference: Patterns .......................................................................................... 271
7.1. Event Pattern Overview ................................................................................... 271
7.2. How to Use Patterns ....................................................................................... 272
7.2.1. Pattern Syntax ..................................................................................... 272
7.2.2. Patterns in EPL .................................................................................... 272
7.2.3. Subscribing to Pattern Events ............................................................... 273
7.2.4. Pulling Data From Patterns ................................................................... 273
7.2.5. Pattern Error Reporting ......................................................................... 274
7.2.6. Suppressing Same-Event Matches ........................................................ 274
7.2.7. Discarding Partially Completed Patterns ................................................. 275
7.3. Operator Precedence ...................................................................................... 276
7.4. Filter Expressions in Patterns .......................................................................... 278
7.4.1. Filter Expressions and Filter Indexes ..................................................... 280
7.4.2. Controlling Event Consumption ............................................................. 280
7.4.3. Use With Named Windows and Tables .................................................. 281
7.5. Pattern Operators ........................................................................................... 282
7.5.1. Every ................................................................................................... 282
7.5.2. Every-Distinct ....................................................................................... 287
-
ix
7.5.3. Repeat ................................................................................................. 289
7.5.4. Repeat-Until ......................................................................................... 291
7.5.5. And ..................................................................................................... 294
7.5.6. Or ........................................................................................................ 296
7.5.7. Not ...................................................................................................... 296
7.5.8. Followed-By ......................................................................................... 297
7.5.9. Pattern Guards ..................................................................................... 299
7.6. Pattern Atoms ................................................................................................. 304
7.6.1. Filter Atoms ......................................................................................... 304
7.6.2. Observer Atoms Overview .................................................................... 304
7.6.3. Interval (timer:interval) .......................................................................... 304
7.6.4. Crontab (timer:at) ................................................................................. 305
7.6.5. Schedule (timer:schedule) ..................................................................... 308
7.7. Event Pattern Walkthrough .............................................................................. 314
8. EPL Reference: Match Recognize ............................................................................ 317
8.1. Overview ........................................................................................................ 317
8.2. Comparison of Match Recognize and EPL Patterns ........................................... 317
8.3. Syntax ............................................................................................................ 318
8.3.1. Syntax Example ................................................................................... 319
8.4. Pattern and Pattern Operators ......................................................................... 321
8.4.1. Operator Precedence ............................................................................ 321
8.4.2. Concatenation ...................................................................................... 321
8.4.3. Alternation ............................................................................................ 322
8.4.4. Quantifiers Overview ............................................................................ 323
8.4.5. Permutations ........................................................................................ 323
8.4.6. Variables Can Be Singleton or Group .................................................... 324
8.4.7. Eliminating Duplicate Matches ............................................................... 325
8.4.8. Greedy or Reluctant ............................................................................. 326
8.4.9. Quantifier - One or More (+ and +?) ...................................................... 327
8.4.10. Quantifier - Zero or More (* and *?) ..................................................... 327
8.4.11. Quantifier - Zero or One (? and ??) ..................................................... 328
8.4.12. Repetition - Exactly N Matches ............................................................ 329
8.4.13. Repetition - N or More Matches ........................................................... 330
8.4.14. Repetition - Between N and M Matches ............................................... 330
8.4.15. Repetition - Between Zero and M Matches ........................................... 331
8.4.16. Repetition Equivalence ........................................................................ 332
8.5. Define Clause ................................................................................................. 332
8.5.1. The Prev Operator ............................................................................... 333
8.6. Measure Clause .............................................................................................. 334
8.7. Datawindow-Bound ......................................................................................... 334
8.8. Interval ........................................................................................................... 335
8.9. Interval-or-Terminated ..................................................................................... 336
8.10. Use With Different Event Types ..................................................................... 337
8.11. Limiting Runtime-Wide State Count ................................................................ 338
-
Esper Reference
x
8.12. Limitations .................................................................................................... 339
9. EPL Reference: Operators ....................................................................................... 341
9.1. Arithmetic Operators ....................................................................................... 341
9.2. Logical and Comparison Operators .................................................................. 341
9.2.1. Null-Value Comparison Operators ......................................................... 341
9.3. Concatenation Operators ................................................................................. 342
9.4. Binary Operators ............................................................................................. 342
9.5. Array Definition Operator ................................................................................. 343
9.6. Dot Operator ................................................................................................... 344
9.6.1. Duck Typing ......................................................................................... 345
9.7. The 'In' Keyword ............................................................................................. 345
9.7.1. 'In' for Range Selection ......................................................................... 346
9.8. The 'Between' Keyword ................................................................................... 347
9.9. The 'Like' Keyword .......................................................................................... 348
9.10. The 'Regexp' Keyword ................................................................................... 349
9.11. The 'Any' and 'Some' Keywords ..................................................................... 350
9.12. The 'All' Keyword .......................................................................................... 351
9.13. The 'New' Keyword ....................................................................................... 351
9.13.1. Using 'New' to Populate a Data Structure ............................................. 352
9.13.2. Using 'New' to Instantiate an Object ..................................................... 353
10. EPL Reference: Functions ..................................................................................... 355
10.1. Single-Row Function Reference ..................................................................... 355
10.1.1. The Case Control Flow Function ......................................................... 357
10.1.2. The Cast Function .............................................................................. 358
10.1.3. The Coalesce Function ....................................................................... 360
10.1.4. The Current_Evaluation_Context Function ............................................ 360
10.1.5. The Current_Timestamp Function ........................................................ 361
10.1.6. The Event_Identity_Equals Function .................................................... 361
10.1.7. The Exists Function ............................................................................ 361
10.1.8. The Grouping Function ....................................................................... 362
10.1.9. The Grouping_Id Function ................................................................... 362
10.1.10. The Instance-Of Function .................................................................. 363
10.1.11. The Istream Function ........................................................................ 364
10.1.12. The Min and Max Functions .............................................................. 365
10.1.13. The Previous Function ...................................................................... 365
10.1.14. The Previous-Tail Function ................................................................ 368
10.1.15. The Previous-Window Function ......................................................... 369
10.1.16. The Previous-Count Function ............................................................ 371
10.1.17. The Prior Function ............................................................................ 372
10.1.18. The Type-Of Function ....................................................................... 372
10.2. Aggregation Functions ................................................................................... 374
10.2.1. SQL-Standard Functions ..................................................................... 375
10.2.2. Event Aggregation Functions ............................................................... 377
10.2.3. Approximation Aggregation Functions .................................................. 385
-
xi
10.2.4. Additional Aggregation Functions ......................................................... 388
10.3. User-Defined Functions ................................................................................. 391
10.4. Select-Clause Transpose Function ................................................................. 395
10.4.1. Transpose with Insert-Into ................................................................... 396
11. EPL Reference: Enumeration Methods .................................................................. 397
11.1. Overview ...................................................................................................... 397
11.2. Example Events ............................................................................................ 401
11.3. How to Use .................................................................................................. 403
11.3.1. Syntax ............................................................................................... 403
11.3.2. Introductory Examples ........................................................................ 403
11.3.3. Input, Output and Limitations ............................................................... 404
11.4. Inputs ........................................................................................................... 405
11.4.1. Subquery Results ............................................................................... 405
11.4.2. Named Window .................................................................................. 407
11.4.3. Table ................................................................................................. 408
11.4.4. Event Property and Insert-Into With @eventbean .................................. 408
11.4.5. Event Aggregation Function ................................................................ 409
11.4.6. Prev, Prevwindow and Prevtail Single-Row Functions as Input ............... 410
11.4.7. Single-Row Function, User-Defined Function and Enum Types .............. 411
11.4.8. Declared Expression ........................................................................... 413
11.4.9. Variables ............................................................................................ 413
11.4.10. Substitution Parameters .................................................................... 414
11.4.11. Match-Recognize Group Variable ....................................................... 414
11.4.12. Pattern Repeat and Repeat-Until Operators ........................................ 414
11.5. Example ....................................................................................................... 414
11.6. Reference ..................................................................................................... 415
11.6.1. Aggregate .......................................................................................... 415
11.6.2. AllOf .................................................................................................. 416
11.6.3. AnyOf ................................................................................................ 417
11.6.4. Average ............................................................................................. 417
11.6.5. CountOf ............................................................................................. 418
11.6.6. DistinctOf ........................................................................................... 419
11.6.7. Except ............................................................................................... 419
11.6.8. FirstOf ................................................................................................ 420
11.6.9. GroupBy ............................................................................................ 420
11.6.10. Intersect ........................................................................................... 421
11.6.11. LastOf .............................................................................................. 422
11.6.12. LeastFrequent .................................................................................. 422
11.6.13. Max ................................................................................................. 423
11.6.14. MaxBy .............................................................................................. 423
11.6.15. Min .................................................................................................. 424
11.6.16. MinBy .............................................................................................. 425
11.6.17. MostFrequent ................................................................................... 425
11.6.18. OrderBy and OrderByDesc ................................................................ 426
-
Esper Reference
xii
11.6.19. Reverse ........................................................................................... 426
11.6.20. SelectFrom ....................................................................................... 427
11.6.21. SequenceEqual ................................................................................ 427
11.6.22. SumOf ............................................................................................. 428
11.6.23. Take ................................................................................................ 428
11.6.24. TakeLast .......................................................................................... 429
11.6.25. TakeWhile ........................................................................................ 429
11.6.26. TakeWhileLast .................................................................................. 430
11.6.27. ToMap ............................................................................................. 431
11.6.28. Union ............................................................................................... 431
11.6.29. Where .............................................................................................. 432
12. EPL Reference: Date-Time Methods ....................................................................... 433
12.1. Overview ...................................................................................................... 433
12.2. How to Use .................................................................................................. 437
12.2.1. Syntax ............................................................................................... 437
12.3. Calendar and Formatting Reference ............................................................... 438
12.3.1. Between ............................................................................................. 438
12.3.2. Format ............................................................................................... 439
12.3.3. Get (By Field) .................................................................................... 440
12.3.4. Get (By Name) ................................................................................... 440
12.3.5. Minus ................................................................................................. 441
12.3.6. Plus ................................................................................................... 441
12.3.7. RoundCeiling ...................................................................................... 442
12.3.8. RoundFloor ........................................................................................ 442
12.3.9. RoundHalf .......................................................................................... 442
12.3.10. Set (By Field) ................................................................................... 443
12.3.11. WithDate .......................................................................................... 443
12.3.12. WithMax ........................................................................................... 444
12.3.13. WithMin ............................................................................................ 444
12.3.14. WithTime .......................................................................................... 444
12.3.15. ToCalendar ...................................................................................... 445
12.3.16. ToDate ............................................................................................. 445
12.3.17. ToMillisec ......................................................................................... 445
12.4. Interval Algebra Reference ............................................................................ 445
12.4.1. Examples ........................................................................................... 446
12.4.2. Interval Algebra Parameters ................................................................ 446
12.4.3. Performance ....................................................................................... 447
12.4.4. Limitations .......................................................................................... 447
12.4.5. After .................................................................................................. 447
12.4.6. Before ................................................................................................ 448
12.4.7. Coincides ........................................................................................... 449
12.4.8. During ................................................................................................ 450
12.4.9. Finishes ............................................................................................. 451
12.4.10. Finished By ...................................................................................... 452
-
xiii
12.4.11. Includes ........................................................................................... 453
12.4.12. Meets ............................................................................................... 454
12.4.13. Met By ............................................................................................. 455
12.4.14. Overlaps .......................................................................................... 455
12.4.15. Overlapped By .................................................................................. 456
12.4.16. Starts ............................................................................................... 457
12.4.17. Started By ........................................................................................ 458
13. EPL Reference: Aggregation Methods ................................................................... 461
13.1. Overview ...................................................................................................... 461
13.2. How to Use .................................................................................................. 461
13.2.1. Syntax ............................................................................................... 461
13.2.2. Use with Aggregation Function ............................................................ 462
13.2.3. Use with Table Column ....................................................................... 462
13.2.4. Using Dot to Obtain Property Values ................................................... 463
13.2.5. Using Dot to Enumerate Events .......................................................... 463
13.3. Aggregation Methods for Sorted Aggregations ................................................ 463
13.3.1. Overview ............................................................................................ 463
13.3.2. Specifying Composite Keys ................................................................. 467
13.3.3. CeilingEvent, FloorEvent, HigherEvent, LowerEvent, GetEvent .............. 467
13.3.4. CeilingEvents, FloorEvents, HigherEvents, LowerEvents, GetEvents ...... 467
13.3.5. CeilingKey, FloorKey, HigherKey, LowerKey ......................................... 468
13.3.6. FirstEvent, LastEvent, MinBy, MaxBy ................................................... 469
13.3.7. FirstEvents, LastEvents ....................................................................... 469
13.3.8. FirstKey, LastKey ............................................................................... 469
13.3.9. ContainsKey ....................................................................................... 470
13.3.10. CountEvents ..................................................................................... 470
13.3.11. CountKeys ........................................................................................ 470
13.3.12. EventsBetween ................................................................................. 471
13.3.13. Submap ........................................................................................... 471
13.3.14. NavigableMapReference ................................................................... 471
13.4. Aggregation Methods for Window Aggregations .............................................. 472
13.4.1. Overview ............................................................................................ 472
13.4.2. CountEvents ....................................................................................... 473
13.4.3. ListReference ..................................................................................... 473
13.4.4. First, Last ........................................................................................... 474
13.5. Aggregation Methods for CountMinSketch Aggregations .................................. 474
13.6. Aggregation Methods for Custom Plug-In Multi-Function Aggregations .............. 474
14. EPL Reference: Data Windows .............................................................................. 475
14.1. A Note on Data Window Name and Parameters .............................................. 478
14.2. A Note on Batch Windows ............................................................................. 480
14.3. Data Windows ............................................................................................... 481
14.3.1. Length Window (length or win:length) .................................................. 481
14.3.2. Length Batch Window (length_batch or win:length_batch) ...................... 482
14.3.3. Time Window (time or win:time) .......................................................... 482
-
Esper Reference
xiv
14.3.4. Externally-timed Window (ext_timed or win:ext_timed) ........................... 482
14.3.5. Time batch Window (time_batch or win:time_batch) .............................. 483
14.3.6. Externally-timed Batch Window (ext_timed_batch or
win:ext_timed_batch) ...................................................................................... 485
14.3.7. Time-Length Combination Batch Window (time_length_batch or
win:time_length_batch) ................................................................................... 486
14.3.8. Time-Accumulating Window (time_accum or win:time_accum) ............... 487
14.3.9. Keep-All Window (keepall or win:keepall) ............................................. 488
14.3.10. First Length Window(firstlength or win:firstlength) ................................ 488
14.3.11. First Time Window (firsttime or win:firsttime) ....................................... 489
14.3.12. Expiry Expression Window (expr or win:expr) ..................................... 489
14.3.13. Expiry Expression Batch Window (expr_batch or win:expr_batch) ......... 492
14.3.14. Unique Window (unique or std:unique) ............................................... 494
14.3.15. Grouped Data Window (groupwin or std:groupwin) .............................. 495
14.3.16. Last Event Window (std:lastevent) ..................................................... 498
14.3.17. First Event Window (firstevent or std:firstevent) ................................... 499
14.3.18. First Unique Window (firstunique or std:firstunique) ............................. 499
14.3.19. Sorted Window (sort or ext:sort) ........................................................ 500
14.3.20. Ranked Window (rank or ext:rank) ..................................................... 500
14.3.21. Time-Order Window (time_order or ext:time_order) ............................. 502
14.3.22. Time-To-Live Window (timetolive or ext:timetolive) .............................. 503
14.4. Special Derived-Value Windows ..................................................................... 504
14.4.1. Size Derived-Value Window (size) or std:size) ...................................... 504
14.4.2. Univariate Statistics Derived-Value Window (uni or stat:uni) ................... 505
14.4.3. Regression Derived-Value Window (linest or stat:linest) ......................... 506
14.4.4. Correlation Derived-Value Window (correl or stat:correl) ........................ 508
14.4.5. Weighted Average Derived-Value Window (weighted_avg or
stat:weighted_avg) ......................................................................................... 509
15. Compiler Reference ............................................................................................... 511
15.1. Introduction ................................................................................................... 511
15.2. Concepts ...................................................................................................... 511
15.2.1. Module ............................................................................................... 511
15.2.2. EPL-objects ........................................................................................ 513
15.2.3. Dependencies .................................................................................... 513
15.2.4. Dependency Resolution ...................................................................... 514
15.2.5. Access Modifiers ................................................................................ 515
15.2.6. Bus Modifier for Event Types .............................................................. 516
15.3. Compiling a Module ...................................................................................... 517
15.4. Reading and Writing a Compiled Module ........................................................ 517
15.5. Reading Module Content ............................................................................... 517
15.6. Compiler Arguments ...................................................................................... 517
15.6.1. Compiler Configuration ........................................................................ 517
15.6.2. Compiler Path .................................................................................... 518
15.6.3. Compiler Options ................................................................................ 520
-
xv
15.7. Statement Object Model ................................................................................ 520
15.7.1. Building an Object Model .................................................................... 521
15.7.2. Building Expressions ........................................................................... 522
15.7.3. Building a Pattern Statement ............................................................... 523
15.7.4. Building a Select Statement ................................................................ 524
15.7.5. Building a Create-Variable and On-Set Statement ................................. 524
15.7.6. Building Create-Window, On-Delete and On-Select Statements .............. 525
15.8. Substitution Parameters ................................................................................. 526
15.9. OSGi, Class Loader, Class-For-Name ............................................................ 527
15.10. Authoring Tools ........................................................................................... 527
15.11. Testing Tools .............................................................................................. 528
15.12. Debugging .................................................................................................. 528
15.12.1. @Audit Annotation ............................................................................ 528
15.13. Ordering Multiple Modules ........................................................................... 530
15.14. Logging ....................................................................................................... 530
15.15. Debugging Generated Code ......................................................................... 530
15.16. Compiler Version and Runtime Version ......................................................... 531
15.17. Compiler Byte Code Optimizations ............................................................... 531
16. Runtime Reference ................................................................................................. 533
16.1. Introduction ................................................................................................... 533
16.2. Obtaining a Runtime From EPRuntimeProvider ............................................... 533
16.3. The EPRuntime Runtime Interface ................................................................. 534
16.4. Deploying and Undeploying Using EPDeploymentService ................................ 535
16.4.1. Substitution Parameters ...................................................................... 536
16.4.2. Atomic Deployment Management ........................................................ 537
16.5. Obtaining Results Using EPStatement ............................................................ 538
16.5.1. Receiving Statement Results ............................................................... 538
16.5.2. Setting a Subscriber Object ................................................................. 541
16.5.3. Adding Listeners ................................................................................. 546
16.5.4. Using Iterators .................................................................................... 547
16.5.5. Event and Event Type ........................................................................ 549
16.5.6. Interrogating Annotations .................................................................... 553
16.6. Processing Events and Time Using EPEventService ........................................ 554
16.6.1. Event Sender ..................................................................................... 556
16.6.2. Receiving Unmatched Events .............................................................. 557
16.7. Execute Fire-and-Forget Queries Using EPFireAndForgetService ..................... 557
16.7.1. Fire-and-forget Query Single Execution ................................................ 558
16.7.2. Fire-and-forget Query Prepared Unparameterized Execution .................. 559
16.7.3. Fire-and-forget Query Prepared Parameterized Execution ..................... 559
16.8. Runtime Threading and Concurrency .............................................................. 560
16.8.1. Advanced Threading ........................................................................... 562
16.8.2. Processing Order ................................................................................ 565
16.9. Controlling Time-Keeping ............................................................................... 566
16.9.1. Controlling Time Using Time Span Events ............................................ 569
-
Esper Reference
xvi
16.9.2. Time Resolution and Time Unit ........................................................... 570
16.9.3. Internal Timer Based on JVM System Time .......................................... 571
16.10. Exception Handling ...................................................................................... 571
16.11. Condition Handling ...................................................................................... 572
16.12. Runtime and Statement Metrics Reporting .................................................... 573
16.12.1. Runtime Metrics ................................................................................ 574
16.12.2. Statement Metrics ............................................................................. 574
16.13. Monitoring and JMX .................................................................................... 575
16.14. Event Rendering to XML and JSON ............................................................. 575
16.14.1. JSON Event Rendering Conventions and Options ............................... 577
16.14.2. XML Event Rendering Conventions and Options ................................. 577
16.15. Plug-In Loader ............................................................................................ 577
16.16. Context Partition Selection ........................................................................... 579
16.16.1. Selectors .......................................................................................... 581
16.17. Context Partition Administration .................................................................... 581
16.18. Test and Assertion Support .......................................................................... 582
16.18.1. EPAssertionUtil Summary ................................................................. 582
16.18.2. SupportUpdateListener Summary ....................................................... 583
16.18.3. Usage Example ................................................................................ 583
16.19. OSGi, Class Loader, Class-For-Name .......................................................... 584
16.20. When Deploying with J2EE .......................................................................... 584
16.20.1. J2EE Deployment Considerations ...................................................... 585
16.20.2. Servlet Context Listener .................................................................... 585
17. Configuration ......................................................................................................... 589
17.1. Overview ...................................................................................................... 589
17.2. Programmatic Configuration ........................................................................... 589
17.3. Configuration via XML File ............................................................................. 590
17.4. Configuration Common .................................................................................. 590
17.4.1. Annotation Class and Package Imports ................................................ 591
17.4.2. Class and Package Imports ................................................................ 591
17.4.3. Events Represented by Classes .......................................................... 592
17.4.4. Events Represented by java.util.Map ................................................... 597
17.4.5. Events Represented by Object[] (Object-array) ..................................... 598
17.4.6. Events Represented by JSON ............................................................. 600
17.4.7. Events Represented by Avro GenericData.Record ................................ 600
17.4.8. Events Represented by org.w3c.dom.Node .......................................... 602
17.4.9. Event Type Defaults ........................................................................... 608
17.4.10. Event Type Import Package (Event Type Auto-Name) ......................... 611
17.4.11. From-Clause Method Invocation ........................................................ 611
17.4.12. Relational Database Access .............................................................. 612
17.4.13. Common Settings Related to Logging ................................................ 620
17.4.14. Common Settings Related to Time Source ......................................... 621
17.4.15. Variables .......................................................................................... 622
17.4.16. Variant Stream ................................................................................. 622
-
xvii
17.5. Configuration Compiler .................................................................................. 623
17.5.1. Compiler Settings Related to Byte Code Generation ............................. 623
17.5.2. Compiler Settings Related to View Resources ...................................... 626
17.5.3. Compiler Settings Related to Logging .................................................. 627
17.5.4. Compiler Settings Related to Stream Selection ..................................... 627
17.5.5. Compiler Settings Related to Language and Locale .............................. 628
17.5.6. Compiler Settings Related to Expression Evaluation .............................. 629
17.5.7. Compiler Settings Related to Scripts .................................................... 631
17.5.8. Compiler Settings Related to Execution of Statements .......................... 631
17.5.9. Compiler Settings Related to Serializers and Deserializers .................... 633
17.6. Configuration Runtime ................................................................................... 633
17.6.1. Runtime Settings Related to Concurrency and Threading ...................... 633
17.6.2. Runtime Settings Related to Logging ................................................... 638
17.6.3. Runtime Settings Related to Variables ................................................. 639
17.6.4. Runtime Settings Related to Patterns ................................................... 640
17.6.5. Runtime Settings Related to Match-Recognize ..................................... 640
17.6.6. Runtime Settings Related to Time Source ............................................ 641
17.6.7. Runtime Settings Related to JMX Metrics ............................................. 642
17.6.8. Runtime Settings Related to Metrics Reporting ..................................... 642
17.6.9. Runtime Settings Related to Expression Evaluation .............................. 644
17.6.10. Runtime Settings Related to Execution of Statements ......................... 645
17.6.11. Runtime Settings Related to Exception Handling ................................. 648
17.6.12. Runtime Settings Related to Condition Handling ................................. 649
17.7. Passing Services or Transient Objects ........................................................... 649
17.7.1. Service Example ................................................................................ 650
17.7.2. Class-for-Name .................................................................................. 650
17.7.3. Class Loader ...................................................................................... 651
17.8. Type Names ................................................................................................. 652
17.9. Logging Configuration .................................................................................... 652
17.9.1. Log4j Logging Configuration ................................................................ 653
18. Script Support ........................................................................................................ 655
18.1. Overview ...................................................................................................... 655
18.2. Syntax .......................................................................................................... 655
18.3. Examples ...................................................................................................... 656
18.4. Built-In EPL Script Attributes .......................................................................... 657
18.5. Performance Notes ....................................................................................... 658
18.6. Additional Notes ............................................................................................ 658
19. EPL Reference: Spatial Methods and Indexes ....................................................... 659
19.1. Overview ...................................................................................................... 659
19.2. Spatial Methods ............................................................................................ 659
19.2.1. Point-Inside-Rectangle ........................................................................ 659
19.2.2. Rectangle-Intersects-Rectangle ........................................................... 660
19.3. Spatial Index - Quadtree ................................................................................ 661
19.3.1. Overview ............................................................................................ 661
-
Esper Reference
xviii
19.3.2. Declaring a Point-Region Quadtree Index ............................................. 662
19.3.3. Using a Point-Region Quadtree as a Filter Index .................................. 662
19.3.4. Using a Point-Region Quadtree as an Event Index ................................ 663
19.3.5. Declaring a MX-CIF Quadtree Index .................................................... 665
19.3.6. Using a MX-CIF Quadtree as a Filter Index .......................................... 666
19.3.7. Using a MX-CIF Quadtree as an Event Index ....................................... 667
19.4. Spatial Types, Functions and Methods from External Libraries ......................... 668
20. EPL Reference: Data Flow ..................................................................................... 671
20.1. Introduction ................................................................................................... 671
20.2. Usage ........................................................................................................... 671
20.2.1. Overview ............................................................................................ 671
20.2.2. Syntax ............................................................................................... 673
20.3. Built-In Operators .......................................................................................... 678
20.3.1. BeaconSource .................................................................................... 678
20.3.2. EPStatementSource ............................................................................ 680
20.3.3. EventBusSink ..................................................................................... 681
20.3.4. EventBusSource ................................................................................. 682
20.3.5. Filter .................................................................................................. 683
20.3.6. LogSink .............................................................................................. 684
20.3.7. Select ................................................................................................ 685
20.4. API ............................................................................................................... 686
20.4.1. Declaring a Data Flow ........................................................................ 686
20.4.2. Instantiating a Data Flow .................................................................... 687
20.4.3. Executing a Data Flow ........................................................................ 688
20.4.4. Instantiation Options ........................................................................... 689
20.4.5. Start Captive ...................................................................................... 690
20.4.6. Data Flow Punctuation With Markers ................................................... 691
20.4.7. Exception Handling ............................................................................. 691
20.5. Examples ...................................................................................................... 691
20.6. Operator Implementation ............................................................................... 693
20.6.1. Sample Operator Acting as Source ...................................................... 693
20.6.2. Sample Tokenizer Operator ................................................................. 695
20.6.3. Sample Aggregator Operator ............................................................... 696
20.6.4. Passing Operator Parameters ............................................................. 697
21. Integration and Extension ...................................................................................... 701
21.1. Overview ...................................................................................................... 701
21.2. Single-Row Function ..................................................................................... 702
21.2.1. Implementing a Single-Row Function ................................................... 702
21.2.2. Configuring the Single-Row Function Name .......................................... 703
21.2.3. Value Cache ...................................................................................... 703
21.2.4. Single-Row Functions in Filter Predicate Expressions ............................ 704
21.2.5. Single-Row Functions Taking Events as Parameters ............................. 705
21.2.6. Single-Row Functions Returning Events ............................................... 705
21.2.7. Receiving a Context Object ................................................................. 706
-
xix
21.2.8. Exception Handling ............................................................................. 707
21.3. Virtual Data Window ...................................................................................... 707
21.3.1. How to Use ........................................................................................ 708
21.3.2. Implementing the Forge ...................................................................... 710
21.3.3. Implementing the Factory-Factory ........................................................ 711
21.3.4. Implementing the Factory .................................................................... 712
21.3.5. Implementing the Virtual Data Window ................................................. 714
21.4. Data Window View and Derived-Value View ................................................... 715
21.4.1. Implementing a View Forge ................................................................. 716
21.4.2. Implementing a View Factory .............................................................. 717
21.4.3. Implementing a View .......................................................................... 718
21.4.4. View Contract ..................................................................................... 718
21.4.5. Configuring View Namespace and Name ............................................. 719
21.4.6. Requirement for Data Window Views ................................................... 720
21.4.7. Requirement for Derived-Value Views .................................................. 720
21.5. Aggregation Function ..................................................................................... 720
21.5.1. Aggregation Single-Function Development ........................................... 721
21.5.2. Aggregation Multi-Function Development ............................................. 728
21.6. Pattern Guard ............................................................................................... 737
21.6.1. Implementing a Guard Forge ............................................................... 738
21.6.2. Implementing a Guard Factory ............................................................ 739
21.6.3. Implementing a Guard Class ............................................................... 740
21.6.4. Configuring Guard Namespace and Name ........................................... 741
21.7. Pattern Observer ........................................................................................... 741
21.7.1. Implementing an Observer Forge ........................................................ 742
21.7.2. Implementing an Observer Factory ...................................................... 743
21.7.3. Implementing an Observer Class ......................................................... 744
21.7.4. Configuring Observer Namespace and Name ....................................... 745
21.8. Date-Time Method ......................................................................................... 745
21.8.1. Implement the DateTimeMethodForgeFactory Interface ......................... 746
21.8.2. Implement the Static Methods ............................................................. 748
21.8.3. Add the Date-Time Method Extension to the Compiler Configuration ....... 751
21.8.4. Use the new Date-Time Methods ........................................................ 751
21.9. Enumeration Method ..................................................................................... 751
21.9.1. Implement the EnumMethodForgeFactory Interface .............................. 752
21.9.2. Implement the EnumMethodState Interface .......................................... 755
21.9.3. Implement the Static Method for Processing ......................................... 756
21.9.4. Add the Enumeration Method Extension to the Compiler Configuration .... 757
21.9.5. Use the new Enumeration Method ....................................................... 757
21.9.6. Further Information to Lambda Parameters .......................................... 757
22. Examples, Tutorials, Case Studies ......................................................................... 759
22.1. Examples Overview ....................................................................................... 759
22.2. Running the Examples .................................................................................. 761
22.3. AutoID RFID Reader ..................................................................................... 762
-
Esper Reference
xx
22.4. Runtime Configuration ................................................................................... 762
22.5. JMS Server Shell and Client .......................................................................... 762
22.5.1. Overview ............................................................................................ 762
22.5.2. JMS Messages as Events ................................................................... 763
22.5.3. JMX for Remote Dynamic Statement Management ............................... 764
22.6. Market Data Feed Monitor ............................................................................. 764
22.6.1. Input Events ....................................................................................... 764
22.6.2. Computing Rates per Feed ................................................................. 764
22.6.3. Detecting a Fall-Off ............................................................................ 765
22.6.4. Event generator .................................................................................. 765
22.7. OHLC Plug-In Data Window .......................................................................... 765
22.8. Transaction 3-Event Challenge ...................................................................... 766
22.8.1. The Events ........................................................................................ 766
22.8.2. Combined Event ................................................................................. 766
22.8.3. Real-Time Summary Data ................................................................... 767
22.8.4. Find Problems .................................................................................... 767
22.8.5. Event Generator ................................................................................. 767
22.9. Self-Service Terminal .................................................................................... 768
22.9.1. Events ............................................................................................... 768
22.9.2. Detecting Customer Check-In Issues ................................................... 768
22.9.3. Absence of Status Events ................................................................... 769
22.9.4. Activity Summary Data ........................................................................ 769
22.9.5. Sample Application for J2EE Application Server ................................... 769
22.10. Assets Moving Across Zones - An RFID Example .......................................... 771
22.11. StockTicker ................................................................................................. 772
22.12. MatchMaker ................................................................................................ 772
22.13. Named Window Query ................................................................................. 772
22.14. Sample Virtual Data Window ........................................................................ 773
22.15. Sample Cycle Detection ............................................................................... 773
22.16. Quality of Service ........................................................................................ 773
22.17. Trivia Geeks Club ........................................................................................ 774
23. Performance ........................................................................................................... 775
23.1. Big O Notation .............................................................................................. 775
23.1.1. Big-O Complexity of Matching Events to Statements and Context
Partitions ....................................................................................
top related