arcgis geoevent processor an...

Esri EMEAUC13

December 2013 CT GIS User 2 User Network MeetingDecember 13 | Hartford, CT

ArcGIS GeoEvent ProcessorAn Introduction

Al Frauenfelder

Esri EMEAUC13 ArcGIS GeoEvent Processor - An Introduction

• GIS data typically represents state at a specific moment in time: “historic”, “current”, or “future”.

GIS Data

Credit: iStockphoto/chris_lemmens


• Real-time GIS data is a continuous stream of events flowing from sensors where each event represents the latest state of the sensor.

- Emergency Response- Utility Networks and Warehouses- Environmental

FeaturesChallenge #1

Real-time GIS Data

ApplicationsNetwork SensorWarehouse Item

110°

StormWind

Temperature

EarthquakeWild Fire

Police Person

Ambulance

Police Car


• Are my field personnel working within the designated project area?

Features

Challenge #2

ContinuousAnalysis

OutsideBoundary

Real-time Analytics

EquipmentAlert

Applications

Esri EMEAUC13

Challenge #3

ArcGIS GeoEvent Processor - An Introduction

• Tell customer when their delivery truck is 15 minutes away.Real-time Notifications and Alerting

ContinuousAnalysis

Features

Enter Customer Area

Delivery Vehicle

Applications

Esri EMEAUC13

GeoEvent Services


ArcGIS GeoEvent Processor for Server

GeoEventProcessor

ArcGIS ServerInputs

ws://

Outputs

Integrates and Exploits real-time data

• Integrates real-time streaming data into ArcGIS• Performs continuous processing and real-time analytics• Sends updates and alerts to those who need it where they need it

Esri EMEAUC13

Working with real-time data

Esri EMEAUC13

You can add yourown connectors.

GeoEvent Processor

Inpu

ts

Out

puts

GeoEvent Services


• You can easily integrate real-time streaming data with ArcGIS by using an input connector.

Receiving real-time data

Receive RSS

Receive text from a TCP Socket

Receive text from a UDP Socket

Receive JSON on a REST endpoint

Receive Features on a REST endpoint

Receive JSON on a Web Socket

Receive JSON on external Web Socketws://

http://

http://

ws://

Poll an ArcGIS Server for Features

Poll an external website for JSONhttp://

Watch a folder for new .csv files

Watch a folder for new .json files

.csv

.json

out-o

f-the

-box

Gal

lery

Receive Tweets

Receive Instagram feed

Receive NMEA *

Receive Sierra Wireless *

Receive Trimble *

NMEA

Receive Vessels

Receive VMF

Receive Cursor on Target

Receive Air Traffic Control

Receive Geomessages

CoT

VMF

Message Geo

Receive from RabbitMQ *

Receive from ActiveMQ *

Esri EMEAUC13

Social monitoring

Demonstration

Demo

Esri EMEAUC13

You can add yourown connectors.

GeoEvent Processor

Inpu

ts

Out

puts

GeoEvent Services

out-o

f-the

-box


Sending real-time data• You can easily send updates and results to those who need it

where they need it using an output connector.

Add a feature

Update a feature

Publish JSON to a Web Socket

Push JSON to an external Web Socketws://

ws://

Send a text message

Send an instant message

Send an email

im

Publish text on a TCP Socket

Publish text on a UDP Socket

Write to a .csv file Write to a .json file.csv .json

Send a Tweet

Gal

lery

Publish JSON to an external website

Publish on a REST endpoint

http://

Send to RabbitMQ

Send to ActiveMQ

Write to Hadoop

Write to MongoDB

Send to CESIUM


Using local feature servicesSending real-time data to features

Operations Dashboard for ArcGIS

Your Dashboards

ArcGIS Online /Portal for ArcGIS

operation view

web map

YourApplications

JavaScript

Flex

Silverlight/WPF

iOS

Android

Windows Phone

.Net

Mac OS X

Java

Qt

GeoEvent Processor

Inpu

ts

GeoEvent Services

ArcGIS Server

Out

puts

feature layers


Using remote feature servicesSending real-time data to features

GeoEvent Processor

YourApplications

JavaScript

Flex

Silverlight/WPF

iOS

Android

Windows Phone

.Net

Mac OS X

Java

Qt

ArcGIS Server

Inpu

ts

GeoEvent ServicesO

utpu

ts


Your Dashboards

operation view

web map

ArcGIS Server


feature layers


Using ArcGIS Online / Portal for ArcGIS feature servicesSending real-time data to features

YourApplications

JavaScript

Flex

Silverlight/WPF

iOS

Android

Windows Phone

.Net

Mac OS X

Java

Qt


Your Dashboards

operation view

web map

GeoEvent Processor

Inpu

ts

GeoEvent ServicesO

utpu

ts

ArcGIS Server


feature layers

Esri EMEAUC13

Performing continuous processingand real-time analytics


GeoEvent Services

• A GeoEvent Service configures the flow of GeoEvents,- the Filtering and GeoEvent Processing steps to perform,- what input(s) to apply them to,- and what outputs(s) to send the results to.

Continuous processing and analytics

GeoEvent Processor

Inpu

ts

Out

puts

GeoEvent Services

Twitter Input Add Tweet FeatureGeolocatedFilter

Write Tweets to data

Twitter Monitor (GeoEvent Service)


ProcessorsContinuous processing and analytics

GeoEvent Service

Field Enricher

Field Reducer

Field Calculator

GeoTagger

Field Mapper

Track Gap Detector

• Modify a geoevent: Field Enricher, Field Reducer• Calculate new fields on a geoevent: Field Calculator, GeoTagger• Derive geoevent: Field Mapper, Track Gap Detector, Incident Detector

Input 1 Output 1

Incident Detector

• A Field Enricher processor- enriches the geoevent with new fields derived from a data source:

feature service or file.

EnrichedGeoEvent

GeoEvent

Feature Service

• A Field Reducer processor- removes fields from a geoevent.

ReducedGeoEvent

Fields to remove

Continuous processing and analyticsProcessors – modify a geoevent

• A Field Calculator processor uses an expression to- calculate a new field or update an existing field.- Expressions can be mathematical expressions or regular expressions.

• A GeoTagger processor- uses a spatial condition to tag the event with related geometries.

ExpressionEnriched

GeoEvent

GeoEvent

Spatial ConditionEnriched

GeoEvent

GeoEvent

Continuous processing and analyticsProcessors – calculate new fields on a geoevent

• A Field Mapper processor- translates from one geoevent format to another.

TranslatedGeoEvent

IncomingGeoEvent

Mapping

Continuous processing and analyticsProcessors – derive a new geoevent

• A Track Gap Detector processor- Detects the absence of events and alerts about the situation.


• An Incident Detector processor- creates an incident upon an opening condition being met,- maintains state for the duration of the incident,- closes the incident based on a closing condition or expiration.

GeoEvent Service

Incident Detector

91909265689091909281736866

CloseIncident

81

OpenIncident

90noiseDecibel >= 90Opening Condition

Input 1 Output 1

Continuous processing and analyticsProcessors – derive a new geoevent

Esri EMEAUC13

Demo

Noise violation alerting

Demonstration

JSON

external network (amazon ec2) internal network (my laptop)

Noise App

GeoEvent ProcessorGeoEvent Service

ws://

ArcGIS Server

GeoEvent Processor

ArcGIS Server

GeoEvent Service

ws:// feature

featureincident detector

ws://


Applying real-time GIS

• A number of sample applications and solutions leverage GeoEvent Processor to enable real-time GIS capabilities within them including:

- Defense and Intelligence: http://resources.arcgis.com/en/communities/defense-and-intelligence/

- ArcGIS for the Military- ArcGIS for the National Guard- ArcGIS for Intelligence

- Asset Monitor- Route Monitor

ArcGISfor the Military

ArcGIS forIntelligence

RouteMonitoring

AssetMonitoring

Solutions and sample applications

Esri EMEAUC13

Demo

Ambulance monitoring

Demonstration

Features AssetMonitor

OperationView

Alerts

Asset Monitor

Vehicle MonitoringExcessive Speed MonitoringDangerous Area Monitoring

Proximity MonitoringTerritory Monitoring

Assets

ArcGIS for Server

GeoEvent Processor

Route Monitor

FieldWorker

Vehicle MonitoringWorker Monitoring

Idle/Stop MonitoringDangerous Area Monitoring

Territory MonitoringProximity Monitoring

Auto-Arrival / Auto-DepartureContinuous ETARoute Adherence

GeoEvent Services

Vehicle Streams

Dispatchers

FleetManagers

RoutesStops

RoutePlanning

RouteMonitoringWorker Streams

ArcGIS for Server

Operation View

SampleApplication or 3rd Party

SampleApplication

Features

ArcGIS for the National Guard

Local TacticalGroup 2


Operations Server

Commanders

Analysts

Areas

Sensor ObservationsTrack ReportsSpot Reports

Digital Chemlights

GeoEvent Services

ArcGIS

VehicleCommander

GuardOperations

SquadLeader

Sample Application Sample Application

Operation Views

Federal & State Government

ArcGIS for the National Guard


Summary

• ArcGIS is a dynamic platform that enables real-time visualization and continuous analytics to better understanding of our world.

• GeoEvent Processor allows you to:- be alerted when interesting events occur- react and make smarter decisions faster- to know what is happening, as it happens

• To learn more, visit the tutorials in the Gallery:- http://pro.arcgis.com/share/geoevent-processor

ArcGIS GeoEvent Processor – An Intro


Real-timedata

• GIS Tools for Hadoop- http://github.com/Esri/gis-tools-for-hadoop

ArcGIS for DesktopGeoprocessing

GIS Tools for Hadoop

2013/03/06/10/vehicles-2013-03-06-10.json2013/03/06/11/vehicles-2013-03-06-11.json2013/03/06/12/vehicles-2013-03-06-12.json2013/03/06/13/vehicles-2013-03-06-13.json

2013/03/06/10/vehicles-2013-03-06-10.tsv2013/03/06/11/vehicles-2013-03-06-11.tsv2013/03/06/12/vehicles-2013-03-06-12.tsv2013/03/06/13/vehicles-2013-03-06-13.tsv

GeoEvent Processor

ArcGIS for Server

Filters Processors

Inpu

ts

Out

puts

GeoEvent Services

Write to Hadoop

Write to MongoDB

Write to a Hadoop Distributed File System (HDFS)Write to a MongoDB Document Store

Sending real-time data to big data stores

ArcGIS for the MilitaryMaritime Operations

Task Force

Task Force

Operations Server

Commanders

Analysts

NAIs, TAIs

AISTrack ReportsSpot Reports

Digital Chemlights

GeoEvent Services

ArcGIS

VehicleCommander

MaritimeOperations

SquadLeader


Operation Views

ArcGIS for the MilitaryMaritime Operations


ArcGIS for the MilitaryLand Operations


Operations Server

Commanders

Analysts

NAIs, TAIs


Digital Chemlights

GeoEvent Services

ArcGIS

VehicleCommander

LandOperations

SquadLeader


Operation Views

Supporting HQs

ArcGIS for the MilitaryLand Operations

ArcGIS for Intelligence

National Assets

Agency Assets

Operations Server

Commanders

Analysts

NAIs, TAIs


Digital Chemlights

GeoEvent Services

ArcGIS

VehicleCommander

IntelligenceOperations

SquadLeader


Operation Views

Tactical Assets

Other Agencies & Nations

ArcGIS for IntelligenceTemplate

arcgis geoevent processor an...

Documents