Building Real-Time Web Applications Using ArcGIS GeoEvent Processor

James Cardona | Web Developer DC Development Center

jcardona@esri.com

Ryan Elliott | Software Engineer ArcGIS GeoEvent Processor for Server

relliott@esri.com

ArcGIS GeoEvent Processor for Server

• Integrates real-time streaming data into ArcGIS • Performs continuous processing and real-time analytics • Sends updates and alerts to those who need it where they need it

Integrates and exploits real-time data

GeoEvent Services

GeoEvent Processor

ArcGIS Server Inputs

ws://

Outputs

ArcGIS Web API for JavaScript

• Gives developers tools to add GIS functionality to web applications

- Interactive maps for visualizing data.

- Widgets for finding addresses, editing data, making legends…

- Analysis – Run a model and view results, enrich existing data with detailed demographic information

- Embed into existing web page or make new focused application

Demonstration Field Worker Monitoring

Demo

Real-Time Data on the Map

• SCREENSHOT

Special Handling of Alerts

• SCREENSHOT

Selectively Display Historical Data

• SCREENSHOT

Receiving Real-Time Data Ryan Elliott

You can create your own

connectors.

Receiving real-time data

GeoEvent Processor

Inpu

ts

Out

puts

GeoEvent Services

Receive RSS

Receive text from a TCP Socket

Receive text from a UDP Socket

Receive JSON on a REST endpoint

Receive Features on a REST endpoint

Receive JSON on a Web Socket

Receive JSON on external Web Socket ws://

http://

http://

ws://

Poll an ArcGIS Server for Features

Poll an external website for JSON http://

Watch a folder for new .csv files

Watch a folder for new .json files

.csv

.json

Out

of t

he B

ox

Connectors • You can easily integrate real-time streaming data with ArcGIS by using an input

connector.

Esr

i Gal

lery

Twitter

Instagram

VMF

Cursor-on-Target

CAP

TAIP (Trimble)

NMEA

RabbitMQ

RAP (Sierra Wireless)

GeoMessage

ActiveMQ

Part

ner G

alle

ry

GNIP

Geofeedia

OSIsoft

ASDI (FAA)

exactEarth

Zonar

NetworkFleet

CompassCom

Valarm

Harris

Demonstration Connecting to Real-Time Data Feeds

Demo

Applying Real-Time Analytics Ryan Elliott

Applying real-time analytics

• A GeoEvent Service configures the flow of GeoEvents, - the Filtering and GeoEvent Processing steps to perform, - what input(s) to apply them to, - and what outputs(s) to send the results to.

GeoEvent Services

You can create your own

processors.

GeoEvent Processor

Inpu

ts

Out

puts

GeoEvent Services

Applying real-time analytics

Out

of t

he B

ox Field Enricher

Field Reducer

Track Gap Detector

Incident Detector

Field Mapper

Geotagger

Field Calculator

Track Idle Detector

Esri

Gal

lery

ETA Calculator

Service Area

Buffer

Ellipse

Range Fan

Visibility

Query Report

Slope Calculator

SDK

Volume Control

GeoEvent Processing • You can perform continuous analytics on geoevents as they are received using a

processor.

Demonstration Detecting GeoFence Violations

Demo

Ryan Elliott & James Cardona

Sending Real-Time Data to the Map

Sending Real-time Events to Clients Patterns – pull and push • Pull via feature services

- Must be backed by an enterprise geodatabase (EGDB) - Clients poll to get updates

• Push via Web Socket output - Low latency, high throughput - Clients subscribe to features of interest

GeoEvent Processor

ArcGIS Server

ArcGIS Server

feature layers

Your

Applications

Update a Feature Add a Feature

Polling (Pull)

EGDB

Broadcast Features ws://

Subscribe (Push)

Stream Layer GeoEvent Services

Map Services Feature Services

…

Feature Layer

Demonstration Publishing Events through Web Sockets

Demo

Ryan Elliott

Quick Tour of the GeoEvent Processor REST Interface

REST Admin API

Administering GeoEvent Processor REST Admin API • A complete set of REST administrative endpoints enable you to manage GeoEvent

Processor programmatically: - Everything you can do in ‘GeoEvent Processor Manager’ you can also do via REST since it

exclusively uses the REST Admin API

GeoEvent Processor Manager

GeoEvent Processor

Out

puts

Inpu

ts

GeoEvent Services

ArcGIS Server

https://localhost:6143/geoevent/admin

JavaScript

Exploring the REST api Ryan Elliott

Demo

James Cardona

Displaying Real-Time Web Socket data on a map with the Stream Layer

ArcGIS Server

Feature service

Stream Layer What is it?

• A new type of layer in the Javascript API - Introduced in version 3.6

• Draws data on map using client-side graphics

POST

Stream Layer

GET HTTP PUSH Web

Socket

esri/layers/GraphicsLayer esri/layers/FeatureLayer

esri/layers/StreamLayer

ArcGIS Server

GeoEvent Processor

Feature Layer

RESPONSE § Geometry • Attributes • Symbol

Graphic

Graphics Layer

Stream Layer Advantages

• More efficient transfer of data. - Only negotiate a connection once. - Messages sent without extra headers

• Avoids need to poll for data updates Stream Layer

Feature Layer

Request Response

ArcGIS Server

Feature service

ArcGIS Server

GeoEvent Processor

Data Close

Connection Establish

Connection

Stream Layer Lifecycle

Stream Layer (Constructor) new StreamLayer( featureCollection, options )

FeatureCollection: { layerDefinition: { geometryType: esriGeometryPoint, timeInfo: { startTimeField: “StartTime”, trackIdField: “Name” }, fields: [ … ] }, featureSet: null }

• Options • webSocketUrl:

ws://gep:6180/urlpath • purgeOptions:

{ displayCount: 500 }

Map StreamLayer

ArcGIS Server

GeoEvent Processor

Stream Layer Lifecycle

{ geometry: { x:-77, y:42, spatialReference: { wkid:4326}, attributes: { name:”Buffy”}}

Stream Layer

• Parses message to generate geometry and attributes

• Emits message event

• Adds graphic to layer (Symbol generated using layer renderer)

• Emits graphic-add event

• Checks if number of graphics is over threshold

• Removes oldest graphic if necessary

• Emits graphic-remove event if necessary

(WebSocket.onmessage) ArcGIS Server

GeoEvent Processor

Stream Layer What is needed

• GeoEvent Processor Output Connector - Feature JSON over Web Socket

• Browser that supports Web Sockets http://caniuse.com/websockets

• Web Socket protocol allowed on network ws://, wss://

10 +

6 +

6 +

14 +

<meta http-equiv="X-UA-Compatible" content="IE=10" />

Demonstration Consuming Streams of Features with the Stream Layer

Demo

Ryan Elliott & James Cardona

Road Ahead Stream Services

Stream Services – The Future of Streaming Data

• Developer Productivity - Provides all the metadata needed by clients to find and consume a stream of

features.

• Customizable - Individualized client connections provide filtering and projection.

• Scalable - Features published to a stream services are accessible from any machine in the

cluster.

Stream Services – The Future of Streaming Data

GeoEvent Processor

Stream Layer Subscription changes

Subscribe ws://

Subscription changes

Subscribe ws://

Stream Layer

Stream Service

Stream Service

ArcGIS Server

ArcGIS Server

GeoEvent Processor

Demonstration Stream Services

Demo

What We Covered Today

• Consumed Live data from Sensors and a Web Service • Filtered and generated incidents from spatial behavior • Pushed events to a web app through feature services and web sockets

• Used JavaScript API Stream Layer to receive messages pushed from server

and display them on a map • Saw a preview of the Stream Service that will allow developers to easily receive

data through a web socket and set filters that are processed on the server

ArcGIS GeoEvent Processor

• Use Cases for Applying Real-Time Analytics Using ArcGIS GeoEvent Processor Tuesday 4:00pm - 5:00pm Primrose B

• ArcGIS GeoEvent Processor—An Introduction Wednesday 10:30am - 11:30am Primrose B

• Extending ArcGIS GeoEvent Processor with New Connectors Wednesday 1:00pm - 2:00pm Pasadena/Ventura/Siearra

• The Internet of Things (IoT) and ArcGIS GeoEvent Processor Wednesday 4:00pm - 5:00pm Primrose C/D

• Extending ArcGIS GeoEvent Processor with New Processors Wednesday 5:30pm - 6:30pm Primrose C/D

• Use Cases for Applying Real-Time Analytics Using ArcGIS GeoEvent Processor Thursday2:30pm - 3:30pm Primrose C/D

Additional Workshops

ArcGIS API for JavaScript

• ArcGIS API for JavaScript: What Have You Done for Me Lately? Tuesday 5:30 - 6:30 Oasis 4 and Thursday 2:30 – 3:30 Catalina/Madera

• Transitioning to JavaScript: What to Expect and How to Quickly Come Up to Speed Tuesday 5:30 – 6:30 Primrose B

• Introduction to ArcGIS API for JavaScript Wednesday 2:30 – 3:30 Primrose B

• Tips and Tricks for Debugging Apps Built with ArcGIS API for JavaScript Wednesday 4:00 – 5:00 Catalina/Madera and Thursday 2:30 – 3:30 Primrose A

• Dojo: The Good Parts Thursday 10:00 – 11:00 Smoketree F and Thursday 2:30 – 3:30 Smoketree F

• Introduction to the ArcGIS WebApp Builder: JavaScript Apps Made Easy Thursday 1:00 – 2:00 Primrose B

Additional Workshops

Additional resources for GeoEvent Processor

• Resource Center http://pro.arcgis.com/share/geoevent-processor

• Forum http://forums.arcgis.com/forums/257-GeoEvent-Processor

Additional resources

• ArcGIS API for JavaScript Resource Center https://developers.arcgis.com/javascript

• Code Samples https://github.com/jcardonadcdev/DevSummit2014-realtime

• GeoEvent Processor Tutorial for Web Sockets (Coming soon) In the GeoEvent Processor Gallery

https://developers.arcgis.com/javascript

Questions / Feedback? To learn more:

James Cardona | Web Developer DC Development Center

jcardona@esri.com

Ryan Elliott | Software Engineer ArcGIS GeoEvent Processor for Server

relliott@esri.com

http://pro.arcgis.com/share/geoevent-processor