real-time gis:best practices - esri€¦ · real-time gis. integration and exploitation of...
TRANSCRIPT
![Page 1: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/1.jpg)
Real-Time GIS:Best Practices
C. Adam MollenkopfReal-Time GIS Capability Lead
[email protected]@amollenkopf
RJ SundermanProduct Engineer
GeoEvent [email protected]
![Page 2: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/2.jpg)
• Input and Output Connector usage
• Where, when, and why GeoEvent Definitions are created
• Feature Services and Stream Services workflows
• Performance
• High Availability & Scalability
What are we going to discuss?Best Practices on
![Page 3: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/3.jpg)
Real-Time GISIntegration and exploitation of streaming data
• Integrates real-time streaming data into ArcGIS
• Performs continuous processing and real-time analytics
• Sends updates and alerts to those who need it where they need it
ArcGIS Server
GeoEvent Extension
DesktopWeb Device
![Page 4: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/4.jpg)
Best Practice:How do I get my real-time data into ArcGIS?
![Page 5: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/5.jpg)
How do I get my real-time data into ArcGIS?Easily integrate real-time streaming data into ArcGIS using an Input Connector
Use an existing connector
You can createyour own
connectors.
GeoEvent Extension
Inpu
ts
Out
puts
GeoEvent Services
Poll an ArcGIS Server for Features
Poll an external website for GeoJSON, JSON, or XML
Receive Features, GeoJSON, JSON, or XML on a REST endpoint
Receive RSS
Receive GeoJSON or JSON on a WebSocket
Receive Text from a TCP or UDP Socket
Subscribe to an external WebSocket for GeoJSON or JSON
Watch a Folder for new CSV, GeoJSON, or JSON Files
Out
of t
he B
ox REST
.csv
WS
WS
HTTP
Esri
Gal
lery
ActiveMQ
CAP
Exploitation Support Data
Cursor-on-Target
RabbitMQ
NMEA 0183
MQTT
Sierra Wireless (RAP)
KML
Kafka *
Trimble (TAIP)
Part
ner G
alle
ry
CompassLDE
enviroCar
GNIP
FAA (ASDI)
exactEarth AIS
Zonar
Valarm
Networkfleet
OSIsoft ***
*
*
![Page 6: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/6.jpg)
How do I get my real-time data into ArcGIS?Easily integrate real-time streaming data into ArcGIS using an Input Connector
Input Connector = Adapter + Transport
GeoEvent Extension
Inpu
ts
Out
puts
GeoEvent Services
WebSocket JSON
Connectors Transport Adapter
HTTP GeoJSON
Feature Service
Transports
File
HTTP
HTTP+BasicAuth
HTTP+OAuth
TCP
UDP
WebSocket
Receive JSON on a WebSocket
Receive GeoJSON on a REST endpoint
WS
REST
Feature-JSON
Adapters
GeoJSON
JSON
RSS
Text
XML
Esri
Gal
lery ActiveMQ
IRC
Kafka
MQTT*
RabbitMQ
CAP
Cursor-on-Target
GeoMessage
NMEA
Sierra Wireless (RAP)
Trimble (TAIP)
VMF
Esri
Gal
lery
![Page 7: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/7.jpg)
How do I get my real-time data into ArcGIS?Easily integrate real-time streaming data into ArcGIS using an Input Connector
Adjust an existing connector
• You can copy any of the out-of-the-box connectors and tailor their properties.
![Page 8: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/8.jpg)
How do I get my real-time data into ArcGIS?Easily integrate real-time streaming data into ArcGIS using an Input Connector
![Page 9: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/9.jpg)
How do I get my real-time data into ArcGIS?Easily integrate real-time streaming data into ArcGIS using an Input Connector
![Page 10: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/10.jpg)
How do I get my real-time data into ArcGIS?Configure a new connector using an out-of-the-box adapter / transportEasily integrate real-time streaming data into ArcGIS using an Input Connector
![Page 11: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/11.jpg)
How do I get my real-time data into ArcGIS?Download a connector, adapter, or transport from an Esri repositoryEasily integrate real-time streaming data into ArcGIS using an Input Connector
• https://github.com/Esri/solutions-geoevent-java• http://links.esri.com/geoevent-gallery
![Page 12: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/12.jpg)
• Use an existing inbound connector available out-of-the-box
• Copy and customize an out-of-the-box connector to meet your needs
• Configure a new connector using available adapters and/or transports
• Download connectors, adapters, and/or transports from EsriGalleries and GitHub repositories
- Source code is available from GitHub for customization or to use asa reference
• Develop your own adapter / transport using the GeoEvent SDK- The SDK and a Developer’s Guide are included in the
GeoEvent extension product’s installation folder
How do I get my real-time data into ArcGIS?Easily integrate real-time streaming data into ArcGIS using an Input Connector
![Page 13: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/13.jpg)
Best Practice:How do I update & alert those who need it where they need it?
![Page 14: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/14.jpg)
How do I update and alert those who need it where they need it?Easily disseminate notifications, alerts, and updates using an Output Connector
You can createyour own
connectors.
GeoEvent Extension
Inpu
ts
Out
puts
GeoEvent Services
Out
of t
he B
oxAdd or Update a feature
Publish Text to a UDP Socket
Send a Text Message
Send an Email
Push Text to an external TCP Socket
Push GeoJSON or JSON to an external WebSocket
Push GeoJSON or JSON to an external Website
Send an Instant Message
Send Features to a Stream Service
Write to a CSV, GeoJSON, or JSON File .csv
WS
im
HTTP
CESIUM
Part
ner
*
ActiveMQ
Esri
Gal
lery
Cursor-on-Target
Hadoop
Kafka
MongoDB
MQTT
*
RabbitMQ
![Page 15: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/15.jpg)
How do I update and alert those who need it where they need it?Output Connector = Adapter + Transport
GeoEvent Extension
Inpu
ts
Out
puts
GeoEvent Services
Message Formatter SMTP
Connectors TransportAdapter
Feature-JSON Stream Service
Send an Email
Send Features to a Stream Service
Feature Service
Transports
File
HTTP
HTTP+BasicAuth
HTTP+OAuth
SMS
SMTP
Stream Service
TCP
UDP
WebSocket
XMPP
Cache
Adapters
Feature-JSON
GeoJSON
JSON
Message Formatter
RSS
Text
WebSocket
ActiveMQ
Hadoop
IRC
Kafka
MongoDB
MQTTEsri
Gal
lery
*
RabbitMQ
TCP-Squirt
Cursor-on-Target
Esri
Gal
lery
CESIUM
Part
ner
*
![Page 16: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/16.jpg)
Best Practice:When and why are GeoEvent Definitions created?
![Page 17: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/17.jpg)
- Connect an output to your feature
- Import the schema of your feature as a GeoEvent Definition
- Configure an input to receive real-time data
- Author and publish a GeoEvent Service
- Visualize your real-time feature GeoEvent Extension
Out
puts
Inpu
ts
GeoEvent Services
ArcGIS Server
Operations Dashboard for ArcGIS
operation views
web maps
ArcGIS Online /Portal for ArcGIS
feature services
GeoEvent Definitions
When and why are GeoEvent Definitions created?Authoring GeoEvent Definitions – Importing from a feature service
![Page 18: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/18.jpg)
When and why are GeoEvent Definitions created?Authoring GeoEvent Definitions – Creating using GeoEvent Manager
You own GeoEvent Definitions you either import or create manually using the GeoEvent Manager
![Page 19: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/19.jpg)
When and why are GeoEvent Definitions created?Auto-generated GeoEvent Definitions – Created by an inbound connector
• The adapter will make a best-guess based on data discovered in the first event received• You can copy and tailor auto-generated GeoEvent Definitions to meet your needs
- But you should not edit an event definition owned or created by an adapter
![Page 20: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/20.jpg)
• A managed GeoEvent Definition is created when a processor adds or removes a field from a GeoEvent
• Publishing changes to a GeoEvent Service signals the owning processor to delete its managed GeoEvent Definition
• This can present you with issues you need to work around…- Field Mapping
- Stream Services
- Custom Processors
When and why are GeoEvent Definitions created?Managed GeoEvent Definitions – Created by a processor from within a GeoEvent Service
- Field Calculator - Geotagger
- Field Enricher - Buffer Creator (etc…)
Be aware of the GeoEvent Definition associated with your GeoEvent(s) at every stage of your event processing
For more information see ‘Understanding GeoEvent Definitions’ blog:http://tinyurl.com/o83wjcv
![Page 21: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/21.jpg)
Best Practice:When should I use a Feature Service vs. a Stream Service?
![Page 22: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/22.jpg)
When should I use Feature Services vs. Stream Services?Two patterns
GeoEvent Extension
ArcGIS Server
YourApplications
Stream Layer
Map ServicesFeature Services
Feature Layer
feature layers
Update a FeatureAdd a Feature
EGDB
Polling(Pull)
Stream Services
• Feature layers pull from feature services- Web apps poll to get periodic updates- Must be backed by an enterprise geodatabase (EGDB)
• Stream layers subscribe to stream services - Web apps subscribe to immediately receive data- Low latency and high throughput
Send Features to a Stream Service
![Page 23: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/23.jpg)
- Connect an output to your feature
- Import the schema of your feature as a GeoEvent Definition
- Configure an input to receive real-time data
- Author and publish a GeoEvent Service
- Visualize your real-time feature GeoEvent Extension
Out
puts
Inpu
ts
GeoEvent Services
ArcGIS Server
Operations Dashboard for ArcGIS
operation views
web maps
ArcGIS Online /Portal for ArcGIS
feature services
GeoEvent Definitions
What is the recommended workflow for working with Feature Services?Feature Service workflow
![Page 24: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/24.jpg)
Inpu
ts
ArcGIS Server
What is the recommended workflow for working with Stream Services?Stream Service workflow
- Configure an input to receive real-time data
- Define a GeoEvent Definition
- When creating an output publish a Stream Service and connect the output to it
- Author and publish a GeoEvent Service
- Visualize your real-time stream service
- Use an ArcGIS Online WebMap
- Use a custom JavaScript web app
GeoEvent Extension
GeoEvent Services
web maps
ArcGIS Online /Portal for ArcGIS
Out
puts
stream services
GeoEvent Definitions
![Page 25: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/25.jpg)
Performance
![Page 26: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/26.jpg)
How do I measure Performance?performance-test-harness-for-geoevent
https://github.com/Esri/performance-test-harness-for-geoevent
![Page 27: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/27.jpg)
Throughput Performance Benchmark @ 10.3Two times more throughput than 10.2.2
0
500
1000
1500
2000
2500
3000
3500
Event Throughput
10.2.2 10.3
Thro
ughp
ut =
Pro
cess
ed #
Eve
nts
per S
econ
d
Velocity = # of produced Events per Second
3,000
1,500
As captured on primary benchmarking machine using ArcGIS 10.3
Two times morethroughput than 10.2.2
![Page 28: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/28.jpg)
Geofencing Performance Benchmark @ 10.3US States benchmark – 51 geofences with 1,617 vertices on average (78 min / 21,970 max)
![Page 29: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/29.jpg)
Geofencing Performance Benchmark @ 10.3US Congressional Districts benchmark – 436 geofences with 512 vertices on average (24 min / 7,285 max)
![Page 30: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/30.jpg)
Geofencing Performance Benchmark @ 10.3US Counties benchmark = 3,143 geofences with 166 vertices on average (9 min / 838 max)
![Page 31: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/31.jpg)
0.00
500.00
1000.00
1500.00
2000.00
2500.00
3000.00
250 500 750 1000 1250 1500 1750 2000 2250 2500 2750 3000
US States51 Geofences
1,617 vertices on average (78 min and 21,970 max)
Disjoint Any Inside Any Intersect Any Outside Any Touches Any Within Any
Geofencing Performance Benchmark @ 10.3US States benchmark
2,4992,245
Operator10.3
events per second
10.2.2events
per secondDisjoint Any 2,499
Inside Any 2,488 150
Intersect Any 2,486
Within Any 2,482
Touches Any 2,248
Outside Any 2,245
16 times fasterthan 10.2.2
150Thro
ughp
ut =
Pro
cess
ed #
Eve
nts
per S
econ
d
Velocity = # of produced events per second
As captured on primary benchmarking machine using ArcGIS 10.3
![Page 32: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/32.jpg)
0.00
500.00
1000.00
1500.00
2000.00
2500.00
3000.00
250 500 750 1000 1250 1500 1750 2000 2250 2500 2750 3000
US Congressional Districts436 Geofences
512 vertices on average (24 min and 7,285 max)
Disjoint Any Inside Any Intersect Any Outside Any Touches Any Within Any
2,249
Geofencing Performance Benchmark @ 10.3US Congressional Districts benchmark
As captured on primary benchmarking machine using ArcGIS 10.3note: this scenario was not benchmarked at 10.2.2
Velocity = # of produced events per second
Thro
ughp
ut =
Pro
cess
ed #
Eve
nts
per S
econ
d
Operator10.3
events per second
Disjoint Any 2,249
Outside Any 2,248
Intersect Any 2,248
Touches Any 2,244
Within Any 2,244
Inside Any 2,244
![Page 33: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/33.jpg)
0.00
500.00
1000.00
1500.00
2000.00
2500.00
3000.00
250 500 750 1000 1250 1500 1750 2000 2250 2500 2750 3000
US Counties3,143 Geofences
166 vertices on average (9 min and 838 max)
Disjoint Any Inside Any Intersect Any Outside Any Touches Any Within Any
1,7401,997
Geofencing Performance Benchmark @ 10.3US Counties benchmark
17 times fasterthan 10.2.2
130
Velocity = # of produced events per second
Thro
ughp
ut =
Pro
cess
ed #
Eve
nts
per S
econ
d
Operator10.3
eventsper second
10.2.2 events
per secondDisjoint Any 1,997
Intersect Any 1,996
Outside Any 1,992
Touches Any 1,747
Inside Any 1,742 130
Within Any 1,740As captured on primary benchmarking machine using ArcGIS 10.3
![Page 34: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/34.jpg)
What are the primary factors I should consider for Performance?Operating environment, Network, RAM, and Processors
• Operating environment: Bare Metal = Windows 7 (64 bit) Enterprise- Virtual Machines – beware! resources need to be shared in an effective way, like EC2.- Bare-Metal machines – dedicated resources are much more deterministic.
• Network- Speed (Mbps) – the faster the better. 1 Gbps
• RAM- size (GB) – 8GB is required at 10.3. 16GB, defaulted JVM max heap size of 4GB used- type – minimum of DDR3 is recommended. DDR3- clock speed (MHz) and transfer rate (Mbps) – the faster the better.
• Processors- speed (GHz) – the faster the better. 3.70GHz, Intel Xeon E5-1620 v2- # of cores – the more the better. 4 physical 8 Virtual
![Page 35: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/35.jpg)
High Availability & Scalability
![Page 36: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/36.jpg)
High Availability & ScalabilityClustering• Clusters administered via ArcGIS Server Manager
- Site, Cluster(s), Machines
• Machines in a cluster share configuration- automatic provisioning upon joining a cluster- including custom components
• High Availability is achievable out-of-the-box
• Scale-out by adding machines to a cluster000
![Page 37: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/37.jpg)
High AvailabilitySite, cluster(s), and machines
![Page 38: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/38.jpg)
High AvailabilityDistributed configuration store
![Page 39: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/39.jpg)
High AvailabilityDistributed configuration store
![Page 40: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/40.jpg)
High AvailabilityDistributed configuration store
![Page 41: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/41.jpg)
High AvailabilityDistributed configuration store
![Page 42: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/42.jpg)
High AvailabilityDistributed configuration store
![Page 43: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/43.jpg)
High AvailabilityDistributed configuration store
![Page 44: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/44.jpg)
High AvailabilityDistributed configuration store
![Page 45: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/45.jpg)
High AvailabilityDistributed configuration store
![Page 46: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/46.jpg)
High AvailabilityDistributed configuration store
![Page 47: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/47.jpg)
ScalabilityOutput transports
![Page 48: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/48.jpg)
ScalabilityInput transports
![Page 49: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/49.jpg)
ScalabilityInput transports that are automatically load balanced
![Page 50: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/50.jpg)
ScalabilityInputs transports that require you to bring your own load balancer
![Page 51: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/51.jpg)
Scalabilitycluster-simulator-for-geoevent
https://github.com/Esri/cluster-simulator-for-geoevent
000
![Page 52: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/52.jpg)
ScalabilityInputs and distributed stream processing
a1
a1
a1 a1 a1
a1
c3 c3
c3
c3
c3
c3b2
b2
b2
b2
b2 b2
a1
a1
![Page 53: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/53.jpg)
High AvailabilityInputs and distributed stream processing
a1
c3
b2
![Page 54: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/54.jpg)
High AvailabilityInputs and distributed stream processing
c3
c3
c3
c3 c3
c3
b2
b2 b2 b2
b2
b2
![Page 55: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/55.jpg)
ScalabilityPinned inputs
![Page 56: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/56.jpg)
ScalabilityPinned inputs and distributed stream processing
a1b2
a1 a1
a1
b2 b2 b2c3
c3 c3
c3
a1
c3
b2
a1
![Page 57: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/57.jpg)
High AvailabilityPinned inputs and distributed stream processing
a1
c3
b2
![Page 58: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/58.jpg)
High AvailabilityPinned inputs and distributed stream processing
a1
a1 a1
a1
b2 b2b2
b2
c3
c3 c3
c3
a1c3
b2
![Page 59: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/59.jpg)
https://github.com/Esri/performance-test-harness-for-geoevent
How do I measure Cluster Performance?performance-test-harness-for-geoevent
![Page 60: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/60.jpg)
ScalabilityClustering for increased throughput• Clusters administered via ArcGIS Server Manager
- Site, Cluster(s), Machines
• Scale-out by adding machines to a cluster
0
1000
2000
3000
4000
5000
6000
1000 2000 3000 4000 5000 6000 7000
Event Throughput with Clustering
1 machine 2 machines 3 machines
Thro
ughp
ut =
Pro
cess
ed #
Eve
nts
per S
econ
d
Velocity = # of produced Events per Second
3,900
As captured on 10.3 benchmarking cluster using ArcGIS 10.3
3,000
4,800
![Page 61: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/61.jpg)
Stream Services:High Availability & Scalability
![Page 62: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/62.jpg)
ScalabilityScaling out Stream Services to support an increased # of concurrent users
![Page 63: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/63.jpg)
High AvailabilityStream Services concurrent user failover
![Page 64: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/64.jpg)
High AvailabilityStream Services concurrent user failover
![Page 65: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/65.jpg)
High AvailabilityStream Services concurrent user failover
![Page 66: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/66.jpg)
ScalabilityScaling out Stream Services to support an increased # of concurrent users with a reverse proxy
![Page 67: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/67.jpg)
High AvailabilityStream Services concurrent user failover with a reverse proxy
![Page 68: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/68.jpg)
High AvailabilityStream Services concurrent user failover with a reverse proxy
![Page 69: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/69.jpg)
GeoEvent Extension: Best PracticesSummary
• ArcGIS is a dynamic platform that enables continuous analytics and real-time visualization for better understanding of our world.
• The ArcGIS GeoEvent Extension for Server allows you to:- know what is happening, as it happens- react and make smarter decisions faster- be notified when interesting events occur
![Page 70: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/70.jpg)
Other Real-Time GIS Sessions• Real-Time GIS: GeoEvent Extension
- Tue 10:15-11:30am, Room 17 A- Wed 10:15-11:30am, Room 14 B
• Real-Time GIS: Leveraging Stream Services- Tue 8:30-9:45am, Room 01 A/B- Wed 8:30-9:45am, Ballroom 06 D
• Real-Time GIS: Applying Real-Time Analytics- Tue 10:15-11:30am, Room 15 B- Wed 8:30-9:45am, Room 14 B
• Real-Time GIS Use Cases and Implementation Patterns- Tue 2:30-3:15PM, Demo Theater 6 - Geodata
• ArcGIS Intelligence: Discern Activities of Interest through Advanced Analytics- Wed 10:15-11:30am, Omni Ballroom A/B
• Real-Time GIS: The Road Ahead- Wed 1:30-2:45pm, Room 14 B
• Real-Time GIS: Best Practices- Thu 8:30-9:45am, Room 14 B
• Real-Time GIS for Asset Readiness, Event Preparation, and Intervention- Thu 8:30-9:45am, Room 29 C
![Page 71: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/71.jpg)
Thank you…• Please fill out the session survey in your mobile app• Select session in the Mobile App
- Use the Search Feature to quickly find this title
• Click “Technical Workshop Survey”• Answer a few short questions and enter any comments
![Page 72: Real-Time GIS:Best Practices - Esri€¦ · Real-Time GIS. Integration and exploitation of streaming data • Integrates real -time streaming data into ArcGIS • Performs continuous](https://reader031.vdocuments.us/reader031/viewer/2022011822/5ec9d93e233be5791b227465/html5/thumbnails/72.jpg)
Questions / Feedback?
http://links.esri.com/geoeventhttps://links.esri.com/geoevent-forum
To learn more:
C. Adam MollenkopfReal-Time GIS Capability Lead
[email protected]@amollenkopf
RJ SundermanProduct Engineer
GeoEvent [email protected]
https://github.com/Esri/performance-test-harness-for-geoeventhttps://github.com/Esri/cluster-simulator-for-geoevent