how we automated the spatializing of data for a web app ... · 2017 esri user conference...
TRANSCRIPT
![Page 1: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/1.jpg)
How We Automated the Spatializing of Data for a Web Map Data PortalAlice PenceGIS & Database SpecialistCity of Auburn, WA
![Page 2: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/2.jpg)
Quick Bio
• BS in Geography from the University of Idaho with a Certificate in GIS
• Been in GIS field about 4 years
• If not at desk making maps, most likely up on a mountain somewhere
or at a Mariners Game
![Page 3: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/3.jpg)
Auburn, WA
• Population: ~ 80,000
• 450 Full-time Employees
• 30 Square Miles
Seattle
Tacoma
![Page 4: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/4.jpg)
BackgroundCity of Auburn Strategic Initiatives: Technology- The Transparent City
![Page 5: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/5.jpg)
Our Crimes Database
• Stored in a Proprietary Linux system
• Raw data with only addresses
• Can’t directly dump into GIS- needs to have data transformed for
public/internal viewing, QA/QC, a spatial component to view on maps
and applications, attributes renamed…
![Page 6: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/6.jpg)
First Impression…
![Page 7: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/7.jpg)
Our MissionHow to view all of this crime data spatially for analysis and reporting?
![Page 8: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/8.jpg)
Our First Plan: Python Script
The Good.• It worked. Can add new offenses, change and update as needed• Ran in windows scheduler to automate.
The Bad.• Long, cumbersome script. • Needed multiple databases (.gbd → SDE) to clean and move data
The Ugly.• Can’t easily change connection to Crimes Database, no direct connection to
source (connected through ArcCatalog -then built through Model Builder- then exported out as a Python Script)
• No custom alerts or in depth logging capabilities.• Not user friendly for non-python users.
![Page 9: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/9.jpg)
Our Next Plan: FME!!
![Page 10: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/10.jpg)
The FME ProcessFrom Linux database to SDE
![Page 11: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/11.jpg)
Easy Database Connections
Over 350 connections available in FME!
![Page 12: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/12.jpg)
Managing Attributes
Python FME
This continues on for a few more pages…
![Page 13: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/13.jpg)
Attribute Mapper Transformer
• As an example: RKBKCrime Type= Felony Crimes Against PersonCrime Name= Robbery
1 2 3
*double bonus: I was able to automatically import codes by a CSV file, AND…FME automatically read in all the source values for me
![Page 14: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/14.jpg)
Attribute Renamer
![Page 15: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/15.jpg)
Geocoder
*double bonus: Not Geocoded Values are routed to Excel spreadsheet for extra review for address.
![Page 16: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/16.jpg)
Into GIS Environment: SDE
![Page 17: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/17.jpg)
The Automation ProcessLETS AUTOMATE!!
![Page 18: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/18.jpg)
FME Server
Email notifications when Workbench succeed or failed
![Page 19: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/19.jpg)
FME Server Logging (FME Jobs)
![Page 20: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/20.jpg)
Comparison
Python FME• Easy to update codes and
field changes• Other users can easily edit-
even w/minimal knowledge of FME
• Share directly to FME Server for logs and alerts
• Friendly user interface for FME Server and easy to set up
• Cumbersome/long script• Not very easy to
customize alerts or logs• Hard to edit- other users
might not know the python language
• No friendly user interface
![Page 21: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/21.jpg)
Demos
Crimes Interactive Public MapOpen Data Portal: Crimes Database
![Page 22: How We Automated the Spatializing of Data for a Web App ... · 2017 Esri User Conference Presentation Keywords: 2017 Esri User Conference—Presentation, 2017 Esri User Conference,](https://reader031.vdocuments.us/reader031/viewer/2022011923/604fde9352277b17f1661290/html5/thumbnails/22.jpg)
Thank You!Alice PenceGIS & Database SpecialistCity of Auburn, [email protected]