![Page 1: Utilizing Data Reviewer and Python to Develop a …proceedings.esri.com/library/userconf/proc17/papers/78_67.pdf · Python to Develop a Quality Control System ... Architecture GIS](https://reader031.vdocuments.us/reader031/viewer/2022013015/5b8991ad7f8b9ae7298cbfb5/html5/thumbnails/1.jpg)
Utilizing Data Reviewer and Python to Develop a Quality
Control System
Jonah Waterman
GIS Technician 2, GISP
Greenville Utilities
![Page 2: Utilizing Data Reviewer and Python to Develop a …proceedings.esri.com/library/userconf/proc17/papers/78_67.pdf · Python to Develop a Quality Control System ... Architecture GIS](https://reader031.vdocuments.us/reader031/viewer/2022013015/5b8991ad7f8b9ae7298cbfb5/html5/thumbnails/2.jpg)
Greenville Utilities since 1905
• Provide Electric, Water, Sewer, and Gas Utilities
• Over 150,000 customers
• 600+ Miles of Water Mains
• 1000+ Miles of Gas Mains
• 500+ Miles of Sewer Mains
• 3000+ Miles of Electric Lines
• About 500 employees
![Page 3: Utilizing Data Reviewer and Python to Develop a …proceedings.esri.com/library/userconf/proc17/papers/78_67.pdf · Python to Develop a Quality Control System ... Architecture GIS](https://reader031.vdocuments.us/reader031/viewer/2022013015/5b8991ad7f8b9ae7298cbfb5/html5/thumbnails/3.jpg)
History
• Natural transition from analog data
• Paradigm shift: “Not just making pretty maps anymore”
• Higher demand for geospatial data
• Multiple system integrations, increased interoperability
GIS
(Geographic Information System)
• Since 90’s
• SQL database
• Silo to enterprise
• Mobile, interactive, traditional
• SCADA, Responder, Customer
Service
WAM
(Work and Asset Management)
• Active since 2015
• Oracle database
• Tracks an Asset lifecycle
• Inspections, Asset
specifications
• Basically metadata about your
Assets
![Page 4: Utilizing Data Reviewer and Python to Develop a …proceedings.esri.com/library/userconf/proc17/papers/78_67.pdf · Python to Develop a Quality Control System ... Architecture GIS](https://reader031.vdocuments.us/reader031/viewer/2022013015/5b8991ad7f8b9ae7298cbfb5/html5/thumbnails/4.jpg)
Architecture
GIS
Geonexus
Quality Control Tool: Data Reviewer
XY Location Asset Information
Management
Quality Control
???
WAM
![Page 5: Utilizing Data Reviewer and Python to Develop a …proceedings.esri.com/library/userconf/proc17/papers/78_67.pdf · Python to Develop a Quality Control System ... Architecture GIS](https://reader031.vdocuments.us/reader031/viewer/2022013015/5b8991ad7f8b9ae7298cbfb5/html5/thumbnails/5.jpg)
Architecture
GIS
Geonexus
WAM
GIS Orphans
WAM Orphans
• Extensional Toolbar within ArcMap
• Available through ArcGIS Server
• Over 40 Checks available
• Single Checks or Batch Checks available
• Can be automated through scripting
• Outputs: reports or spatial violation locations
• Compare records between tables
![Page 6: Utilizing Data Reviewer and Python to Develop a …proceedings.esri.com/library/userconf/proc17/papers/78_67.pdf · Python to Develop a Quality Control System ... Architecture GIS](https://reader031.vdocuments.us/reader031/viewer/2022013015/5b8991ad7f8b9ae7298cbfb5/html5/thumbnails/6.jpg)
Data Reviewer
Table to Table Attribute
![Page 7: Utilizing Data Reviewer and Python to Develop a …proceedings.esri.com/library/userconf/proc17/papers/78_67.pdf · Python to Develop a Quality Control System ... Architecture GIS](https://reader031.vdocuments.us/reader031/viewer/2022013015/5b8991ad7f8b9ae7298cbfb5/html5/thumbnails/7.jpg)
Model Builder Approach: 1st Run
Framework
• 2 Tables
• GIS Table: All FC
that participate
• WAM Table: All
participating
Assets
Limitations
• Cumbersome
• Time Consuming
• “Buggy”
• Troubleshooting
![Page 8: Utilizing Data Reviewer and Python to Develop a …proceedings.esri.com/library/userconf/proc17/papers/78_67.pdf · Python to Develop a Quality Control System ... Architecture GIS](https://reader031.vdocuments.us/reader031/viewer/2022013015/5b8991ad7f8b9ae7298cbfb5/html5/thumbnails/8.jpg)
Python Approach
![Page 9: Utilizing Data Reviewer and Python to Develop a …proceedings.esri.com/library/userconf/proc17/papers/78_67.pdf · Python to Develop a Quality Control System ... Architecture GIS](https://reader031.vdocuments.us/reader031/viewer/2022013015/5b8991ad7f8b9ae7298cbfb5/html5/thumbnails/9.jpg)
Frequency by Asset
• Asset at Risk
• Overlooked Assets
• Workflow guidance
![Page 10: Utilizing Data Reviewer and Python to Develop a …proceedings.esri.com/library/userconf/proc17/papers/78_67.pdf · Python to Develop a Quality Control System ... Architecture GIS](https://reader031.vdocuments.us/reader031/viewer/2022013015/5b8991ad7f8b9ae7298cbfb5/html5/thumbnails/10.jpg)
Timeline
• Abrupt changes
• KPI Overtime
• Upper Management Tool
![Page 11: Utilizing Data Reviewer and Python to Develop a …proceedings.esri.com/library/userconf/proc17/papers/78_67.pdf · Python to Develop a Quality Control System ... Architecture GIS](https://reader031.vdocuments.us/reader031/viewer/2022013015/5b8991ad7f8b9ae7298cbfb5/html5/thumbnails/11.jpg)
Violation Map
• Spatial Identification
• Clustering
• Spatial Relationships
• Interactive quality control
![Page 12: Utilizing Data Reviewer and Python to Develop a …proceedings.esri.com/library/userconf/proc17/papers/78_67.pdf · Python to Develop a Quality Control System ... Architecture GIS](https://reader031.vdocuments.us/reader031/viewer/2022013015/5b8991ad7f8b9ae7298cbfb5/html5/thumbnails/12.jpg)
Road Ahead
• Continual Monitoring
• Providing SOP’s
• Workflow guidance
• Increased communication
• Interactive quality control
Jonah Waterman
GIS Technician 2, GISP
Greenville Utilities
252.717.0057