ArcGIS Data Reviewer:

Overview and Success Stories

Chandan Banerjee

cbanerjee@esri.com

Agenda

• Importance of quality

• Esri platform for data quality control

• Demo – Using ArcGIS Data Reviewer for Electric utility

• Customer examples

• Esri resources/support

Defining quality A business perspective

• Executive

- Confidently make decisions

- Reduce financial risk

- Optimize organizational performance

Defining quality A business perspective

• Executive

- Confidently make decisions

- Reduce financial risk

- Optimize organizational performance

• Manager

- Effective data stewardship

- Drive increased usage of platform

- Maximize productivity

Defining quality A business perspective

• Executive

- Confidently make decisions

- Reduce financial risk

- Optimize organizational performance

• Manager

- Effective data stewardship

- Drive increased usage of platform

- Maximize productivity

• Knowledge worker

- Increased efficiencies

- Confidence in GIS

Defining Quality A technical perspective

Spatial Accuracy

Thematic Accuracy

Completeness

Logical Consistency

Temporal Quality

Usability

ISO-19157:2013 Geographic Information – Data Quality

What is ArcGIS Data Reviewer? Data Quality Management for ArcGIS

• Provides

- Rule-based validation

- Interactive tools

- Track errors

• For individuals and enterprise

- Saves time/money

- Less rework

• For multiple domains

- Configurable

- Extendable

Validate

Review Summarize

What is ArcGIS Data Reviewer? Data quality management in the ArcGIS platform

• Data Reviewer for Desktop

- ArcMap

- ArcGIS Pro

• Data Reviewer for Server

- Standard or higher

• Data Reviewer API for

- FLEX

- JavaScript

Server Online Content and

Services

Portal

Desktop Web Device

What is ArcGIS Data Reviewer? Types of Quality Control

Fast

Consistent and

repeatable

Objective

100% coverage

Automated review

What is ArcGIS Data Reviewer? Types of Quality Control

Fast

Consistent and

repeatable

Objective

100% coverage

Automated review

As subjective as

needed

Better for finding

patterns and missing

elements

Visual review

What is ArcGIS Data Reviewer? Data Quality Reporting

REVIEW

Find &

Record Errors

CORRECT

Perform Edits or

Note Exceptions

VERIFY

Acceptable or

Unacceptable

What is ArcGIS Data Reviewer? Automating Data Validation

• Implementing quality requirements

- 40+ configurable checks

- Feature integrity

- Collection rules

- Attribute

- Feature and table values

- Spatial

- Spatial relationships

- Metadata

- Completeness/Content

www.esri.com/datareviewer

Examples of Data Reviewer Checks Utilized by Electric and Gas Utilities

Check Name Check Description Use Case Example

Domain Validates coded value and range domains to ensure that all values meet domain constraints Useful when migrating GPS-collected data and data

from another format, such as CAD, shapefile, or coverage, into the geodatabase

Subtype Searches for feature classes with improper or null (optional) subtypes

Connectivity Rules Finds features that are part of a geometric network and that violate connectivity rules

Connectivity rules are an important aspect of a network. Identifying features that violate geometric network rules and resolving these violations will enhance data integrity. This helps functions such as trace that use the geometric network.

Relationships Searches for records that are orphans or have improper cardinality in a relationship class

Knowing the exact number of switches and switch units is important. This check can find orphan switch units that do not have a relationship to a switch

Examples of Data Reviewer Checks Utilized by Electric and Gas Utilities

Check Name Check Description Use Case Example

Duplicate Geometry

Finds features of the same geometry type that are colocated and optionally share attributes (Features can be either from two different feature classes or within the same feature class.)

Find locations where two or more junctions in a geometric network are on top of each other. Only one of those junctions can actually be connected to the network. These duplicate features can be identified and addressed to ensure proper connectivity of network features.

Geometry on Geometry

Finds features that have a specific spatial relationship, either from two different feature classes or within the same feature class; for example, finding transformers on top of switches

Find transformers that are connected to primary lines and compare the phase designation. If the phases do not match, it is reported as an error.

Valency

Validates relationships between point and line features or line features within the same feature class, from ensuring that a point has a specified number of lines connected to it to ensuring that specific patterns of features are met with valency

Validates relationships; for example, open point should be connected to two primary lines or a fuse must be connected to one primary line and one secondary line of matching phases.

What is ArcGIS Data Reviewer? Methods for executing automated validation

Execute data validation using

• ArcMap

• ArcGIS Pro

What is ArcGIS Data Reviewer? Methods for executing automated validation

Execute data validation using

• ArcMap

• ArcGIS Pro

• Model/Python script

What is ArcGIS Data Reviewer? Methods for executing automated validation

Execute data validation using

• ArcMap

• ArcGIS Pro

• Model/Python script

• ArcGIS Workflow Manager

What is ArcGIS Data Reviewer? Methods for executing automated validation

Execute data validation using

• ArcMap

• ArcGIS Pro

• Model/Python script

• ArcGIS Workflow Manager

• ArcGIS for Server

Why Data Reviewer? Automating data quality control

• Reduces resource requirements

- Eliminates cost associated with

custom code

- Implementable by non-developers

• Consistency

- Repeatable processes

- Shareable for consistency between

distributed teams

QC Results

Automated

Review

Quality

Reporting

Visual

Review

Why Data Reviewer? Increased data quality transparency

• Lifecycle management process

- Efficiencies in corrective workflows

- Minimizes re-work

• Raises awareness of liabilities

- Technical DQ reports

- Dashboard metrics

QC Results

Automated

Review

Quality

Reporting

Visual

Review

Automating Data

Validation

Powerco New Zealand

Data quality management for Electric and Gas

• Who

- New Zealand’s largest electricity and second largest gas distributor

in terms of network length (electricity 30,000km, gas 6,170km)

- 320,000 electricity connections

- 102,000 gas connections

• Business Problem

- Mergers resulting in

- multiple data management systems

- Different data formats/standards

- Incomplete/inaccurate data after consolidation

• Enabling capabilities

- Rule-based workflows to control the data flow

- Record, track and visualize errors

- Data quality reporting

City of Sioux Falls, SD Data quality management for Water Utilities

• Who

- City population of 160,000 residents

- The GIS Department maintains the water

utility data for the city

• Business Problem

- In-house data management with limited

resources to perform integrity checks

• Enabling capabilities

- Automated data quality using LG

templates

- Data quality reporting

“Use of Data Reviewer has lead

to increased data integrity and

confidence” – Lauri Sohl, GIS Analyst

City of Woodland, California Data quality management for Parcel Data

• Who

- City population of 60,000 residents

• Business Problem

- CAD to GIS conversion created data integrity

issues

• Enabling capabilities

- Automated data quality

- Lifecycle management

- Data quality reporting

“We were no where near where

we wanted to be and needed to

fix that”

“We can now perform QC tasks

and keep our data in top shape”

– Daniel Hewitt, GIS Specialist

San Francisco Estuary Exotic invasive plant management

• Who

- Invasive spartina project (ISP)

• Business Problem

- Validating high-volume field data

• Enabling capabilities

- Automated data quality

- Lifecycle management

- Data quality reporting

… bad data quality could result in

unnecessary expense … , inaccurate

reporting of progress towards

eradication, or just scientifically

questionable data, which would cause

a deterioration of trust in and support

for the project”

– Peggy Olofson, ISP Project Director

Esri Resources Enabling customer success

• Industry templates

- Address Management

- Electrical Utilities

- Roads & Highways

- Tax Parcel Editing

- Water Utilities

• Based on Esri industry

models

• Use as Starting point

Esri Resources Enabling customer success

• Training/Services

- Web courses

- Self-paced

- Instructor-led training

- 2-day

- On-site/Off-site/Virtual offerings

- Jump-Start Packages

- 3-5 day on-site assistance

- Flexible deliverables

- Training and consulting

Esri Resources Enabling customer success

• Using ArcGIS® Data Reviewer to Inspect ArcFM™ Feeder Manager Circuits

• Data Reviewer for Electric Utilities