please share your feedback in the app...download the esri events app and find your event select the...

Download the Esri

Events app and find your event

Select the session

you attended

Scroll down to

“Survey”

Log in to access the

survey

Complete the survey

and select “Submit”

Please Share Your Feedback in the App

James Sill, Senior Solution Engineer, Esri, Boulder, Co

Jacob Czawlytko, Chesapeake Conservancy, Senior Geospatial Analyst

Kumar Mainali, Chesapeake Conservancy, Geospatial Data Scientist

Automated Feature Extraction In ArcGIS

Agenda

• What is feature extraction in ArcGIS

• Methods for solving the problem

- Unsupervised vs. Supervised

- Collecting and managing training samples in ArcGIS Pro, Enterprise and Online

- Deep Learning in ArcGIS for Feature Extraction

• Real Life Examples

- Supervised classification of Landcover with Raster Analytics

- Integrating external deep learning frameworks into ArcGIS

- Chesapeake Conservancy Landcover Classification

Tools for Feature Extraction from Imagery In ArcGIS

ArcGIS

Classification

Clustering

Prediction

Deep Learning

Machine Learning Tools in ArcGIS

• Pixel & Object Based

• Image Segmentation

• Maximum Likelihood

• Random Trees

• Support Vector Machine

• Empirical Bayesian Kriging

• Areal Interpolation

• EBK Regression Prediction

• Ordinary Least Squares Regression and Exploratory Regression

• Geographically Weighted Regression

Classification

PredictionClustering

• Spatially Constrained Multivariate Clustering

• Multivariate Clustering

• Density-based Clustering

• Hot Spot Analysis

• Cluster and Outlier Analysis

• Space Time Pattern Mining

• Generate training samples

• Detect objects

• Classify pixels

• End to End support

• Apply at Scale over large collections

Deep Learning

Built – in tools for Feature Extraction

in ArcGIS Pro

Pixel or Object Based

Unsupervised

Choose Image

ISO Clusters

Assign Classes

Clean Up

Merge Classes

GOAL

Unsupervised Method:

• ISO Clustering:

- Based on K-means nearest neighbor algorithm

- Quick, least cost classification method

- Good for instances where there is a low familiarity with

composition of area of interest

- Generally low overhead

- Advantages where a lack of resources to create training

samples

- Use cases where unsupervised methods excel…

- Flood extent mapping – ISO/Kmeans clustering to delineate

water from non-water

- Presence absence of vegetation in post wild fire burn

zones

- Distinct differences in spectral composition within your

AOI

Pixel and Object Based

Supervised

Choose ImageTraining Samples

Maximum Likelihood

Accuracy

Classify

GOAL

Forest Based Classification

Support Vector Machine

Supervised

Choose Image

Segmentation

Training Samples

Accuracy

Support Vector

Machine

GOAL

Classify

Object Classification

Supervised Feature Extraction and Pixel Level Classification

• Pixel and Object Based Classification/Identification

- Pixels are classified or objects identified through supervised algorithms

- Support Vector Machine

- Forest Based Classification

- Maximum Likelihood

- Requires that the user collects training samples

- Instances where supervised classification excels:

- Ability and time to create training samples

- Example Use cases where Supervised Classification Excels

- Multiclass landcover classification and feature identification

- Impervious/non-impervious surface mapping

- Static data products that produced in a non – time sensitive environment

• Support Vector Machine classification of

Sage Grouse Habitat in Southwest

Colorado (ArcGIS Pro, Image Server,

Raster Analytics)

Demo: Supervised

Machine Learning for

Feature Extraction in

ArcGIS Pro

ImageryAccess

Imagery Prep

Creating Training

data

Inference

Distributed Processing

Feedback Loop

TakeAction

Feature Extraction and Machine Learning with ArcGIS: End to End Cycle

Training Derive Products

Generate Training

Samples

Imagery

Training ToolsTraining sites

ArcGIS – Machine Learning Workflow

ArcGIS Professional Image (Data) Scientist

Inferencing Tools

Inference results

Pixel & Segment Based

Training Engine

Deep Learning BasedDeep Learning

Machine Learning

Feedback Loop

MachineLearning

DeepLearning

Detailed Workflow

Model

Definition

ArcGIS User

Input Images

Machine Learning:- Support Vector Machine- Random ForestDeep Learning:- TensorFlow*- CNTK*- PyTorch*- Custom*- + External via Python

*Requires framework installed

Pixel & Segment Based:- Maximum Likelihood- Support Vector Machine- Random ForestDeep Learning Based:- TensorFlow*- CNTK*- PyTorch*- Custom*

*Run External to ArcGIS

End-to-end from raw imagery to structured information products

Labelling Data

PrepTrain/Fit

Model

Prediction AnalysisField

Mobility, Monitoring

Feature Extraction Workflow in ArcGIS

Image

Service/

Mosaic

Dataset

Imagery

Management

Deep Learning

Key imagery tasks for deep learning

Impervious Surface

Classification

Agricultural Crop

Detection

Building Footprint

Extraction

Damaged House

Classification

Pixel Classification Object Detection Instance Segmentation Image Classification

Where Deep Learning Excels

• Resources to create and maintain robust training datasets

• Access to GPU’s to train and apply model… ☺

• Well defined problem and general knowledge of an area

• Imagery collected under consistent conditions with minimal

variations in quality

• Large scale monitoring problems

- There is a need to repeatedly measure the activity, composition

and change in a particular area over the course of time

- Use cases

- Monitoring and identifying changes in landcover over time

- Object detection- i.e.. Counting specified objects

- Classification of detected objects – i.e.. Damaged or undamaged

houses

From Change Detection to Monitoring…

Deep Learning with Imagery in ArcGIS ArcGIS supports end-to-end deep learning workflows

• Tools for:

• Labeling training samples

• Preparing data to train models

• Training Models

• Running Inferencing

• Supports all 4 imagery deep learning categories

• Supports image space, leverage GPU

• Clients

• ArcGIS Pro

• Map Viewer

• NotebooksPart of ArcGIS Image Analyst

Run distributed on ArcGIS Image Server

• Managing Training Data and

Applying a Deep Learning Model

in ArcGIS Pro for Object

Detection and Monitoring

Demo: Deep

Learning

Conservation Innovation Center

The CIC was created by Chesapeake Conservancy to

help shape proactive responses for one of the world’s

largest environmental efforts—restoring the

Chesapeake Bay.

Since then, the CIC has continued to pioneer high-

resolution GIS mapping that provides new

perspectives about the state of landscapes and

waterways. This information is used to identify

specific project-level priorities that can maximize

conservation outcomes.

The Problem

• Poultry houses are important to Chesapeake

Bay TMDL

- mapping and accounting agricultural BMPs

• USDA data is restricted

- Unknown total number in Chesapeake Bay

Watershed

• USGS data is new but limited to Delmarva

• Very little available geospatial data on poultry

houses

• 5,747 poultry houses in the Delmarva peninsula

identified using 2016 and 2017 USDA NAIP by

USGS

• “Very little” available geospatial data on

poultry houses

What’s the plan?

• Utilize ArcGIS tools and existing datasets to training data(USGS poultry houses)

• Export Training Data For Deep Learning in ArcGIS Pro

• Detect Objects Using Computer Vision

• Run Detect Objects Using Deep Learning (ArcGIS Pro)

• Compare results

Data and Models

• Input layers: red, NIR, thermal, ndsm, NDVI

• Main model: computer vision

- Scale Invariant Feature Transform (SIFT)

- Gray Level Co-Occurrence Matrices (GLCM)

• Other considerations:

- Traditional machine learning models with features extracted manually

• Questions to test:

- How much benefit is there of using computer vision

- do traditional machine learning models perform comparable with extracted features?

Next steps

• Pass data to BMP team to integrate dataset into BMP analysis

• Accuracy assessment

• Use same methods to identify novel classes

Links and Contact

chesapeakeconservancy.org/conservation-innovation-center

sciencebase.gov/catalog/item/5e0a3fcde4b0b207aa0d794e

kmainali@chesapeakeconservancy.org

jczawlytko@chesapeakeconservancy.org

Questions?

You can find me at the Civilian – Sciences Area of the Expo hall

James Sill

jsill@esri.com

Thank You!!!

Presenter(s)

Demo Title