john roberts, dr yaqing gou - copernicus · 3 commands for a change map $ extract_signatures...

24
CLCR PYEO John Roberts, Dr Yaqing Gou

Upload: others

Post on 15-Dec-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

C L C R

PYEOJohn Roberts, Dr Yaqing Gou

Page 2: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

Change Detection map

Page 3: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

Change Detection map

Page 4: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

Change Detection map

Page 5: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

3 commands for a change map

$ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv

$ create_model_from_signatures indonesia_sigs.csv indonesia_basic.pkl

$ simple_classification indonesia.tif indonesia_basic.pkl indonesia_classes.tif

Page 6: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

...and some other work

• Images needed to be found

• Images had to be downloaded

• Images had to be preprocessed

• Images had to be stacked for change detection

• Training data had to be defined and collected

• Map had to be made nice in QGIS

Page 7: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

...and some other work

• Images needed to be found

• Images had to be downloaded

• Images had to be preprocessed

• Images had to be stacked for change detection

• Training data had to be defined and collected

• Map had to be made nice in QGIS Involves value judgements; needs humans

Does not need humans!

Page 8: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

PYthon for Earth Observation (PYEO)• A set of building blocks for making

automatic GIS processing chains• Download to classification

• Goal: GIS without HPC• Many GIS profressionals are NOT

programmers

•3 ways of using• Pre-made processing chains

• Single step scripts

• As a Python library

Page 9: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

Technical bits• Hosted on Github

• github.com/clcr/pyeo• Managed using Conda

• Built on the following core Python packages

• Numpy

• Scipy

• Scikit-learn

• GDAL

• Joblib

• And two Sentinel libraries

• sentinelhub

• sentinelsat

Page 10: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

Pyeo structure• pyeo/

• Contains the core libraries

• apps/

• Processing chains and example scripts

• Built out of functions in pyeo/

• tests

• Test suite

• Contains examples of using individual functions

• docs

• Documentation for functions

Page 11: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

Main libraries

raster_manipulation.py

• Creating rasters

• Stacking/clipping/managing rasters

• Creating base composites

• Preprocessing

• Cloud masking (fmask and sen2cor)

queries_and_downloads.py

• Querying data sources

• Filtering queries

• Downloading data

classification.py

• Extracting training data

• Creating models

• Applying pixel-wise models to rasters

– (segmentation to come)

filesystem_utilities.py

• Creating folders

• Logging

• Managing .SAFE files

• Querying metadata

Page 12: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

Prototype libraries

validation.py

• Stratified sampling

• P-value calculation

• Implementation of Olafson etal

terrain_correction.py

• Applying

• Filtering queries

• Downloading data

your_library.py

Bring your work to the world?

Page 13: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

Example complex data flow

Page 14: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

Example application: Forest Sentinel

Page 16: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

Sentinel-2 NRT change proc-chain

Auto data filtering/

downloadlatest

imagery

Preprocessing, sen2cor/ 6S

BOA correction

Scene classification,

Cloud masking

Repository of scene training

data

Sentinel 2 data

hub

Cloud free base image

Most recentS2

acquisition

Change detection

(RF)

Imagestack

Repository of change

training data

Thematic map of land

cover change (1)

Probability map of forest cover change

(2) Kmz alert file &Json

database (3)

Data download & atmospheric processing Data preparation for deforestation detectionDeforestation detection & output production

1 2

3

Page 17: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures
Page 18: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures
Page 19: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures
Page 20: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

DEFORESTATION ALERT APP

Page 21: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

DEFORESTATION ALERT APP

Page 22: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

Future

Page 23: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

Future

Page 24: John Roberts, Dr Yaqing Gou - Copernicus · 3 commands for a change map $ extract_signatures indonesia.tif indonesia_training.shp indonesia_sigs.csv $ create_model_from_signatures

Thankyou for listening

Any questions?