emerging technologies and methods in earth observation for

Emerging Technologies and Methods in Earth Observation for

Agricultural MonitoringFeb. 13, 2018

National Agricultural Library

USDA Foreign Agricultural ServiceBob Tetrault: Deputy Director, International

Production Assessment

State of the Practice, USDA-FAS Perspective 1

Agenda1) USDA Foreign Agricultural Service, Office

of Global Analysis, International Production Assessment Division (IPAD)

Introduction and decision-making process

2) IPAD’s use of Earth Observation products State of the practice and EO product transition

3) Products in a decision-support portfolio arranged by ARL and product type


USDA’s Economic Information System, Official U.S. government estimates of World Agricultural Production• 18 Commodities,166 Countries,• 1,209 Country-Crop Pairs (e.g. Australia-Wheat) • Crop Analysis:

Where is the crop grown? When is the crop harvested? How is the crop doing? How big is the crop?(area, yield, & production)

What does it mean to food

supply?What does it

mean to agricultural

prices?


USDA-FAS-IPAD uses a well-defined decision-making process

Foreign Agricultural ServiceOffice of Global Analysis

IPA Division

4

Analysis of data sources and

data products

Presentation and discussion of data products

In FY2017, FAS Office of Global Analysis reviewed the business process for USDA’s Interagency Commodity Estimates Committee (ICEC).

Source: MODIS NDVI 8-day & SPAM-IIASA 2005 Soybean Mask,NASA/GSFC/GIMMS, USDA/FAS/IPAD

Southern Rio Grande do Sul

Northern Rio Grande do Sul

Product Example: MODIS NDVI 8-day for Soybean-Producing Areas of Rio Grande do Sul, Brazil

Southern Rio Grande do Sul(2% of Brazil soybean

production)

• NDVI indicates crop growth is delayed and vegetation condition is below last year.

• Crops planted in Nov/Dec had poor germination/low, soil moisture. Development was slow and plants had low vigor (PR SEAB-DERAL). Rain at the end of the January was beneficial.

CLOUD-REDUCEDDATA SAMPLE

Foreign Agricultural ServiceOffice of Global AnalysisIPA Division

Northern Rio Grande do Sul(14% of Brazil soybean

production)

NDVI indicates crop growth is delayed and vegetation condition is below last year.

CLOUD-REDUCEDDATA SAMPLE

5

Earth Observation Products function when they are in context of other products.This example incorporates several support products including: crop statistics crop distribution maps croplands mask precipitation & soil moisture

Crop condition product (EO NDVI) includes several contextual elements including: NDVI for this year & last year minimum, maximum & mean

NDVI count of observations vs.

expected

Critical: Crop analyst’s input and experience


IPA Division

IPAD Products: World Agricultural Production--production briefs, Commodity Intelligence Reports, Lockup presentations to World Agricultural Outlook Board

6

Research to operations—from an operational user’s perspectiveFour issues to be managed during transition

1. Identify / implement funding source Transition from research funds to operational funds

2. Identify / implement IT systems responsibility Identify who is responsible for data generation,

ingest and visualization (RACI: Responsible, Accountable, Consulted, Informed)

3. Product review / Training Blend of scientific review of observations to product

latency to confidence in the product. 4. Data continuity (future satellites)


Table 1 Short Term

Horizon

NASA GIMMS MODIS NDVI

Soil Moisture Palmer Model SMOS @50km

G-REALM

GDA Yield Forecaster

(MODIS NDVI@250)

croplands mask

SSM/I Yield Forecaster

Budget FAS-IPAD FAS-IPAD FAS-IPAD & NASA-ROSES FAS-IPAD FAS-IPAD

RACI R: NASA-GIMMS A:

IPAD

R: NASA-GIMMSR: Inuteq/ASRC

A: IPAD

R: ESSICR: Inuteq/ASRC

A: IPAD

R: GDA A: IPAD

R: WeatherPredictR: Inuteq/ASRC

A: IPADIT System 2 external

websites; managed by

NASA-GIMMS

ftp managed by NASA-GIMMS;

ingest/visualization managed by Inuteq/ASRC

ftp managed by ESSIC/SGT;


External website; managed by GDA

ftp managed by Weather Predict;


ProductReview DONE DONE DONE DONE DONE

Training DONE To increase use NEEDS To increase use To increase use

DataContinuity

MODIS>>NPP/VIIRS

SMOS>>SMAP

JASON-3>>Sentinel 3a

MODIS>>NPP/VIIRS

SSMI>>tbd

ARL 9.0 8.5 8.0 7.5 7.0

Select USDA-FAS-IPAD Products, by ARL Level—Short Term Horizon


IPA Division

Transition to full integration and repeated use (ARL 9.0) requires identified budget, IT systems, and product review (including training)

8

Table 2 Medium Horizon

Soil Moisture Palmer Model SMAP @36km

NASA GIMMS (VIIRS

NDVI@350)

Precipitation(USAF 557WW

@10km)

Harmonized Sentinel-Landsat

(NDVI@30m)

Evaporative Stress Index

(ESI)

Budget FAS-IPAD FAS-IPAD& tbd

USAF 557 WW & FAS-IPAD NASA NASA-ROSES

RACI(notional)

R: NASA-GIMMSR: Inuteq/ASRC

A: IPAD

R: NASA-GIMMSA: IPAD

R: USAF 557 WW R: Inuteq/ASRC

A: IPADR: NASA-GSFC?

R: NASA-Marshall

R: Inuteq/ASRC A: IPAD

IT System(notional)

ftp managed by NASA-GIMMS;


2 external websites;

managed by NASA-GIMMS

ingest managed by Inuteq/ASRC; IPAD internal database

(CADRE) managed by Inuteq/ASRC

TBD TBD

Product(notional) NEEDS NEEDS NEEDS NEEDS NEEDS

Training NEEDS DONE DONE NEEDS NEEDS

DataContinuity

SMAP>>tbd

NPP/VIIRS>>JPSS/VIIRS

GPM>>tbd

L-8/ S-2a&b>>Landsat-9 (small

satellites?)tbd

ARL 6.0 5.0 5.0 4.0 4.0

Select USDA-FAS-IPAD Products, by ARL Level—Medium Horizon


IPA Division

Transition from prototype requires external budget, focus on IT systems, and product review (observations>>algorithms>>training)

9

• X-Axis: NASA’s ARL, 1.0 to 9.0• Y-Axis: Product typeCrop area and crop yield products (e.g. relative crop

yield forecasting, flooded area)Crop condition products (e.g. NDVI, soil moisture,

CHIRPS, ESI, HLS)Support products (e.g. crop distribution maps, crop

calendar, fieldwork data collection, CropSignal—crop stats, precipitation and temperature)

• Z-Axis: Impact (a qualitative assessment)Crop analysts’ will be surveyed about the products

and converted into a quantitative score. Impact score is from 100 to 1,200 with 1,200 indicating that the product has a strong impact on IPAD.

USDA-FAS-IPAD decision-support system product portfolio


IPA Division

10

USDA-FAS-IPAD decision-support system product portfolio

Fifteen products (right side) are currently “state of practice.”ARL on X-axis; Product type on Y-axis; Impact to program on Z-axis.


IPA Division

11

IPAD products—state of practice (15)Products in development (12)

NASA GIMMS MODIS NDVI (charts & maps)

GDA Yield Forecaster (MODIS NDVI@250)--

croplands mask


Precipitation maps and charts (USAF 557WW

@10km)

0.0 2.0 4.0 6.0 8.0 10.0Application Readiness Level (ARL)

Cro

p A

rea

&

Cro

p Yi

eld

Cro

p C

ondi

tion

Supp

ort

Anticipated products—from an operational user’s perspectiveSix products (ARL 1.0 to 3.0)

– GDA yield forecaster using VIIRS @350m– Flooded area estimation using Sentinel-1(baseline plus

change product)– Yield forecasting using Harmonized Landsat-Sentinel

@30m– Effective field edge boundaries for a global common land

unit, using Sentinel-2a&b @10m (machine learning)– Relative crop area estimation using Landsat-8 &

Sentinel-2a&b (machine learning)– Soil moisture—corrected 2-layer Palmer model using

SMAP @13km


USDA-FAS-IPAD Research Needs

1. Complete transition of research products to operations2. Understand product interactions


IPA Division

13

IPAD products—state of practice (15)Products in development (12)

NASA GIMMS MODIS NDVI (charts & maps)

GDA Yield Forecaster (MODIS NDVI@250)--

croplands mask


Precipitation maps and charts (USAF 557WW

@10km)

0.0 2.0 4.0 6.0 8.0 10.0Application Readiness Level (ARL)

Cro

p A

rea

&

Cro

p Yi

eld

Cro

p C

ondi

tion

Supp

ort

1. How does 10km precip. data interact with soil moisture?

2. How does ESI interact with NDVI?

3. How does NDVI @30m interact with crop models?

4. How does fieldwork interactwith soil moisture, CHIRPS and ESI?

5. How does effective field edge identification interactwith crop area estimation?

Emerging Technologies and Methods in Earth Observation for

Agricultural MonitoringFeb. 15, 2018

National Agricultural Library

USDA Foreign Agricultural ServiceBob Tetrault: Deputy Director, International

Production Assessment

Research to Operations, USDA-FAS Perspective

15

Acknowledge the different cultures, different communication, and different time scales for researchers vs. operational users

– Long-term strategic horizon: for developments that may need years of work, and might need substantial resources. (early stage research ARL 1.0 to ARL 3.0)

– Medium-term horizon: for development work over a 1-4 year time scale. (typically ARL 4.0 to ARL 7.0)

– Short-term horizon: for smaller developments or concludes long- and/or medium-term work; responsive to user feedback; responsive to changes in the computing environment (security upgrades), and/or changes in data products. (typically ARL 7.0 to ARL 9.0)


Adapted from: The ECMWF research to operations (R2O) process, Buizza et al. 2017

16

Research OperationsGoals Explore untested technologies;

new knowledge; uncertain or distant future application

Robust technologies; continuity of previous data source and methods;

value in practicalityDrivers Quest for new results; publish

or perish; new grantsRepeatable results; system

securityFunding Supports entirely new topics Supports existing productsCosts Research personnel Software and system maintenanceIT System Code standards and

documentation secondary importance;

Code standards essential; documentation essential

Products Research papers; no fixed schedule

Products to support customers; Routine and rigid delivery schedule

User Community

Narrow; highly trained Broad based, often untrained; decision-making process with

multiple users

Differences between researchers and operations


IPA Division

Researchers and operations have different goals, drivers, measurements of success, and end users.

17

Adapted from An Emerging Protocol for Research-to-Operations (R2O) at CPC, Lenic et al. 2011

Issue Research Satellite

Operational Satellite

MODIS example

Data continuity Rarely replaced, no spare satellite

on orbit

Plans for high-priority sensors; spare satellite

Transition to VIIRS

Data format and collection

change over mission’s life

No change MODIS now on Collection 6

Data latency Hours to months Hours or less NRT within 24 hours; NRT to STD 82 hours

Impact of reduced data quality

Reduced science Lower value products

Will VIIRS products be reduced data quality

compared to MODIS?

Time value of data High long-term value

High immediate value

Aqua MYD NRT 7/3/2017 failure; produced on 7/10/2017, but ICEC

meetings finished. Terraproduct on time. (minimal

impact to lockup)

Key Differences between research satellites and operational satellites


IPA Division

Operational agricultural monitoring uses research satellites but need operational products.

18

Adapted from: Satellite Observations of the Earth’s Environment: Accelerating the transition of research to operations (2003) Anthes et al. National Research Council

Research to operations—from an operational user’s perspectiveFour issues to be managed during transition

1. Identify / implement funding source Transition from research funds to operational funds

2. Identify / implement IT systems responsibility Identify who is responsible for data generation,

ingest and visualization (RACI: Responsible, Accountable, Consulted, Informed)

3. Product review / Training Blend of scientific review of observations to product

latency to confidence in the product. 4. Data continuity (future satellites)


ID Product QA Process Steps Prime Second Tertiary

1 Does the algorithm produce the product? Research organization

COR

2 Does the observation or measurements correspond to in situ measurements?

Research organization

COR

3 Can the IT system reliably produce the product in a timely fashion? (product latency)


On-site contractor

COR

4 Has the products’ generation, ingest, storage and visualization been assigned RACI?

COR Research organization

On-site contractor

5 Does the new product compare well with a previously used product?


On-site contractor

6 Do the crop analysts understand the science and the algorithm that underlie the product?


Crop Analysts

7 Are the crop analysts trained? Can they adequately explain the product to their end users?

COR Crop Analysts Research organization

8 Do the geospatial products adequately represent what is found on the ground?

Crop Analysts COR Research organization

9 Do the crop analysts have confidence in the product? Does the product compare fit within convergence of evidence?


10 Is the visualization of the product suitable for the crop analysts? (e.g. units of measurement, legend, color scheme, font size, line width, etc.)

Crop Analysts COR & on-site contractors


11 How are the crop analysts using the product? And how much? COR IPAD management

Crop Analysts

12 What prevents the crop analysts from using the product? COR IPAD management

Crop Analysts

Product Quality Assurance—Complicated Hand-off 20


IPA Division

ID Product QA Process Steps Prime Second Tertiary

7 Are the crop analysts trained? Can they adequately explain the product to their end users?

COR Crop Analysts Research organization

8 Do the geospatial products adequately represent what is found on the ground?


9 Do the crop analysts have confidence in the product? Does the product compare fit within convergence of evidence?


10 Is the visualization of the product suitable for the crop analysts? (e.g. units of measurement, legend, color scheme, font size, line width, etc.)

Crop Analysts COR & on-site contractors


Product Quality Assurance—Complicated Hand-off

05

1015202530

11/3/2014

11/17/2014

12/1/2014

12/15/2014

12/29/2014

1/12/2015

1/26/2015

2/9/2015

2/23/2015

3/9/2015

Surface Soil Moisture 536-193 (Fazenda Oilema)

PM-SMOS_surface PM_surface

1. How can we better understand the EO products in the field?

2. Will this improve the confidence in the EO products? Foreign Agricultural Service

Office of Global AnalysisIPA Division

at Fazenda Oilema, Feb. 2015

21

Research to operations—from an operational user’s perspective


Summary ARL 4.0 to 7.0 ARL 7.0 to 9.0Time scale Medium horizon Short termBudget Mixed sources Operational fundingRACI & IT System

Notional Operational & documented

Product Notional to developed Developed & documentedTraining Focused on understanding

science and algorithmsFocused on increasing use

Data Continuity

Notional Identifiable

“Well trained people are, therefore, one of the most important components of the remote sensing technology transfer process” (NRC, 2001b).

22

• Extra slides

IPAD(on-site

contractor)

ESA/VITO

Proba-VNASA

GIMMS (NDVI)

USGS (Landsat)

NASA MODIS Daily

Deimos (DMC)

UCSB (CHIRPS)

USAF 557th

Weather Wing

WMO

NOAA (CMORPH)

NASA (GPM & TRMM)

NASA/ ARS

(SMOS &

SMAP)

ESSIC (G-

REALM)

Pacific Disaster Center

GDA

Weather Predict (SSM/I)

USGS/ FEWS (Eta)

Storm Data (wind

speeds)

USDA-FAS-IPAD Earth Observation Data Sources

Precipitation

Satellite Imagery

IPAD ingests data or accesses websites from more than 17 data sources.

1. Data Generation (partner)

2. Data Ingest (on-site contractor)

3. Product Generation (partner or on-site contractor)

4. Visualization(partner or on-site contractor)

24

emerging technologies and methods in earth observation for

Documents