goes-r field campaign workshop, apr 8-9, 2015 goes-r algorithm working group (awg) planning and...

GOES-R Field Campaign Workshop, Apr 8-9, 2015

GOES-R Algorithm Working Group (AWG) GOES-R Algorithm Working Group (AWG) Planning and Participation in GOES-R Field Planning and Participation in GOES-R Field

Campaign ActivitiesCampaign Activities

Jaime DanielsJaime Daniels

Center for Satellite Applications and Center for Satellite Applications and

ResearchResearchNational Environmental Satellite Data and Information Surface

National Oceanic and Atmospheric Administration

1

Contributions from AWG Team Leads and Team Members


OutlineOutline• The Algorithm Working Group (AWG): A Brief

Introduction

• AWG Level-2 Product Validation Methods, Reference Data, and Activities

• AWG Team Participation in Targeted Field Campaigns

2



Introduction



3


GOES-R Algorithm Working GroupGOES-R Algorithm Working Group

• End-to-End Capabilities– Instrument Trade Studies

– Proxy Dataset Development

– Algorithm Development and Testing

– Product Demonstration Systems

– Development of Cal/Val Tools

– Integrated Cal/Val Enterprise System

– Radiance and Product Validation

– Algorithm and application improvements

– User Readiness and Education

4


STAR GOES-R AWG PM

Jaime DanielsManagement Support

Tess Valenzuela (Budget/Schedule)

Algorithm Integration

Walter Wolf

Algorithm Integration

Walter Wolf

Proxy Fuhzong Weng

Proxy Fuhzong Weng

Radiation Budget Istvan Laszlo

Radiation Budget Istvan Laszlo

Land Bob YuLand

Bob Yu

CryosphereJeff Key

CryosphereJeff Key

Ocean DynamicsEileen Maturi

Ocean DynamicsEileen Maturi

SSTAlexander Ignatov

SSTAlexander Ignatov

LightningSteve Goodman

LightningSteve Goodman

Hydrology Bob Kuligowski

Hydrology Bob KuligowskiAerosols

Shobha KondraguntaAerosols

Shobha Kondragunta

Winds Jaime Daniels

Winds Jaime Daniels

Imagery Tim SchmitImagery

Tim Schmit

SoundingsTim SchmitSoundingsTim Schmit

AviationKen Pryor

Mike Pavolonis

AviationKen Pryor

Mike Pavolonis

CloudsAndy Heidinger

CloudsAndy Heidinger

Cal/Val (Sensor) Fred Wu

Cal/Val (Sensor) Fred Wu

Algorithm Working GroupAlgorithm Working Group

Team leads are STAR Scientists

5


GOES-R ProductsGOES-R ProductsAdvanced Baseline Imager (ABI)

Aerosol Detection (Including Smoke and Dust)Aerosol Optical Depth (AOD)Clear Sky MasksCloud and Moisture ImageryCloud Optical DepthCloud Particle Size DistributionCloud Top HeightCloud Top PhaseCloud Top PressureCloud Top TemperatureDerived Motion WindsDerived Stability IndicesDownward Shortwave Radiation: SurfaceFire/Hot Spot CharacterizationHurricane Intensity EstimationLand Surface Temperature (Skin)Legacy Vertical Moisture ProfileLegacy Vertical Temperature ProfileRadiancesRainfall Rate/QPEReflected Shortwave Radiation: TOASea Surface Temperature (Skin)Snow CoverTotal Precipitable WaterVolcanic Ash: Detection and Height

Geostationary Lightning Mapper (GLM)

Lightning Detection: Events, Groups & Flashes

Space Environment In-Situ Suite (SEISS)

Energetic Heavy Ions

Magnetospheric Electrons & Protons: Low Energy

Magnetospheric Electrons: Med & High Energy

Magnetospheric Protons: Med & High Energy

Solar and Galactic Protons

Magnetometer (MAG)

Geomagnetic Field

Extreme Ultraviolet and X-ray Irradiance Suite (EXIS)

Solar Flux: EUVSolar Flux: X-ray Irradiance

Solar Ultraviolet Imager (SUVI)

Solar EUV Imagery

Baseline ProductsBaseline Products

Level-2 products outlined in red

6

GOES-R Field Campaign Workshop, Apr 8-9, 2015 7http://www.goes-r.gov/products/baseline.html7


OutlineOutline

• The Algorithm Working Group (AWG): A Brief Introduction



8


Overarching Product Science Overarching Product Science Validation MethodsValidation Methods

• Product inspection Visualization of product and/or any output fields in the product, intermediate, and/or

diagnostic files using in-house software tools Provides a quick qualitative assessment of product performance

• Comparison to reference/correlative/ground truth data Collocate product and applicable reference/correlative/ground truth datasets and

compute quantitative statistics (accuracy, precision, etc) Visualization of product and/or any output fields in the product, intermediate, and/or

diagnostic files together with reference/correlative/ground truth data using in-house software tools

Provides a quantitative assessment of product performance. Focuses on assessing and characterizing product quality (ie., accuracy and precision) that needs to be conveyed to the user community.

9


Overarching Product Science Overarching Product Science Validation ProcessValidation Process

Diverse earth (atmospheric & surface) and space weather conditions are represented over space, time and measurement range.

Compute estimates of on-orbit product performance

Characterize errors

Goal: Determine if product meets success criteria

Product Generation

Validate (Inspect and/or Compare with Reference/ Ground Truth)

Algorithm

Approach for validating performance of products derived from GOES-R data and Approach for validating performance of products derived from GOES-R data and correlative data sources…correlative data sources…

Validation Tools

10


Reference/”Ground Truth” Reference/”Ground Truth” Data SourcesData Sources

Aeronet Stations

Aerosol Optical Depth

Radiosondes

Winds,Temperature, Moisture, Stability

CALIPSO, CLOUDSAT Clouds, Icing

NWP Analyses Winds, Temperature, Moisture

Bouys, Ships

SST

SURFRAD, ARM LST, Radiation

Rain Guages Precipitation

Sfc Snow Reports, NESDIS IMS

Snow

National Lightning Detection Network (NLDN)

Lightning

Pilot, aircraft Reports Icing,Turbulence, winds

Ground-based Ozone Ozone

Validation teams use a wide variety of Reference/“Ground Truth” datasets to Validation teams use a wide variety of Reference/“Ground Truth” datasets to assess and validate product performance.assess and validate product performance.

GOES EPS and HEPADProtons, Alpha Particles and Electrons

NASA SDO Solar Imagery

11


• Types – Primarily a mix of in-house developed software & COTS software– In-house developed software includes: visualization, collocation, and

analysis tools– Types of COTS software include: McIDAS-X, McIDAS-V, IDL, GrADS

• Current State– Mature. – AWG teams have been working to develop these tools since 2006.– Nearly all tools are in a working state with updated versions being

developed.

Status of AWG L2 Product Status of AWG L2 Product Validation ToolsValidation Tools

12


Product Validation ToolsProduct Validation ToolsAWG teams have developed a cadre of product validation tools as part

of their ongoing product validation activities

Significance: The tools will enable the routine monitoring of L2 product performance and for “Deep-dive” assessments and analysis of products to resolve any issues/anomalies that may arise

Clouds

LST

Aerosols

13



Introduction



14


• AWG product application teams asked to identify where gap filling measurements would enable more complete validation for L2 products they are responsible for

• The Unmanned Aircraft Systems (UAS) capability has had strong support from a number of the AWG product teams (Radiation Budget, Land, Aerosol)

• AWG has also provided critical support since early discussions with the NOAA UAS team through numerous TIMs and support in the development of a draft GOES-R UAS Near Surface Science Requirements document as well as an attendant CONOPS plan

AWG Team Participation in AWG Team Participation in Targeted Field CampaignsTargeted Field Campaigns

15


AWG Team Participation in AWG Team Participation in Targeted Field CampaignsTargeted Field Campaigns

• Land Team (Bob Yu)– Land surface temperature– Fire

• Radiation Budget Team (Istvan Laszlo)– Shortwave radiation

• Cloud Team (Andy Heidinger)– Cloud-top type, cloud-top height,

cloud optical depth, cloud particle size, liquid water path, ice water path

• Aerosols/Air Quality/Atmospheric Chemistry Team (Shobha Kondragunta)– Aerosol optical depth (AOD), but

more importantly, PM2.5 concentrations derived from retrieved AOD

16

• Lightning Team (Steve Goodman)

• Lightning Detection

In collaboration with external partners…


Land Surface Temperature

Slides courtesy of Bob Yu

17

GOES-R Field Campaign Workshop, Apr 8-9, 2015 18

• Components of LST Validation In-situ measurement comparisons and analyses Cross-satellite comparisons and analyses Successful applications– users promotion

• Strategy of In-situ measurement comparisons and analyses Existing ground station observations (e.g. SURFRAD Network), serve as long-term validation data source

Field campaign data plays three important roles

High quality observations for direct comparison and analysis

Calibrating co-site ground station observations

Characterizing the heterogeneity of the ground station site

Towards the field campaign readiness

Platform: Low altitude, small Unmanned Aircraft Systems (UAS)

Instrument readiness: Accurate infrared radiometers cover ABI bands

Site selection: Better to cover SURFRAD/CRN station

Data processing and algorithms: noise filtering, spatial

characterization, calibration to station data, etc.

Coordination with the Field Campaign Team

Towards Field Campaign Towards Field Campaign for LST Validationfor LST Validation

Down-looking PIR at 8 meter height from the ground

UP-looking PIR

Diffuse Radiometer

Down-looking PIR on the towerAt 8-m from ground

Thermometer

Anemometer


Fire

Slides courtesy of Ivan Csiszar

19


• In general, for proper validation we need the mapping of the thermal conditions at high resolution within the entire pixel footprint. This needs to be done with sensors that have the proper bands (ideally ~ 4 microns, but there is some flexibility here), and simultaneously with the satellite overpass due to the very dynamic nature of fires.

• These data help determine detection probabilities as a function of sub-pixel fire characteristics. As a minimum, statistically we can determine the probability of detection as a function of sub-pixel fire size or fraction. If we can also measure unsaturated thermal radiance we can also do this in a two-dimensional space - detection probability as a function of fire size and temperature (or intensity).

• Getting these data is very hard and opportunistic. Realistically, we cannot sample the entire size/temperature space, but rather, use such reference data as anchor points to confirm simulated performance statistics.

Benefits to Fire Product Algorithm

VIIRS 375m Fire data

20


Shortwave Radiation

Slides courtesy of Istvan Laszlo

21


Benefits to AOD and SRB Algorithms

• Better characterization of surface and atmospheric state– Can test/diagnose algorithm* under

optimal conditions (many normally remote-sensed, modeled or climatological parameters are “measured”)

– Can get information on spatial variability of surface and atmosphere conditions, and on L2 parameters on a scale approaching/comparable to that of an ABI pixel

• Measurements* of AOD and SRB at locations (e.g., open ocean) where reference data are not readily available.*NOTE: data are limited in space and time; it is understood that it will not provide true measure of algorithm performance and product quality

Su

rface?

Cloud?

Aero

sol?

Gas?

SURFRAD+ sites

22


Plan for Using Field-Campaign Data Plan for Using Field-Campaign Data

• Use field-campaign AOD and DSR to evaluate ABI retrievals of like quantities

• Use field-campaign AOD, surface reflectance (directional and spectral), water vapor profile, sky condition (cloudiness), etc., to identify sources of errors.– Retrievals will be performed by replacing derived,

modeled or climatological data with field-campaign data – one at a time, and

– Results will be compared to those from “normal” retrievals.

23


AOD ExampleAOD Example

• UAV measurements can be used to check components of the ocean model values

• UAV measures sum of Terms 1 and 2 (approximately).

• “Integration” of directional UAV measurements gives the sum of all four Terms.

• Dividing the “integrated” value by the separately measured downward irradiance gives the albedo needed in AOD retrieval over land (and in SRB retrieval).

Term 2

diffuse indirect out

•Ocean surface reflectance in ABI AOD algorithm is the sum of

1. water-leaving reflectance2. whitecap reflectance3. bi-directional reflectance

•Bi-directional reflectance is modeled as the sum of 4 components.

Term 1

direct indirect out

Term 3

direct indiffuse out

Term 4

diffuse indiffuse out

+++

24


SRB ExampleSRB Example

• Large negative downward shortwave radiation (DSR) bias at Cape Cod during an ARM field campaign in 2012.

• Deep-dive analysis: On average, retrieved AOD > ground-measured AOD; retrieved spectral reflectance albedo < ground-albedo derived from spectral values => retrieved DSR < ground-measured DSR.

DSR is severely underestimated on this easy, dominantly clear-sky day.Why? Too small surface albedo? Too large AOD?

25


UAV Measurements Quantify Spatial UAV Measurements Quantify Spatial Representativeness of Station DataRepresentativeness of Station Data

• Illustration of problem

• UAV data can help

• Satellite “sees” larger area than downward looking instrument does.

• Upward looking instrument “sees” radiation from an entire hemisphere.

• This represents inconsistency unless atmosphere and surface are homogeneous (uniform)

• Deployment of UAV at several different locations within the satellite footprint can characterize degree of uniformity within footprint.

• Ideally, this should be done for all reference (e.g. SURFRAD) sites in different seasons.

Satellite footprint on ground

Box size of satellite footprint

station

UAV

26


Cloud Products

Slides courtesy of Andy Heidinger

27


Use of LIDAR for Cloud ValidationUse of LIDAR for Cloud Validation

Airborne lidars provide higher spatial resolution data than spaceborne.

Our CALIPSO CALIOP tools have been applied to ground-based lidar during DC3

Provides almost ideal validation source of cloud vertical profiles when viewed from above.

Depolarization also provide cloud-top phase.

Multi-layer detection component of cloud type and height algorithms can also be validated.

Example Comparison of CALIPSO and ACHA for one AQUA/MODIS Granule .

28


Use of Surface Based Radiometers for Daytime Cloud Optical and Microphysical

Properties Validation (DCOMP)

• The Solar Spectral Flux Radiometer (SSFR) is a shortwave spectrometer operated by U. Colorado / LASP.

• During CALNEX 2010 it was operated on a ship looking up.

• It provides retrievals of DCOMP cloud properties using radiation travelling through the cloud (not reflected off the top like DCOMP).

• This provides a more independent validation and can be accomplished with any upward looking well-calibrated radiometers.

• AVIRIS also is useful for this, if available.

Red lines are the DCOMP Error Bars

Liquid Water Path (lwp) is an option 2 DCOMP product

29


Aerosols

Slides courtesy of Shohba Kondragunta and Xinrong Ren (UMD)

30


Benefits to Aerosol Algorithms

• The field campaign focus is NOT to validate Aerosol Optical Depth (AOD) as this can be done sufficiently with AERONET observations.

• Rather, the field campaign focus is to evaluate surface PM2.5 derived from AOD

• AWG Aerosol Team has partnered with the University of Maryland (UMD) to do the flights and get aerosol vertical profile and Single Scattering Albedo (SSA) information.

31


Evaluating Surface PM2.5 Derived from Evaluating Surface PM2.5 Derived from Retrieved Aerosol Optical DepthRetrieved Aerosol Optical Depth

32


UMD Research Aircraft Capabilities for UMD Research Aircraft Capabilities for GOES-R ValidationGOES-R Validation

Results from the summer 2013 flights over the Eastern Shore

0 0.1 0.2 0.3 0.40.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Aircraft integrated AOD

AO

D V

IIR

S

y = 1.07x + 0.129 r2 = 0.61

0 0.05 0.1 0.15 0.2 0.25 0.3 0.350

5

10

15

20

25

30

35

Aircraft Integrated AOD

[PM

2.5]

(g

m-3

)

y = 118.4x + 1.05 r2 = 0.87

0 0.1 0.2 0.3 0.4 0.50

5

10

15

20

25

30

VIIRS AOD

[PM

2.5]

(g

m-3

)

y = 69.0x - 5.00r2 = 0.57

GPS Position (Lat, Long, Altitude)

Met (T, RH, P, wind speed/direction)

Trace gases:O3: UV Absorption, modified TECOSO2: Pulsed Fluorescence, modified TECO

CH4/CO2/CO: Cavity Ring Down, PicarroNO2: Cavity Ring Down, Los Gatos

Aerosol Optical Properties:Scattering: bscat (@450, 550, 700 nm),

Nephelometer

Absorption: bap (565 nm), PSAPBlack Carbon: Aethalometer (7 wavelengths)

Data Acquisition: 1 sec

Aerosol Inlet

Gas Inlet

Met Sensors

(see the poster Ren et al.)

33


-78 -77.5 -77 -76.5 -76 -75.537.8

38

38.2

38.4

38.6

38.8

39

39.2

39.4

39.6

Longitude ( )

La

titu

de

( )

Extinction (Mm-1)

0

10

20

30

40

50

60

Extinction = Absorption + Scattering and can be measured at 3 wavelengths

0 10 20 30 40 500

1000

2000

3000

4000

5000

6000

7000

Ext (Mm-1)

Altitu

de

(ft)

0 10 20 30 40 500

1000

2000

3000

4000

5000

6000

7000

Ext (Mm-1)A

ltitu

de

(ft)

0 10 20 30 40 500

1000

2000

3000

4000

5000

6000

7000

Ext (Mm-1)

Altitu

de

(ft)

Extinction measured during 3 spirals

Results from a FLAGG-MD Winter 2015 flight on 02/20/2015

UMD Research Aircraft for Measuring UMD Research Aircraft for Measuring Urban EmissionsUrban Emissions

34


GLM Lightning Product

Slide information courtesy of Steve Goodman

35


Benefits to GLM Lightning Detection Algorithm

• Validation of GLM flash detection efficiency: collect coincident and collocated high altitude data over thunderstorms with the Fly’s Eye GLM Simulator (FEGS*):

Minimum Collection Set: Over-flights of thunderstorms

over well characterized total lightning super sites. Emphasis on large scale convection such as Mesoscale Convective Systems (MCSs) from pre-storm through entire evolution (include all times of day & other storm types)

Secondary Collection Set: Over-flights of thunderstorms at other locations (day / dawn or dusk/ night; high / low latitudes; land / ocean; various storm types / regimes)

• Validation of GLM flash location & time-stamp accuracy, and INR

• Validation of optical energy calibration for the GLM product (lightning, which consists of events, groups, and flashes)

SURFRAD+ sites

36


SummarySummary• AWG team participation identified along with their associated Level-2

products whose validation can benefit from GOES-R Field Campaign measurements

• AWG teams will perform data analysis at their respective local compute facilities– Leverage many of the validation tools already developed as part of AWG program– Product reprocessing capability exists and can be exercised

• AWG teams involved in this will perform data analysis and document their analysis results (Expect some extra cost beyond current funding)

• For Discussion– Who/where/how will data archival and management be done?

37


BACKUP


Sensor characterization - Radiometric calibration

- Geolocation/navigation

Post-Launch Tests (PLT) and engineering tests (compliance)

Established sensor stability QC/QA processes in place

Proxy data generation Calibration Processing;

Analysis of L1b products

Sensor characterization - Radiometric calibration

- Geolocation/navigation

Continuous assessment & monitoring, trend analysis of product quality

Algorithm assessment and verification

Quick look analysis of L2 products; comparisons to NWP model/analyses

Finalize L2 algorithm tuning and testing; Establish L2 product stability

Algorithm improvements

Determination of validation strategies, including identification and acquisition of “ground-truth”/reference datasets

Work to establish sensor stability;

Work to establish L1b and L2 product stability;

L1b and L2 algorithm testing and tuning

L1b/L2 product validation processes in place;

L1b and L2 product validations

Full and continuous data release to the user community

Cal/Val tool development Establish routine validation processes

Increasing data release to the user community

Cal/Val tool improvements

Development of L1b & L2 Cal/Val Plans

Data released to users, but data is understood to be non-operational

Cal/Val tool improvements Data Archival

Data Archival Data Archival

Early Orbit Check-out

Long Term Monitoring & OperationsIntensive Cal/Val Pre-launch Cal/Val

Pre-Launch Post-LaunchLaunch

Launch

Timeline ~6 months

Calibration/ValidationCalibration/Validation PhasesPhases

39

goes-r field campaign workshop, apr 8-9, 2015 goes-r algorithm working group (awg) planning and...

Documents

r algorithm

r field campaign workshop

goesr fps requirements

goesr ground segment

r awg pm jaime daniels

product validation algorithm

group awg planning

brief introduction awg