1

GOES-R AWG Aviation Team: Convective Initiation

June 14, 2011

Presented By: John R. Walker University of Alabama in Huntsville

In Close Collaboration With: John R. Mecikalski, Christopher Jewett, and Lori Schultz

2

Outline

Executive Summary (1 slide) Algorithm Description (4 slides) ADEB and IPR Response Summary (5 slides) Requirements Specification Evolution (1 slide) Validation Strategy (6 slides) Validation Results (3 slides) Summary (1 slide)

3

Executive Summary

The ABI Convective Initiation (CI) algorithm generates one Option 2 product.

Software Version 5 was just delivered in June. The ATBD (100%) is on track to be delivered in late June 2011.

Algorithm performs spectral and temporal differencing tests using several ABI IR channels to monitor cloud growth properties.

Validation Datasets: Use of the MSG 5-minute temporal resolution dataset is used as proxy along with radar data from within the MSG domain when available. Additionally, model simulated ABI brightness temperatures and model-derived radar reflectivities were used for validation datasets. Finally, the current proxy version of the algorithm has been used with GOES data and WSR-88D radar for validation.

Validation studies indicate spec compliance for the CI product.

4

Algorithm Description

5

Convective Initiation Algorithm Description

INPUT for 2 consecutive image times: INPUT for 2 consecutive image times: -ABI Channels 8,10,11,14,15,16-ABI Channels 8,10,11,14,15,16-AWG Cloud Type product-AWG Cloud Type product

(Temporal overlap method)(Temporal overlap method)

Define “Cloud Objects”Define “Cloud Objects”

Track “Cloud Objects”Track “Cloud Objects”

Perform Spectral/Temporal Perform Spectral/Temporal Differencing Tests on Differencing Tests on Tracked Cloud ObjectsTracked Cloud Objects

Determine Null or Positive Determine Null or Positive Forecast of CI for each Forecast of CI for each Tracked Cloud ObjectTracked Cloud Object

(12 tests to monitor cloud top properties …(12 tests to monitor cloud top properties …height, rate of growth, etc..)height, rate of growth, etc..)

7 of 12 tests must be “passed” in order to 7 of 12 tests must be “passed” in order to produce a “Positive” CI forecast. Then, produce a “Positive” CI forecast. Then, CI CI forecasts are OUTPUT.forecasts are OUTPUT.

11

22

33

44

“Cloud Objects” are defined from the Cloud Typing algorithm output from water cloud, assuming these clouds are immature and pre-convective.

Then, tracking is accomplished by using an overlap detection scheme between two times:» Take all defined objects from two successive times and search for temporal

overlap between the objects. Then, assign each overlap and contributing objects a unique Integer ID number. For both ‘Time 1’ and ‘Time 2’, each object with the same overlap will have the same ID number for tracking.

6

Time 1 Time 2 Time 1 + Time 2

Yellow regions are overlap regions

Yellow regions are overlap regions

Convective Initiation Algorithm Description

Channel brightness temperatures are used to perform spectral analyses on updraft regions of the storm that represent growing cumulus.

Use the coldest 25% of 11 μm channel data pixels associated with each object (potential updraft region) and average the brightness temperature of those pixels for all input IR channels to come away with a representative brightness temperature for each spectral channel for each object.

Compute spectral and temporal differencing for each object using the representative average brightness temperatures.

7

Convective Initiation Algorithm Description

Spectral and Temporal Differencing critical values were developed using the information from Mecikalski and Bedka (2006) on current GOES and MSG interest fields by Mecikalski et al. (2010) and further tested by Siewert et al. (2010).

7 out of 12 critical values must be met for a cloud object to be flagged with a “Positive” CI forecast.

8

Siewert et al. (2009)

Interest Field Physical Basis (Mecikalski et al.. 2009)

Critical Value

6.2-10.8 μm Cloud Depth -35oC to -10oC

6.2-7.3 μm Cloud Depth -25oC to 3oC

10.8 μm Cloud Depth/Glaciation

-20oC to 0oC

8.7-10.8 μm Glaciation -10oC to 0oC

Tri-channel Diff Glaciation -10oC to 0oC

15 min Tri-Channel Glaciation Trend >0oC

15 min 12.0-10.8 μm Cloud Depth >0oC

12.0-10.8 μm Cloud Depth -3oC to 0oC

15 min 10.8 μm Cloud Growth < -4oC

15 min 6.2-7.3 μm Cloud Depth Trend >0oC

15 min 6.2-10.8 μm Cloud Depth Trend >3oC

13.4-10.8 μm Cloud Depth -25o to -5oC

Convective Initiation Algorithm Description

9

ADEB Report Response Summary

10

ADEB Report Response Summary

ADEB Report Comment:“The team needs to demonstrate that it meets spec by further validation. In particular, validation outside the area of radar coverage needs to be conducted under a full range of convective cases, including, for example, convection under cirrus. The board specifically recommends using man-in-the-loop analysis.”

Response:We completely agree that more validation needs to be performed on the product output in order to make sure it meets the required specifications under a variety of convective scenarios. We have already added a great deal of validation cases toward meeting this goal by conducting an extensive evaluation/validation of …

11

ADEB Report Response Summary

Response (continued):… the proxy (current GOES) version of this algorithm, using National Weather Service WSR-88D radar data for validation. Furthermore, we have also finished validation of the latest RAMS model-simulated data made available from CIRA. Together, these efforts have added well over 14,000 new validated CI forecasts to our growing list of statistics, which has produced statistical accuracies well above the required 70% specification.Regarding the issue of seeking validation under cirrus-contaminated scenes, the algorithm is not designed to perform processing in these conditions. The spectral input for the algorithm is based on the AWG Cloud Type algorithm output, and we select from that data only the parts of the domain filled with warm water and mixed phase clouds. Because of the …

12

ADEB Report Response Summary

Response (continued):… thin and varying optical depths of most cirrus clouds, the satellite-retrieved spectral signatures of underlying clouds are distorted, hiding the true spectral signals of potentially growing convective clouds. Nevertheless, we are seeking out as diversified cases as possible in order to further validate the algorithm. Currently, our main focus is to acquire more validation data from radar sites under the MSG domain in Europe on convectively active days. So far, this has proven to be a very arduous task, but we are now working on a few lead ins that we have recently encountered and expect to add to our growing set of statistics from this.

13

ADEB Report Response Summary

ADEB Report Specific Comment:“I note that the CI definition as it stands now- 35 dBZ reflectivity in the base scan- is actually more representative of a mature storm that is already raining, rather than developing convection that has a 35 dBZ echo aloft and is newly glaciated.  A radar-based CI discriminator would surely be different. In fact, after the NEXRAD dual-pol upgrade, convective initiation might be better defined as the first occurrence of ice phased precipitation indicated by negative Zdr for example.  A more advanced GOES-R algorithm for earlier detection of deep convection might be considered for the post-launch science infusion.”

14

ADEB Report Response Summary

Response:Certainly, there is great debate within the community over what metric defines convective initiation, ranging from the first lightning strike, on one end of the spectrum, all the way up to the first detection of a newly forming cumuliform cloud. For our purposes here, we established and agreed early on that we would follow the metric found most prominently in the literature (Roberts and Rutledge, 2003; Mecikalski and Bedka, 2006, etc..), which is the first instance of a ≥35 dBZ radar echo, specifically, at a low elevation, such as 1,000 meters. Furthermore, the most commonly available radar data is that which originates from the base scan, which is difficult enough to obtain from radar sites existing within the MSG domain.

15

RequirementsConvective Initiation

M – Mesoscale C - CONUS

Nam

e

User &

Priority

Geograp

hic

Coverage

(G, H

, C, M

)

Vertical

Resolution

Horizon

tal

Resolution

Mapp

ing

Accu

racy

Measurem

ents

Ran

ge

Measurem

ents

Accu

racy

Produ

ct Refresh

Rate/C

overage Tim

e

Ven

dor Allocated

Grou

nd

Latency

Produ

ct Measurem

ent P

recision

Tem

poral Coverage Q

ualifiers

Produ

ct Exten

t Qu

alifier

Cloud

Cover C

ondition

s Q

ualifier

Produ

ct Statistics Q

ualifier

Convective Initiation

GOES-R

C N/A 2 km 1 km Binary Yes/No detection

70% Probability of Correct Detection

5 min 159 sec 5% Day and night

Quantitative out to at least 65 degrees LZA and qualitative beyond

Clear conditions down to feature of interest associated with threshold accuracy

Over specified Geographic area

Convective Initiation

GOES-R

M N/A 2 km 1 km Binary Yes/No detection

70% Probability of Correct Detection

5 min 159 sec 5% Day and Night

Quantitative out to at least 70 degrees LZA and qualitative beyond

Clear conditions down to feature of interest associated with threshold accuracy

Over specified Geographic area

C - CONUS M - Mesoscale

16

Validation Strategy

17

CI forecasts generated for cloud objects are subjectively compared to base radar reflectivity.

The first instance of a base ≥35 dBZ radar echo is used to validate the forecasts, per the literature (Roberts and Rutledge, 2003). Both “null” and “positive” CI forecasts are validated and used to fill the Dichotomous Forecast Verification contingency table below.

Validation Strategy

Dichotomous Forecast VerificationDichotomous Forecast Verification

Was CI Forecasted? Was CI Forecasted? YESYESDid CI Occur? Did CI Occur? YESYES

HitsHits

Was CI Forecasted? Was CI Forecasted? YESYESDid CI Occur? Did CI Occur? NONO

False AlarmsFalse Alarms

Was CI Forecasted? Was CI Forecasted? NONODid CI Occur? Did CI Occur? YESYES

MissesMisses

Was CI Forecasted? Was CI Forecasted? NONODid CI Occur? Did CI Occur? NONO

Correct NegativesCorrect Negatives

18

Validation Strategy Validation of individual cloud object forecasts starts from the time

each forecast is generated, and extends out to ~90 minutes.

Cloud motion must be subjectively analyzed and compared to potential downstream CI events, as observed from radar data.

Though the validation is performed subjectively, strict criteria are followed to ensure consistency.

Because each set of forecasts is based on IR imagery from independent sets of two input image times, each set of forecasts is completely independent of the previous set of forecasts, even for the same discernable cloud objects from different times. Therefore, different forecasts for the same cloud objects through time can be included in the contingency table statistics.

19

Validation Strategy Positive forecasts of CI upstream of corresponding radar-detected

CI events are considered as “HITS” in the contingency table.

Null forecasts of CI just prior to and upstream of corresponding radar-detected CI events are considered as “MISSES”.

Positive forecasts of CI upstream of non-existent CI events are considered as “FALSE ALARMS”.

Null forecasts of CI upstream of non-existent CI events are considered as “CORRECT NEGATIVES”.

Diagnostic (aka: “0-Lead time”) and Negative Lead Time forecasts are considered as “MISSES”. The first step of validation is to check if a CI forecast is given for convection that has already initiated.

20

Validation Strategy1702 UTC

1700 UTC 1717 UTC 1758 UTC

Validation example Validation example from AWG CI proxy from AWG CI proxy algorithm:algorithm:

June 6, 2011June 6, 2011North Florida/North Florida/Southern GeorgiaSouthern Georgia

Red= Positive CI FcstRed= Positive CI FcstBlue= Null CI FcstBlue= Null CI Fcst

21

Validation Strategy1702 UTC 1715 UTC

1700 UTC 1717 UTC 1758 UTC

Validation example Validation example from AWG CI proxy from AWG CI proxy algorithm:algorithm:

June 6, 2011June 6, 2011North Florida/North Florida/Southern GeorgiaSouthern Georgia

Red= Positive CI FcstRed= Positive CI FcstBlue= Null CI FcstBlue= Null CI Fcst

22

Validation Strategy1702 UTC 1715 UTC 1732 UTC

1700 UTC 1717 UTC 1758 UTC

Validation example Validation example from AWG CI proxy from AWG CI proxy algorithm:algorithm:

June 6, 2011June 6, 2011North Florida/North Florida/Southern GeorgiaSouthern Georgia

Red= Positive CI FcstRed= Positive CI FcstBlue= Null CI FcstBlue= Null CI Fcst

23

Validation Results

Three unique data set types have been used to run and to validate the AWG CI algorithm:

1) MSG SEVIRI data over Europe with 5-minute temporal resolution and available radar data within the domain.

2) RAMS Model simulated ABI radiances with 5-minute temporal resolution and simulated radar reflectivities (courtesy of CIRA).

3) Current GOES-East data over the CONUS with 15-30 minute temporal resolution and available WSR-88D radar data within the domain (CI forecasts generated by the AWG CI proxy algorithm, which is limited by poor spectral, temporal, and spatial resolutions).

24

Validation Results

Total CI forecasts validated: 14,671

Total validated: 184Accuracy: 80.4%POD: 87.0%FAR: 15.8%POFD: 32.8%

25

Validation Results

HITSHITS427427

False AlarmsFalse Alarms10441044

MISSESMISSES1818

Correct NegativesCorrect Negatives30553055

RAMS Simulated ABI Dataset

MSG SEVIRI Dataset

HitsHits255255

False AlarmsFalse Alarms308308

MISSESMISSES9999

Correct NegativesCorrect Negatives92819281

GOES CI Proxy Dataset

HitsHits107107

False AlarmsFalse Alarms2020

MissesMisses1616

Correct NegativesCorrect Negatives4141

Total validated: 4,544Accuracy: 76.6%POD: 96.0%FAR: 71.0%POFD: 25.5%

Total validated: 9,943Accuracy: 95.9%POD: 72.0%FAR: 54.7%POFD: 3.0%

Validation efforts from all 3 data sets have produced “Statistical Accuracies” above the required 70% specification.

However, more validation studies still need to be performed using our closest available ABI proxy for input data, the MSG SEVIRI data set, since it is neither model-simulated nor limited by antiquated spectral, spatial, and temporal resolutions.

26

Validation Results

27

The Convective Initiation algorithm provides a unique multi-spectral and temporal differencing approach to forecasting the locations of near-term future convection. It takes advantage of the many IR spectral channels available with the ABI.

This product meets all required specifications.

More validation is planned, using data from the MSG SEVIRI instrument.

Version 5 was just delivered, and the 100% ATBD is slated for delivery soon.

Summary