Assessing Forecast Assessing Forecast Uncertainty in the Uncertainty in the National Digital National Digital Forecast DatabaseForecast Database

AMS Weather Analysis and ForecastingAMS Weather Analysis and ForecastingMatt Peroutka, Greg Zylstra, John WagnerMatt Peroutka, Greg Zylstra, John WagnerNOAA/NWS Meteorological Development NOAA/NWS Meteorological Development LabLabAugust 1, 2005August 1, 2005

National Digital National Digital Forecast Database Forecast Database (NDFD)(NDFD) Contains a mosaic of Contains a mosaic of

NWS digital forecastsNWS digital forecasts Is available to all users Is available to all users

and partners – public and partners – public and privateand private

Allows users and Allows users and partners to create wide partners to create wide range of text, graphic, range of text, graphic, and image productsand image products

How is NDFD Produced?How is NDFD Produced?

User-Generated ProductsUser-Generated Products

– Detailed– Interactive– Collaborativ

e

NWS Automated ProductsNWS Automated Products

TextText

GraphicGraphic

DigitalDigital

VoiceVoice

National Digital National Digital ForecastForecast

Database Database

Local Digital Local Digital ForecastForecast

Database Database

Field Field OfficesOffices

National National CentersCentersCollaborateCollaborate

Data and Science FocusData and Science Focus

National CentersNational Centers Model GuidanceModel Guidance

GridsGrids

TODAY...RAIN LIKELY.

SNOW LIKELY ABOVE

2500 FEET. SNOW

ACCUMULATION BY

LATE AFTERNOON 1 TO

2 INCHES ABOVE 2500

FEET. COLDER WITH

HIGHS 35 TO 40.

SOUTHEAST WIND 5 TO

10 MPH SHIFTING TO

THE

SOUTHWESTEARLY

THIS AFTERNOON.

CHANCE OF

PRECIPITATION 70%.

TODAY...RAIN LIKELY.

SNOW LIKELY ABOVE

2500 FEET. SNOW

ACCUMULATION BY

LATE AFTERNOON 1 TO

2 INCHES ABOVE 2500

FEET. COLDER WITH

HIGHS 35 TO 40.

SOUTHEAST WIND 5 TO

10 MPH SHIFTING TO

THE

SOUTHWESTEARLY

THIS AFTERNOON.

CHANCE OF

PRECIPITATION 70%.

Single-valued Single-valued ForecastsForecasts Most NDFD weather Most NDFD weather

elements are elements are single-valuedsingle-valued

Can be viewed as a Can be viewed as a limitationlimitation

Forecast Forecast uncertainty uncertainty information can information can enhance customer enhance customer decision processes decision processes

Numeric Uncertainty Numeric Uncertainty Assessment of NDFD via Assessment of NDFD via Climatology and Ensembles Climatology and Ensembles (NUANCE)(NUANCE)

NDFDForecast

NDFDPerformance

RelatedGuidance

ExpectedDistribution ofObservations

NUANCE

Development and Development and ImplementationImplementation

DevelopmentDevelopment– Amass observation Amass observation (x)(x) and forecast and forecast (f) (f)

pairs pairs – Model joint distribution, Model joint distribution, p(f,x)p(f,x)– Refine with diagnostic data Refine with diagnostic data (d)(d) to form to form p(f,x,d)p(f,x,d)

ImplementationImplementation– Use current values of Use current values of ff and and dd, and , and p(f,x,d)p(f,x,d)– Infer conditional distribution of observations, Infer conditional distribution of observations, p(x|f,d)p(x|f,d)

Data SourcesData Sources

Ideally, NDFD grids and Analysis of Ideally, NDFD grids and Analysis of RecordRecord

Prototype with NDFD point forecasts and Prototype with NDFD point forecasts and METAR observationsMETAR observations

Ensemble MOS (ENSMOS) archivesEnsemble MOS (ENSMOS) archives– One set of forecasts from control runOne set of forecasts from control run– Five sets of forecasts created from runs with Five sets of forecasts created from runs with

positive perturbationspositive perturbations– Five sets of forecasts created from runs with Five sets of forecasts created from runs with

negative perturbationsnegative perturbations

Diagnostic DataDiagnostic Data

Standard Deviation (SD) of 11 Standard Deviation (SD) of 11 ENSMOS forecasts.ENSMOS forecasts.

““Ensemble Deviation” (ED)Ensemble Deviation” (ED)– Difference each perturbed forecast Difference each perturbed forecast

with control forecastwith control forecast– Compute root mean squareCompute root mean square

Transformation to Transformation to PercentilesPercentiles Useful to transform both Useful to transform both ff and and xx from from

native values to climatological native values to climatological percentilespercentiles– Data from U. S. Historical Climatology Data from U. S. Historical Climatology

Network (USHCN)Network (USHCN) Addresses lack of extreme cases in Addresses lack of extreme cases in

development data development data Encourages combining of dataEncourages combining of data

– NDFD has a short historyNDFD has a short history– Expect Expect p(f,x)p(f,x) to vary by region and by to vary by region and by

forecastforecast

Development PhaseDevelopment Phase

Observations (x)

Forecasts (f)

Diagnostic Data (d)

MergeJoint

Distribution Model p(f,x,d)

Transform to

Percentiles

Implementation PhaseImplementation Phase

Forecasts (f)

Diagnostic Data (d)

Joint Distribution

Model p(f,x,d)

Extract

Inferred Conditional Distribution

p(x | f,d)

Transform from

Percentiles

Transform to

Percentiles

Results: Results: Transformation to Transformation to PercentilesPercentiles Obtained daily maximum temperature Obtained daily maximum temperature

(MaxT) observations for 168 stations from (MaxT) observations for 168 stations from USHCNUSHCN

Computed percentile function at 5-day Computed percentile function at 5-day intervals throughout the yearintervals throughout the year

Used Generalized Lambda Distribution to Used Generalized Lambda Distribution to model percentile functionmodel percentile function– Percentile function fitted to observations.Percentile function fitted to observations.– Fit parameters expressed as cosine series over day Fit parameters expressed as cosine series over day

of the year.of the year.– Quality of fit judged subjectively.Quality of fit judged subjectively.– Additional terms added to cosine series, if needed.Additional terms added to cosine series, if needed.

Results of Percentile Transform Technique for Blythe, California

Baudette, Minnesota (KBDE; blue)Fort Lauderdale, Florida (KFLL; purple)Blythe, California (KBLH; red)

Comparison of MaxT Percentiles

Results: Modeling Results: Modeling Joint DistributionJoint Distribution

Day 1 vs. Day 7Day 1 vs. Day 7

Day 1 forecasts Day 1 forecasts verify better verify better

Day 7 points cluster Day 7 points cluster around the 0.50 around the 0.50 forecast value moreforecast value more

Fewer extreme Fewer extreme forecasts on Day 7forecasts on Day 7

Results: Diagnostic Results: Diagnostic DataData Day 7 forecasts, Day 7 forecasts,

stratified by Ensemble stratified by Ensemble Deviation.Deviation.

ED < 6° F (above) vs. ED < 6° F (above) vs. ED ≥ 6° F (below). ED ≥ 6° F (below).

Suggests more skillful Suggests more skillful forecasts when ED < 6° forecasts when ED < 6° F, but relationship is not F, but relationship is not obvious. obvious.

Future PlansFuture Plans

Quantitatively assess uncertaintyQuantitatively assess uncertainty Expand to include minimum Expand to include minimum

temperaturetemperature Work with gridsWork with grids Prototype experimental guidance Prototype experimental guidance

productsproducts