bringing it all together - the conference …...bringing it all together: a prototype for a...
TRANSCRIPT
BRINGINGITALLTOGETHER:APROTOTYPEFORAPROBABILSITICNATIONALBLEND
Acknowledgements:KevinKelleher,MattPeroutka,Yuejian Zhu,JohnWagner,GearyLayne,JeffreyCraven
1
ZoltanTothOAR/ESRL/GSD
RomanKrzysztofowiczUniversityofVirginia
MarkAntolikNWS/MDL
Malaquias PenaNWS/NCEP/EMC&IMSG
MelissaPetty,GearyLayneOAR/ESRL/GSD
AMS2017AnnualMeeting
OUTLINE / SUMMARY• Role & relevance of NWP & SPP in forecasting
– NWP provides predictive information– SPP fixes NWP miscalibration - Potentially major user impact
• Proposed framework for National Blend of Models (NBM)– Combine & calibrate guidance for NWP prognostic variables– Derive user variables from calibrated prognostic variables
• A prototype for probabilistic NBM– Bayesian Processor of Ensemble (BPE)
• Combines multiple ensemble / high res forecasts & obs/analyzed climate• Unified processing of all continuous variables• Comprehensive set of output variables
• BPE status– Algorithm, software developed– Ongoing transfer to MDL– Comparative test against EKDMOS next
2
STATISTICAL POST-PROCESSING (SPP) IN NWP• NWP is basis of weather forecasting
– Provides all predictive information beyond 6 hrs• Data assimilation, numerical model, ensemble approach
• Systematic errors limit utility of NWP– Forecasts are miscalibrated (mislabeled)
• Statistical Post-processing– Re-labels forecast information
• Traditional approach to SPP– Process each NWP guidance product individually
• NAM-MOS, GFS-MOS, NAEFS-EKDMOS, etc• Multitude of products overwhelm forecasters
• Birth of NWS National Blend of Models (NBM)– Merge & calibrate forecasts
3
• Drift of model solutions from real to model world– 1st moment calibration
• Uncertainty not captured well by single/ens fcsts– 2nd moment calibration
• Abundance of guidance products– Fusing all predictive information
Lead-time dependent behavior ofNWP Model Prog Variables (MPV) on model grid
• Missing variables– Limited model resolution in space, processes, variables
Simultaneous relationships btwUser Specific Variables on fine & MPVs on model grid
PROBLEMS WITH NWP FORECASTS
4
5
FORECASTDAY-1
FORECASTDAY-2
FORECASTDAY-N
USERSPECIFICVARIABLES
OBJECTIVEOFSTATISTICALPOST-PROCESSINGMultiplelead
tim
es
Uncalibratedmodelvariables Calibrateduservariables
Coarsermodelgrid
Finescalegrid
6
USERSPECIFICVARIABLES
Modelgridpoints (M) <<(N)Finescalegridpoints
COMMONAPPROACHTOPOST-PROCESSING1-Step,directapproach
Multiplelead
tim
es(L)
Uncalibratedmodelvariables Calibrateduservariables
Multitudesoflead-timeandfine-scalegriddependentrelationships
O(10-100)
FORECASTDAY-1
FORECASTDAY-2
FORECASTDAY-N
Cost:LxMx(N/M);Retrainforeachmodelupdate
7
NWPANALYSIS
FORECASTDAY-1
FORECASTDAY-2
FORECASTDAY-N
CALIBRATEDMODELPRO
GVARS
Mo d e l g r i d ( M )
CalibrationToolbox
CALIBRATIONOFMODELPROGNOSTICVARIABLESMultiplelead
tim
es
(Re)forecastsample NWP(re)analysis
Cost:LxM
8
NWPANALYSIS
FORECASTDAY-1
FORECASTDAY-2
FORECASTDAY-N
CALIBRATEDMODELPRO
GVARS
Mo d e l g r i d
CalibrationToolbox
CALIBRATIONOFMODELPROGNOSTICVARIABLESMultiplelead
tim
es
(Re)forecastsample NWP(re)analysis
9
NWPANALYSIS
FORECASTDAY-1
FORECASTDAY-2
FORECASTDAY-N
CALIBRATEDMODELPRO
GVARS
Mo d e l g r i d ( M )
CalibrationToolbox
CALIBRATIONOFMODELPROGNOSTICVARIABLESMultiplelead
tim
es
(Re)forecastsample NWP(re)analysis
N/Msavingsfromfixing
modelproblemsonmodelgrid
10
NWPAN
ALYSES
USERSPECIFICVARIABLES
Mo d e l g r i d Finescalegrid
DerivationToolbox
DERIVATIONOFUSERVARIABLES
Noneedforhindcasts
NWPreanalysis Finescale(re)analysis
Single, instantaneousrelationship
11
NWPAN
ALYSES
USERSPECIFICVARIABLES
Modelgrid Finescalegrid
DerivationToolbox
DERIVATIONOFUSERVARIABLES
Noneedforhindcasts
NWPreanalysis Finescale(re)analysis
Single, instantaneousrelationship
Cost:Mx(N/M)Retrainonlyforreanalysisupdate
12
NWPANALYSIS
FORECASTDAY-1
FORECASTDAY-2
FORECASTDAY-N
CALIBRATEDMODELPRO
GVARS
USERSPECIFICVARIABLES
Mo d e l g r i d Finescalegrid
CalibrationToolbox
DerivationToolbox
PUTTINGITTOGETHER:2-STAGEPOST-PROCESSING
Noneedforhindcasts
Multiplelead
tim
es
(Re)forecastsample NWPreanalysis Finescale(re)analysis
Single,simultaneousrelationship
1-STEPvs2-STAGEAPPROACHTOPOST-PROCESSING• Disadvantagesof1-stepapproach
– Addressescoarsescalemodelproblemonfinescalegrid– Redundancy– Separatederivationofuservariablesforeachleadtime- Redundancy– Requireshindcast sample
• Benefitsof2-stageapproach– Addressesindependentproblemsseparately
• Betterunderstanding/separatemetricstofocusonissues– Modulardesigneasescommunityinvolvement,supportscollaboration– Shareddevelopmentofderivationtoolbox
• Vastarrayof“perfectprog”toolsavailable– UPP,DAforwardmodels,satelliteproductgeneration,etc
– Calibratedmodelvariablescanbeusedindownstreamcoupledappls• Liberatesdownstreamusersfromeverchangingmodelversionspecificbiases
– Developandusereanalysis– basedrelationships• One-waycoupledusermodels(eg,hydro)canbecalibratedwithreanalysis
– Simplifiesforecastediting– Modifycalibratedmodelprog variables• Multitudeofconsistentuservariablesautomaticallyderivedw/oaddition.edits
– Thoroughanalysisofsystematicmodelerrorsformodeldevelopers 13
RECOMMENDED APPLICATION• Calibrate all model prognostic variables
– Lead-time dependent behavior eliminated• NWP (re)analysis as proxy for truth• Calibrated forecasts statistically behave like analyses• Systematic error estimates useful for model developers
– NWP-based predictive info fused on model grid– Scales with number of model gridpoints x leadtimes
• Derive additional user variables– Fine scale analysis as proxy for truth (RUA)– Instantaneous relationships between analyses of
• Model Prog Vars (MPV) & User Specific Variables (USV)– No lead-time dependency - No need for hindcasts
– Toolbox with various types of routines• UPP, DA forward models and other physical & statistical methods
– Scales with number of fine scale gridpoints14
Mode l P rognost i c Va r i ab le sREALTIMENWPFORECASTEnsemble,HighRes.ControlHindcast,(Re)analysis
FineScaleUserVar.
Reanalysis
15
Calibrate&ConsolidateNWPOutput
CalibratedModelProgVariables
CalibratedUser
VariablesForecastEditor
NWSFORE
CASTER
S
DeriveFineScaleUserVars.
6D-DATACUBE
Product Generator
Pregenerated Products,InteractiveServices
EXTERN
ALUSERS
3rd PartyAppls.
CoupledModels
Climate&ForecastDistributionsEnsembleScenarios
H ISTOR ICAL DATA
16
BayesianProcessorofEnsemble
CalibratedModelProgVariables
CalibratedUser
VariablesProposediEdit
NWSFORE
CASTER
S
PlannedBays.Deriv.,UPP,etc
6D-DATACUBE
BPEProduct Generator,othertools
Pregenerated Products,InteractiveServices
EXTERN
ALUSERS
3rd PartyAppls.
CoupledModels
Climate&ForecastDistributionsEnsembleScenarios
Mode l P rognost i c Va r i ab le sREALTIMENWPFORECASTEnsemble,HighRes.ControlHindcast,(Re)analysis
FineScaleUserVar.
ReanalysisHISTOR ICAL DATA
BAYESIANPROCESSOROFENSEMBLE- BPE
1) Estimateclimatedistribution
2) Assessskillin&combineallpredictors
3) Castforecastinclimatecontext
4) Offermultipleoutputformats17
!
• Quantifies&reducesuncertainty• Calibrates(de-biases)ensemble
Krzysztofowicz &Toth 2008,Krzysztofowicz 2010
BAYESIANPROCESSOROFENSEMBLE(BPE)BASICS• Useclimatologyasbasis– linkwclimateprediction
– Fitparametricdistributiontolargeclimatesample• Calibrationensured,extremeshandledgracefullyw/olargehindcast
– Expressforecastsasanomaliesfromclimate• Noneedforlargehindcast sampletodefineobs.Climatology
• Transformeachvariableintonormalspace– UnifiedapproachforALLcontinuousvariables
• Assesspredictiveinformationinguidanceproducts– Multilinearregression,1st &2nd momentsaspredictors
• Wellestablishedtechnology(MOS);Forecastinformationmaximized
• Processcurrentforecast– Calibratedforecastasmodifiedclimatedistribution(percentiles)
• Mosteconomicstorage– 2modifiedparsofclimatedistribution– Prob.dist.,quantiles,posteriorensemble(scenarios/jointprob)
18
ConnectsNWPoutputw.usersviavalueaddedsteps• Role
– Repositoryofauthoritativeprobabilisticguidance• Forecastermodificationviaeditortool
• Function– Holdsanswertoallquestionsaboutfutureenviron.
• Accessinfoviainterrogatortool• Content
– Calibratedprog.vars.frompost-processingtool– Calibrateduservariables– incl.climatepercentiles
• Format– 6D - Space(3),time,variables,uncertainty
• Implementation– Phase1 – Only 2-foldexpans.ofcurrentNDFD
• Climatologicaldistributions– negligiblecost• 2BPEposteriorparameterstodescribefcst distribs
– Phase2• Ensemblemembersforjointprobabilities&scenarios
6DNDFD– CENTRALTENETOFFORECASTPROCESS
19
NATIONAL BLEND OF MODELS –BRINGING ALL THE PIECES TOGETHER
• Modular/comprehensive/expandable framework• Academic research & NOAA operations• Array of theory based techniques• End-to-end workflow w up-to-date software• Variety of forecast products & climate info• All model prog vars processed in unified way• Calibrated and informative guidance• Comprehensive array of output formats• Serving diverse applications
20
BACKGROUND
2
OUTLINE / SUMMARY• Role & relevance of NWP & SPP in forecasting
– NWP provides predictive information– SPP fixes NWP miscalibration - Potentially major user impact
• Proposed framework for National Blend of Models (NBM)– Combine & calibrate guidance for NWP prognostic variables– Derive user variables from calibrated prognostic variables
• A prototype for probabilistic NBM– Bayesian Processor of Ensemble (BPE)
• Combines multiple ensemble / high res forecasts & obs/analyzed climate• Unified processing of all continuous variables• Comprehensive set of output variables
• BPE status– Algorithm, software developed– Ongoing transfer to MDL– Comparative test against EKDMOS next
22
CURRENT GFS CONFIGURATION
23
GFSModel UPPGFS
COMPONENTS
OUTPUTRawModelPrognosticVariables
BiasedUser
Variables
APPLICATIONS Coupled Models & Other Downstream User Applications
NGGPS SYSTEM DESIGN CONSIDERATIONSComponent functionalities, links, software infrastructure
24
CalibrationStat.Post-Pr.
NWPModel
DerivationUPP,SPP,etc
NGGPSCOMPONENTS
OUTPUTRawModelPrognosticVariables
CalibratedModelProgVariables
CalibratedFineScaleUserVars
APPLICATIONS Coupled Models & Other Downstream User Applications
Two-
way
Cou
plin
g
One
-way
Cou
plin
g
BPEPROJECTSTATUS• OMMversion- Oneensemble,Multiplecontrol&Multipleauxiliary
predictors– Algorithm&documentationcomplete(U.Va)– Codescomplete(GSD)– Off-linetesting(GSD,ongoing)– TransitiontoMDL(Oct16)– TestingatMDL(Nov16)
25
• MMMversion– Multipleensemblesadded– Algorithm&documentation(U.Va,Sep16)– Codescomplete(GSD,Nov16)– Off-linetesting(GSD,Jan17)– ComparisonwEKDMOSinoperationalenvironment(MDL,Mar17)
• Assessquality&computationalspeedatobservationsites
• Finalreport(All,May2017)
LikelihoodFunctionEstimator