give me odds forecasting ensemble comap symposium 00-1 presented by steve tracton wednesday, 15...
TRANSCRIPT
GIVE ME ODDSGIVE ME ODDS
FORECASTING
ENSEMBLE
COMAP Symposium 00-1Presented by Steve TractonWednesday, 15 December
1999
KEY POINTSKEY POINTS
• THERE ARE INEVITABLE UNCERTAINTIES IN NWP DUE TO UNCERTAINTIES IN INITIAL CONDITIONS AND MODEL FORMULATION
• WEATHER FORECASTING, THEREFORE, IS INHERENTLY STOCHASTIC, NOT DETERMINISTIC IN NATURE
• ENSEMBLE PREDICTION - REVOLUTIONARY CHANGE IN THE THRUST OF OPERATIONAL NWP (“WAVE OF THE FUTURE”) - CONSISTS OF MULTIPLE PREDICTIONS FROM SLIGHTLY DIFFERENT INITIAL CONDITIONS AND/OR WITH VARIOUS VERSIONS OF MODELS, THE OBJECTIVES BEING TO:
– IMPROVE SKILL THROUGH ENSEMBLE AVERAGING, WHICH ELIMINATES NON-PREDICTABLE COMPONENTS
– PROVIDE RELIABLE INFORMATION ON FORECAST UNCERTAINTIES (E.G., PROBABILITIES) FROM THE SPREAD (DIVERSITY) AMONGST ENSEMBLE MEMBERS
• REALITY - POSITIVE RESULTS ON BOTH COUNTS WITH OPERATIONAL GLOBAL MODEL ENSEMBLE SYSTEM; EXPERIMENTAL REGIONAL MODEL ENSEMBLES ENCOURAGING (OPERATIONAL EARLY 2000?)
– NET RESULT - ENHANCE UTILITY OF NWP FOR VIRTUALLY ALL APPLICATIONS
• REALIZING THE PRACTICAL UTILITY OF ENSEMBLES ACCOMPLISHED VIA A VARIETY OF NEW PRODUCTS DESIGNED TO CONDENSE AND MAXIMIZE INFORMATION CONTENT FOR USERS; USER FEEDBACK ESSENTIAL AND ENCOURAGED!!!
KEY CONSIDERATIONSKEY CONSIDERATIONS
• STRATEGIES FOR CREATING ENSEMBLES
– PROCDEDURES FOR GENERATING INITIAL STATE PERTURBATIONS
• RANDOM
• TIME LAGGING
• ANALYSES FROM OTHER CENTERS
• “BREEDING”
• SINGULAR VECTORS
– PERTURBING MODEL (E.G., CONVECTIVE PARAMETERIZATION) AND/OR MULTI-MODEL ENSEMBLES
– MODEL CONFIGURATION?
• RESOLUTION
• PHYSICAL SOPHISTICATION
• DOMAIN
– ENSEMBLE SIZE
NOTE: OPTIMUM STRATEGY UNKNOWN (NO CONCENSUS)!!
IDEAL: EFFECTIVE/EFFICIENT SAMPLING OF ALTERNATIVE SCENARIOS, I.E., PROBABILITY DISTRIBUTIONS. LIMITED COMPUTER RESOURCES GENERALLY REQUIRE COMPROMISES RELATIVE TO PERCEIVED OPTIMUM, E.G., MODEL RESOLUTION VERSUS ENSEMBLE SIZE)
KEY CONSIDERATIONS(CONT.)KEY CONSIDERATIONS(CONT.)
• PRODUCT DEVELOPMENT
OBJECTIVE:CONDENSE LARGE AMOUNTS OF OUTPUT INTO A “USER FRIENDLY” FORM THAT PROVIDES RELIABLE ESTIMATES OF THE RANGE AND LIKLIHOOD OF ALTERNATIVE SCENARIOS
– PRODUCTS CAN RANGE FROM DISPLAY OF ALL FORECASTS THROUGH MEANS/SPREAD AND CLUSTERS TO FULL PROBABILITIY DISTRIBUTIONS DISPLAYED IN VARIOUS FORMATS
• STATISTICAL POSTPROCESSING (E.G., BIAS CORRECTIONS, CALIBRATION OF PROBABILITIES
• ENSEMBLE OUTPUT STATISTICS
• ADDITIONAL/ALTERNATIVE PRODUCTS CONTINUAL INTERACTION AMONGST CONTINUAL INTERACTION AMONGST DEVELOPERS AND USERSDEVELOPERS AND USERS
• VALIDATION
– STANDARD SKILL SCORES
– MEASURES OF SPREAD
– MEASURES OF RELIABILITY
• EDUCATION AND TRAINING
– COMET SYMPOSIUM
– TRAINING MODULES
– ON SITE VISITS
– WEB BASED
– ??
N-AWIPS GRAPHICAL PRODUCTSN-AWIPS GRAPHICAL PRODUCTS(GEMPAK META FILES)(GEMPAK META FILES)
• SPAGHETTI CHARTS– 500 Z– 1000Z– 1000/500 TCK– MSLP– 850 T– 700 RH
• SPREAD– 1000 Z– 500 Z
• CLUSTERS– 1000 Z– 500 Z
• PROBABILITIES– 500 Z > THRESHOLDS– 700 RH > 70%– TCK <540– 250 V > THRESHOLDS– 850 T > 0C
• MSLP CENTERS
PRODUCT DEVELOPMENT INCLUDESPRODUCT DEVELOPMENT INCLUDES
• PROBABILITIES
– VIRTUALLY ALL RELEVANT AND MODEL DERIVED PARAMETERS, E.G.,
• SEVERE WEATHER INDICES• AVIATION WINDS > THRESHOLD• SENSIBLE WEATHER ELEMENTS (MODEL
DERIVED/INFERRED• CIRCULATION INDICES (E.G., BLOCKING)
• EXPANDED CLUSTERED PARAMETERS AND FOR SPECIALIZED REGIONS
• VERTICAL PROFILES
• METEOGRAMS
• ENSEMBLE DERIVED MOS
• TROPICAL STORM TRACKS
– DIRECT FROM ENSEMBLES– BACKGROUND FOR GFDL MODEL ENSEMBLES
SOME APPLICATIONSSOME APPLICATIONS
• FORECASTS OF ENSEMBLE MEAN, SPREAD, PROBABILITY DISTRIBUTIONS, ETC. OF ANY MODEL FIELD/PARAMETER OR QUANTITIES DERIVED THEREFROM ENHANCE THE ENHANCE THE UTILITY OF FORECASTSUTILITY OF FORECASTS
• APPLICABLE TO MODELS FROM VERY SHORT RANGE CLOUD SCALE THROUGH REGIONAL MESOSCALE SHORT RANGE AND GLOBAL MEDIUM RANGE TO COUPLED OCEAN/ATMOSPHERE CLIMATE PREDICTION SYSTEMS
• IMPROVE DATA ASSIMILATION SYSTEMS
• ADAPTIVE/TARGETED OBSERVATIONS
• DATA SETS FOR FUNDAMENTAL RESEARCH ON PREDICTABILITY ISSUES
NOTE: LARGE CURRENT USER COMMUNITY (OPERATIONAL GLOBAL SYSTEM) INCLUDES NCEP SERVICE CENTERS, WFO’S, USAF, OH, PRIVATES/BROADCASTERS
CLUSTER ANALYSISCLUSTER ANALYSIS
OBJECTIVELY GROUP TOGETHER FORECASTS WHICH ARE SIMILAR ACCORDING TO SOME CRITERIA
GOAL: IDENTIFY EXTREMES, GROUPINGS (CLUSTERS) WITHIN ENVELOPE OF POSSIBILITIES (“ATTRACTORS”)
• ISSUES:
– QUANTITY
• MSLP• 500 Z• ETC.
– MEASURE
• ANOMALY CORRELATION• CIRCULATION PARAMETERS• PATTERN RECOGNITION• PHASE-SPACE MEASUREMENTS
– REGION
EVALUATION/VERIFICATIONEVALUATION/VERIFICATION
• SITUATIONAL AND PHENOMENOLOGICAL CASE STUDIES (E.G., CYCLOGENESIS, FLOOD POTENTIAL)
• STATISTICAL
– STANDARD AC, RMS, SCORES (E.G., APPLIED TO ENSEMBLE MEAN VS. CONTROL, RELATIVE CLOSENESS OF MEMBERS TO ANALYSIS)
– “TALAGRAND” (VERIFICATION RANK) DIAGRAMS - MEASURES OF BIASES IN DISTRIBUTION OF ENSEMBLE MEMBERS INCLUDING FREQUENCY OF OUTLIERS)
– BRIER, RANKED PROBABILITY SCORES (PROBABILITY SKILL SCORES)
– RELIABILITY DIAGRAMS (OBSERVED VERSUS FORECAST FREQUENCIES; ENABLES CALIBRATION OF PROBABILITIES)
– MOS VERSUS ENSEMBLE POPS
– RELATIVE OPERATING CHARACTERISTICS (ROC); (EXPLICIT COMPARISON OF THE RELATIVE UTILITY OF DETERMINISTIC AND ENSEMBLE PREDICTIONS)
SHORT RANGE ENSEMBLE FORECASTING SHORT RANGE ENSEMBLE FORECASTING (SREF)(SREF)
• OBJECTIVE: DEVELOP A REGIONAL MODEL, SHORT-RANGE (0-3 DAYS) ENSEMBLE PREDICTION SYSTEM TO PROVIDE OPERATIONALLY RELEVANT AND USEFUL GUIDANCE ON THE PROBABILITY DISTRIBUTION OF WEATHER ELEMENTS OR EVENTS, ESPECIALLY FOR QPF
• GOAL: IMPLEMENT INITIAL OPERATIONAL PRODUCTION OF A REGIONAL MODEL BASED ENSEMBLE SYSTEM AND PRODUCT SUITE (SREF-I) BY ~ JANUARY, 2000
– TARGET SYSTEM:• ETA PLUS RSM MULTI-MODEL• 10 MEMBER• 40 KM RESOLUTION • ~ETA DOMAIN• RUN TWICE PER DAY• PERTURBATIONS; REGIONAL “BREEDING”
– PRODUCT SUITE:• ENSEMBLE MEAN/SPREAD CHARTS• SPAGHETTI CHARTS• PROBABILITY CHARTS• METEOGRAMS
• STATUS MILESTONES• CONDUCT PILOT STUDIES (10/96-3/98)• PARTICIPATE IN STORM AND MESOSCALE ENSEMBLE
EXPERIMENT (SAMEX) (3/98-11/98)
• SOME ISSUES– ALTERNATIVE PERTURBATION STRATEGIES– TRADEOFFS; RESOLUTION, ENS SIZE/DOMAIN– PRODUCT DEVELOPMENT– VALIDATION PROCEDURES– DATA/PRODUCT DISSEMINATION– EDUCATION AND TRAINING
STATUS/MILESTONES
COMPLETED (PILOT STUDIES)
• MAJOR TASKS/ACCOMP MAJOR TASKS/ACCOMP (CONT.)(CONT.)
– ILLUSTRATE SIGNIFICANCE ILLUSTRATE SIGNIFICANCE
OF UNCERTAINTIES IN SREFOF UNCERTAINTIES IN SREF
– DEMONSTRATE THE DEMONSTRATE THE
POTENTIAL POTENTIAL OF SREF TO OF SREF TO
PROVIDE OPERATIONALLY PROVIDE OPERATIONALLY
USEFUL INFORMATIONUSEFUL INFORMATION
– PROVIDE BASIS FOR A PROVIDE BASIS FOR A
PROTOTYPE OPERATIONAL PROTOTYPE OPERATIONAL
SYSTEMSYSTEM
STATUS/STATUS/MILESTONESMILESTONES
COMPLETED (PILOT STUDIES)COMPLETED (PILOT STUDIES)
• SOME KEY FINDINGSSOME KEY FINDINGS
– ENHANCED DIVERSITY OF ENHANCED DIVERSITY OF
SOLUTIONS (SPREAD) SOLUTIONS (SPREAD)
WITH:WITH:
• MULTI-MODEL ENSEMBLEMULTI-MODEL ENSEMBLE
• HIGHER RESOLUTION HIGHER RESOLUTION
• GLOBAL BRED (VS GLOBAL BRED (VS
“RANDOM”)“RANDOM”)
• REGIONAL ENHANCEMENTREGIONAL ENHANCEMENT
STATUS/MILESTONESSTATUS/MILESTONESCOMPLETED (PARTICIPATE IN SAMEX)COMPLETED (PARTICIPATE IN SAMEX)
• BOTTOM LINE:BOTTOM LINE:
– MUCH GAINED, MUCH GAINED, ACCOMPLISHED, LEARNEDACCOMPLISHED, LEARNED
– RESULTS GENERALLY RESULTS GENERALLY FAVORABLEFAVORABLE
– SOME DISSAPPOINMENTS SOME DISSAPPOINMENTS RELATIVE TO RELATIVE TO EXPECTATIONS, EXPECTATIONS, BUTBUT WE WE UNDERSTAND WHYUNDERSTAND WHY
– REMAIN COMMITED TO REMAIN COMMITED TO BASIC STRATEGYBASIC STRATEGY
STATUS/STATUS/MILESTONESMILESTONES
COMPLETED (PARTICIPATE IN SAMEX)COMPLETED (PARTICIPATE IN SAMEX)
• SOME KEY FINDINGSSOME KEY FINDINGS
– IMPROVED ENS MEAN IMPROVED ENS MEAN
SKILL, RELIABILITY, RPSS SKILL, RELIABILITY, RPSS
WITH MULTI-MODEL WITH MULTI-MODEL
APPROACH APPROACH
– INSUFFICENT SPREAD INSUFFICENT SPREAD
• DOMAIN TOO SMALL - DOMAIN TOO SMALL -
NEGATIVE IMPACT OF NEGATIVE IMPACT OF
BC’SBC’S
– PRECIPITATION FORECASTS PRECIPITATION FORECASTS
“WOEFUL”“WOEFUL”
• WEAK FORCING, WEAK FORCING,
SUMMER LIKE PATTERN SUMMER LIKE PATTERN
SAMEX DOMAIN
LARGE
SMALL
SAMEX SYSTEM:
MULTI-MODEL (ETA/RSM)10 MEMBERS32 KM RESOLUTIONSAMEX DOMAINREGIONAL ENHANCEMENT