NOAA Climate Test Bed (CTB) Overview

Jin Huang

October 21, 2014

NOAA’s 39th Climate Diagnostics and Prediction Workshop

Outline: • Introduction to CTB • CTB recent activities and accomplishments • Summary

2

• EMC WRF Developmental Test Center (DTC) Joint Center for Satellite Data Assimilation • CPC Climate Test Bed

• NHC Joint Hurricane Test Bed

• HPC Hydrometeorological Test Bed

• SPC Hazardous Weather Test Bed with NSSL

• SWPC Space Weather Prediction Test Bed with AFWA

• AWC Aviation Weather Test Bed

• OPC IOOS Supported Test Bed

NCEP Test Beds Service – Science Linkage with the Outside Community:

Accelerating the R2O Transition Process

3

Organization structure, scope and funding sources are different for different test beds

4

NOAA Climate Test Bed (CTB)

AONCEP Co-PI

LOIProposal

R2O

O2R

Improved products and

services

Research OperationsClimate Forecast

ProductsMME

CFS Improvements

• Reanalysis / Reforecasts

• Earth System Modeling

• Tropical oscillations

• Model physics

• Etc.

Res

earc

h To

pics

AONCEP Co-PI

LOIProposal

R2O

O2R

Improved products and

services

Research OperationsClimate Forecast

ProductsMME

CFS Improvements

• Reanalysis / Reforecasts

• Earth System Modeling

• Tropical oscillations

• Model physics

• Etc.

Res

earc

h To

pics

Mission: Advancing Operational Climate Prediction and Products • Accelerate research-to-operation

(R2O) transition to improve NCEP operational climate prediction

• Provide O2R (operation-to-research) support to the climate research community with access to operational models, forecast tools and datasets

Three priorities: • CFS improvements • Multi-model ensembles • Climate forecast products

• Joint NCEP and CPO Effort • NCEP FTEs • CTB grants funded

by CPO/MAPP

• CTB Science Advisor Board (SAB)

• Established in 2005

Requirements For MAPP-CTB Projects

• A MAPP-CTB proposal much include a section with metrics to be used to evaluate the outcomes of the project ans assess readiness for transition into NCEP’s operations.

• A MAPP-CTB proposal much include NCEP Co-Is or collaborators.

• NCEP management (CTB, CPC,EMC) will review the MAPP-CTB proposals and provide support letters before the proposals are sent out for external reviews.

• Once the projects are funded, transition plans for operational deployment need to be developed (a new requirement and process!)

Discussion on CTB Transition Plans at 5-6:20pm: • For currently funded CTB PIs and NCEP Co-Is and collaborators, but others

are welcome. • Short (5 minutes) presentations from the funded projects • A good opportunity for questions and discussions.

CTB Priority (1): Multi-Model Ensembles NMME (North American Multi-Model Ensemble) An unprecedented MME system to improve intra-seasonal to interannual (ISI) operational predictions based on the leading US and Canada climate models.

NMME Phase-I: An experimental system initiated as a CTB research project supported by CPO/MAPP in FY11. NMME Phase-II: An improved experimental seasonal prediction system as a FY12-13 MAPP/CTB project with contributions from NSF, DOE and NASA.

• Ended July 2014 • Extended to July 2015 • Post-project review in

September 2014 NMME Expansion Project in High Impact Weather Prediction Pilot (HIWPP) Project from Sandy Supplemental Fund

Organizations Models

NOAA/NCEP CFSv2

NOAA/GFDL CM2.1 FLOR (March 2014)

NASA/GMAO GEOS5

Environment Canada CMC1-CanCM3 CMC2-CanCM4

NCAR CCSM3.0 CCSM4.0 (July 2014)

NCAR CESM1.0 (Jan 2015)

*new/upgraded models

Current NMME Forecast Providers

NMME Major Accomplishments

• Contributed to NCEP real-time seasonal forecasts since August 2011

• Demonstrated that the diversity of models in NMME is enhancing the forecast skill beyond CFSv2 alone

• NMME data is the most comprehensive seasonal prediction data set

(1.5 PB) accessible by the public o All participating models follow strictly on the NMME protocol o Data are available in both hindcast and real-time => applications

• Served as a facility that enables predictability and prediction

research and informs model development

• Introduced four new/upgraded models during Phase-II period

• Tested a sub-seasonal MME prediction protocol

July 1 start DJF SST forecast Ranked Probability Skill Score based on 30-year NMME hindcast data (B. Kirtman et al.)

NMME

CFSv2

Comparison of CFSv2 skill vs NMME

NMME Improves Forecast Reliability

Figure 7: Reliability diagram for T2m Northern Hemisphere land (23N-75N). Upper left is CFSv2, upper right is the results from the mini-NMME and the bottom is the full NMME. Histograms are given for each panel. Red corresponds to above normal, brown for near normal and blue for below normal.

CFS (1 model, 24 members)

Mini-NMME (6 models, 24 members)

Full-NMME ( 6 models, ~ 100 members)

NMME increases forecast reliability due to both ensemble size and model diversity (after van den Dool)

Real-time Verification (March 2014, Becker)

NMME Data Available to Public

1. Realtime forecasts and verifications from CPC website • http://www.cpc.ncep.noaa.gov/products/NMME/

2. Phase-I Reforecast data in IRI website available now

• Monthly Mean of 30 year reforecast • 8 variables (P, T, SST, Z200, Tmax, Tmin Soil Moisture, Runoff ) • http://iridl.ldeo.columbia.edu/SOURCES/.Models/.NMME/

3. Phase-II Reforecast data in NCAR • Data available starting July 2014, March 2015 for complete set • Selected (22) daily atmospheric and land variables • Daily atmospheric pressure level fields (5) • Monthly seas ice and ocean fields (9) • https://www.earthsystemgrid.org/search.html?Project=NMME

• Bastola, Satish, Vasubandhu Misra, Haiqin Li, 2013: Seasonal Hydrological Forecasts for Watersheds over the Southeastern United States for the Boreal Summer and Fall Seasons. Earth Interact., 17, 1–22. doi: http://dx.doi.org/10.1175/2013EI000519.1

• Becker, Emily, Huug van den Dool, Qin Zhang, 2014: Predictability and Forecast Skill in NMME. J. Climate, 27, 5891–5906. doi: http://dx.doi.org/10.1175/JCLI-D-13-00597.1.

• DelSole, T., J. Nattala and M. K. Tippett, 2014: Skill improvement from increased ensemble size and model diversity. Geophys. Res. Lett., (submitted). • DelSole, T., and M. K. Tippett, 2014: Comparing forecast skill (submitted) • Infanti, Johnna M., Ben P. Kirtman, 2014: Southeastern U.S. Rainfall Prediction in the North American Multi-Model Ensemble. J. Hydrometeor, 15, 529–550.

doi: http://dx.doi.org/10.1175/JHM-D-13-072.1. • Jia, L. and coauthors (2014): Improved Seasonal Prediction Skill of Land Temperature and Precipitation in a GFDL High-Resolution Climate Model, J. Climate (submitted). • Kirtman, B. P., and co-authors, 2014: The North American Multi-Model Ensemble (NMME): Phase-1 Seasonal-to-Interannual Prediction, Phase-2 Toward Developing

Intra-Seasonal prediction. Bull. Amer. Met. Soc.,doi: http://dx.doi.org/10.1175/BAMS-D-12-00050.1. • Kirtman, B. P., and D. Min, 2009: Multi-model ensemble ENSO prediction with CCSM and CFS. Mon. Wea. Rev., DOI: 10.1175/2009MWR2672.1. • Kumar, Arun, Peitao Peng, Mingyue Chen, 2014: Is There a Relationship between Potential and Actual Skill?. Mon. Wea. Rev., 142, 2220–2227.

doi: http://dx.doi.org/10.1175/MWR-D-13-00287.1 • Larson, S. M., and B. P. Kirtman, 2014: Assessing Pacific Meridional Mode forecasts and its role as an ENSO precursor and predictor in the North American Multi-Model

Ensemble. J. Climate (in press). • William J. Merryfield, Woo-Sung Lee, George J. Boer, Viatcheslav V. Kharin, John F. Scinocca, Gregory M. Flato, R. S. Ajayamohan, John C. Fyfe, Youmin Tang, and

Saroja Polavarapu, 2013: The Canadian Seasonal to Interannual Prediction System. Part I: Models and Initialization. Mon. Wea. Rev.,141, 2910–2945. • Misra, V., and H. Li (2014), The seasonal climate predictability of the Atlantic Warm Pool and its teleconnections, Geophys. Res. Lett., 41, 661–666,

doi:10.1002/2013GL058740. • Mo, Kingtse C., Dennis P. Lettenmaier, 2014: Hydrologic Prediction over the Conterminous United States Using the National Multi-Model Ensemble.J. Hydrometeor, 15,

1457–1472. doi: http://dx.doi.org/10.1175/JHM-D-13-0197.1 • Paolino, D. A., J. L. Kinter III, B. P. Kirtman, D. Min, D. M. Straus, 2012: The impact of land surface initialization on seasonal forecasts with CCSM. J. Climate, 25, 1007-

1021. • Rodrigues, L. R. L., J. García-Serrano, and F. Doblas-Reyes (2014), Seasonal forecast quality of the West African monsoon rainfall regimes by multiple forecast systems, J.

Geophys. Res. Atmos., 119,7908–7930, doi:10.1002/2013JD021316. • Saha, Suranjana, and Coauthors, 2014: The NCEP Climate Forecast System Version 2. J. Climate, 27, 2185–2208. doi: http://dx.doi.org/10.1175/JCLI-D-12-00823.1. • Tian, D., et al, 2014: Statistical downscaling of multi-model forecast for seasonal precipitation and surface temperature in the southeast US. J. Climate (in presss). • Vecchi, G.A., T. Delworth, R. Gudgel, S. Kapnick, A. Rosati, A.T. Wittenberg, F. Zeng, W. Anderson, V. Balaji, K. Dixon, L. Jia, H.-S. Kim, L. Krishnamurthy, R.

Msadek, W.F. Stern, S.D. Underwood, G. Villarini, X. Yang, S. Zhang (2014): On the Seasonal Forecasting to Regional Tropical Cyclone Activity. J. Climate (in press). doi:10.1175/JCLI-D-14-00158.1.

• Vernieres, G., C. Keppenne, M.M. Rienecker, J. Jacob, and R. Kovach, 2012: The GEOS-ODAS, description and evaluation. NASA Technical Report Series on Global Modeling and Data Assimilation, NASA/ TM–2012–104606, Vol. 30.

• Sikder, S., F. Hossain, J. B. Roberts, F. R. Robertson, C. K. Shum, F., J. Turk, 2014: How Skillful is Precipitation Forecast from General Circulation Models for Making Seasonal Water Management Decisions in South Asia, J. Hydromet., in review.

• Wang, H., S. Schubert, R. Koster, Y.-G. Ham, and M. Suarez, 2014: On the Role of SST Forcing in the 2011 and 2012 Extreme U.S. Heat and Drought: A Study in Contrasts. J. Hydro. Meteor., 15, 1255-1273, 2014.

• Yang, X. and coauthors (2014): Seasonal predictability of extratropical storm tracks in GFDL's high-resolution climate prediction model. J. Climate (submitted). • Yuan, X., and E. F. Wood (2013), Multimodel seasonal forecasting of global drought onset, Geophys. Res. Lett., 40, 4900–4905, doi:10.1002/grl.50949. • Yuan, X., and E. F. Wood (2012), On the clustering of climate models in ensemble seasonal forecasting, Geophys. Res. Lett., 39, L18701, doi:10.1029/2012GL052735.

NMME Next Steps

• Sustain/operationalize the experimental NMME seasonal system in FY15 • A transition plan has been developed, following the NMME Post-

Project Review and discussions with NCEP and CPO managemeent

• Community efforts to analyze the NMME Phase-II data • A FY15 call proposals from CPO/MAPP also involving DOE and Navy/ONR

• NMME Sub-seasonal Workshop (~ March 2015) • To identify and respond to user needs and operational requirements for weeks

3 – 4 forecast; • To discuss scientific opportunities and issues (e.g., predictability, prediction

skills, forecast protocol) for an NMME Sub-seasonal Forecast System

14

• To accelerate evaluation of and improvements to the operational Climate Forecast System (CFS) and to enhance its use as a skillful tool in providing NCEP’s climate predictions and applications

CTB Priority (2): CFS Evaluation and Improvements

(1) Support R2O testing/demonstration projects • Test and evaluate new parameterizations, schemes, model

components in NCEP operational models

(2) Engage the external community in planning for CFSv3

(3) Provide NCEP in-house O2R support to the external users

CTB Recent Efforts to Engage the Community in Development of next version CFS

• Organized CFSv3 Planning Meeting in August, 2011

• Organized CFSv2 Evaluation Workshop in April, 2012

• Developed a CFSv3 Vision document

• Led the publication of the Special CFSv2 Collection in Climate Dynamics (23 articles)

• Leading MAPP Climate Model Development Task Force with a focus on CFSv3 development in 2014-2016 • Synthesis of Task Force inputs to EMC’s CFSv3 spreadsheet

MAPP/CTB Grants Projects to Improve NCEP GFS/CFS

16

FY13: 1. C. Bretherton, J. Teixeira, J. Han, J-C Gola: A CPT for Improving

Turbulence and Cloud Processes in the NCEP Global Models 2. S. Krueger, S.Moorthi, R. Pincus, D. Randall, P. Bogenschutz: A

CPT for improving turbulence and cloud processes in the NCEP global models:

FY14: 1. F. Chen, M. Barlage, ZL Yang, M. Ek, R. Yang, J.

Meng: Improving the NCEP Climate Forecast System (CFS) through Enhancing the Representation of Soil-Hydrology-Vegetation Interactions.

2. J. Jin, M. Ek, Y. Wu: Advances in Lake-Effect Process Prediction within NOAA’s Climate Forecast System for North America.

3. S. Lu, YT Hou, A. Silva, Jr., S. Moorthi, F. Yang, Q. Min, A. Darmenov, D. Barahona: Improving Cloud Microphysics and Their Interactions with Aerosols in the NCEP Global Models.

CTB Priority (3): Improving Climate Forecast Tools and Products

Enhancing operational drought monitoring and prediction products (Lettenmaier, Wood and Mo)

Evaporative Stress Index (Anderson)

Goal: To provide reliable climate forecast products that are responsive to the needs of users and incorporate state-of-the-art science and research

Note: The CTB effort in products development has been focused on drought due to the link to the Drought/NIDIS budget. CTB hopes to expand to broader areas to address NOAA’s societal challenges.

CPC Real-Time NMME SPI Forecasts: bias correction and spatial downscaling

(18

Available at http://www.cpc.ncep.noaa.gov/products/Drought/Monitoring/spi_outlooks_3.shtml (After Mo and Li)

19

New FY14 MAPP/CTB Projects on Improving Prediction Methods/Tools

1. Anthony G. Barnston, Huug van den Dool, Emily J. Becker, Michael K. Tippett, Shuhua Li: Improved probabilistic forecast products for the NMME seasonal forecast system.

2. Timothy DelSole, Michael K. Tippett, Kathleen Pegion, Arun Kumar: Subseasonal NMME Forecasts: Skill, Predictability, and Multi-model combinations.

3. Michael K. Tippett, Gregory Carbin, Jon Gottschalck: Assessment of CFS predictions of U.S. severe weather activity.

4. Shang-Ping Xie, Nathaniel Johnson, Steven Feldstein, Michelle L’Heureux, Stephen Baxter: Bridging the gap in NOAA’s extended and long range prediction systems through the development of new forecast products for weeks 3 and 4.

Assessing Drought Monitoring and Prediction Capabilities

CTB is leading the MAPP Drought Task Force Research to Capability (RtC) Assessment Effort.

Key predictand (s) for drought variable (e.g., P, T, soil moisture, streamflow)

Metric(s) and skill scores comparing

Onset and recovery of drought condition

Lead time of prediction Error of identification

Duration and severity of drought condition

Error, bias, correlation (time, value)

Indication (detection, prediction) of drought condition: deterministic

Categorical metrics: Critical Success Index (CSI), Equitable Threat Score (ETC) Probability of Detection (POD), False Alarm Rate (FAR), and others.

Probability of drought condition: probabilistic

Brier Skill Score (binary); secondarily, Brier decompositions for reliability and resolution

Value, overall Value given drought occurring in the observed or forecast period

1. Error, bias, correlation (of ensemble mean or median for probabilistic)

2. Ranked Probability Score (CRPS)

Development of MAPP Drought Assessment Protocol • Guidance for CPO/MAPP

PIs to address the benefits of research

• Assessment metrics for drought monitor and forecast

• Verification data • Verification period and

case studies • Baselines and

benchmarking

21

Summary CTB Major Activities and accomplishments

• Conducting R2O testing projects with grants funding from CPO/MAPP

• Leading the development, planning, and implementation of NMME.

• Working with NWS and CPO to improve the NOAA’s R2O transition process.

• Engaging the external community to contribute to NCEP CFSv3 development

• Leading the MAPP Drought Task Force Research to Capability (RtC) Assessment Effort.

The CTB goal is i) to accelerate research-to-operation (R2O) transition to improve NCEP operational climate prediction and products, and ii) to provide O2R support to the external community