Preparing CMOR for CMIP6 and other WCRP Projects

Charles Doutriaux and Karl E. Taylor

Presented at the 4th Annual ESGF & UV-CDAT Meeting

Livermore, CA11 December 2014

ESGF F2F 11 December 2014

K. E. Taylor PCMDI

Outline

• What is CMOR (the Climate Model Output Rewriter)?

• What changes are planned?

ESGF F2F 11 December 2014

K. E. Taylor PCMDI

Why CMOR?

• PCMDI and now WCRP advocate adoption of common standards/conventions for the:

Structure and format of climate data Metadata used to describe climate data Vocabulary used for categorizing the diversity of model output

• Standardization enables/facilitates

Automation in the preparation of model output Analysis by researchers using uniform methods for reading and

interpreting data unique identification of files sharing of data across the ESGF network

• But model output stored in native formats following a diversity of approaches must be “translated” into the standard form

• CMOR was written to facilitate this.

ESGF F2F 11 December 2014

K. E. Taylor PCMDI

What is CMOR?

• CMOR is a library of C functions which facilitate/enforce compliance with MIP requirements.

Includes interface to both Fortran and python code

• CMOR was designed to be adapted to the different metadata requirements of each “model intercomparison project” (e.g., CFMIP, HTAP, PMIP)

• Most of the CMIP3 and CMIP5 output served by ESGF was processed through CMOR.

CMOR has been key to ensuring uniformity across all models

ESGF F2F 11 December 2014

K. E. Taylor PCMDI

Project-specific CMOR tables facilitate and ensure consistency of model output from contributing groups

• Proper specification of coordinate and variable attributes, including:

Correct standard names and units Use of a common time coordinate model (with flexibility) Required dimensions (but with different grids allowed)

• A capability to

Reorder axis order Reverse axis direction (or translate longitude dimension) Convert units (through udunits)

ESGF F2F 11 December 2014

K. E. Taylor PCMDI

What standards does CMOR ensure compliance with?

• netCDF – (www.unidata.ucar.edu/software/netcdf/)

• CF Conventions – provides for standardized description of data contained in a file (cf-convention.github.io)

• Data Reference Syntax (DRS) – defines vocabulary used in uniquely identifying MIP datasets and specifying file and directory names (cmip-pcmdi.llnl.gov/cmip5/output_req.html).

• CMIP output requirements – specify the data structure and metadata requirements for CMIP data (cmip-

pcmdi.llnl.gov/cmip5/output_req.html)

ESGF F2F 11 December 2014

K. E. Taylor PCMDI

CMOR flags common errors, including

• Pointing out when required metadata are omitted.

• Rejecting incorrect metadata (wrong units, inadmissible attribute values, etc.)

• Rejecting inconsistent coordinate dimensions passed by user to CMOR.

• Rejecting non-monotonic coordinate values or inconsistent boundary values, as passed by user.

• Rejecting values that are clearly unrealistic (likely indicating improper units conversion or incorrect sign).

ESGF F2F 11 December 2014

K. E. Taylor PCMDI

CMOR history

• Original CMOR used in CMIP3

Written in FORTRAN 95. accommodated only data stored on cartesian longitude-latitude

grids.

• CMOR-2 used in CMIP5

CMOR’s structure and functions were largely unchanged. Written in C (with FORTRAN 95 and python interfaces) Can write output on model “native” grids that are not cartesian

longitude-latitude. Station data (needed, for example, by the HTAP project), including

metadata for station names and station locations Regional model output handled smoothly. Relies on netCDF-4 (rather than NetCDF-3), which provides for

automated compression.

ESGF F2F 11 December 2014

K. E. Taylor PCMDI

What is planned for CMOR-3?

• Generalization to better handle a wider range of model output and observational data.

CMOR was developed (and to a certain extent “hard-wired” to meet modeling needs

Some of the attributes written by CMOR don’t apply to observations (e.g., model name, experiment name)

Modifications are needed to simplify the reconfiguration of the input tables for each new MIP.

• Modularization of the CMOR input tables, separating

Project-specific metadata from Variable information

ESGF F2F 11 December 2014

K. E. Taylor PCMDI

What needs to be done for CMIP6?

• Write and release CMOR-3

• Generate new CMOR tables

Allowing additional required global attributes? (e.g., version info.) Recognizing CMIP6 controlled vocabulary Including any new requested variables.

• Provide more complete QC information to CMOR (e.g., valid max and min for each field)

• Modularize the CMOR input tables to separately specify

Project-specific information List of variables and their metadata

ESGF F2F 11 December 2014

K. E. Taylor PCMDI

For further information: https://github.com/PCMDI/cmor

IssuesDocumentation