Naming, Storage, and Representation of WRF

Metadata Attributes in myLEAD

Sangmi Lee Pallickara, Scott Jensen and Beth Plale

Indiana university

Thanks to Suresh Marru, Dan Weber, and Kevin Thomas for their effort and time in discussions about the input parameters for the model run

Storing Metadata: how and when

• An XML document containing metadata for an experiment (e.g., WRF parameters) is delivered to the myLEAD server, parsed, and the metadata is stored in a relational database.

• The actual setup files, data files, and derived result files are stored in a file system (SRB). Only metadata about the files is stored in myLEAD.

• As users create experiments through the portal interface, information about the experiment is delivered to myLEAD at each step (e.g. which project, which experiment, which workflow template, etc.).

• As the workflow runs, application-specific metadata is delivered to myLEAD, (e.g. which workflow template, instance, WRF parameters, etc.) as well as metadata regarding each file used or generated.

Data Model: how metadata is stored

• Collection table– Projects/experiments/collections are stored in

the collection table. The hierarchical Parent/Child relationships are recorded in the Lead Parent table.

• Attributes– The metadata about collections and files are

stored as attributes under the particular collection or file.

– Static/dynamic attributes.

Example• Create a new project : Project 2005• Create a new experiment : ExpAugust2005• Add a workflow template (template1) from existing workflow

templates• Add a collection input_Data

Project2005

ExpAugust2005

Project2005ExpAugust2005

template1

View from myWorkspace portal In backend RDBMS (simplified)

Collection attributes

template1

Input_Data

My Workspace

Lead parent

Attribute Definition

Collection Blob ElemActual data of template1

Collections

Input_Data

Parent-Child Relationship

Parent IDParent Type

Parent Level Child ID

Child Type

Child Level

1 7 1 2 6 2

Collection AttributesCol ID Attr ID

2 12 2

Attribute Definitions

Attr ID Name1 Run Name2 Grid Setup

Collection String ElementsCol ID Attr ID Elem ID Element Value

2 1 1 Run001

Collections

Col IDCreator

IDCol

Type Name

1 2 7 MyProject2 2 6 My WRF Run

Collection Float ElementsCol ID Attr ID Elem ID Element Value

2 2 2 1000.002 2 3 500.002 2 4 36.002 2 5 -100.00

Element Definitions

Elem ID Attr IDElem Type Name

1 1 6 Run Name2 2 5 Horizontal Grid Space3 2 5 Vertical Grid Space4 2 5 Latitude of Domain Center5 2 5 Longitude of Domain Center

Type 5is Float

Type 6is String

Project: MyProjectExperiment: My WRF Run

Attribute: Run NameElement: Run Name

Attribute: Grid SetupElement: Horizontal Grid SpaceElement: Vertical Grid SpaceElement: Latitude of Domain CenterElement: Longitude of Domain Center

Note: Elements within an attribute can be of different types.

UsersCreator

ID DN Name1 /C=US/O=National Center for Supercomputing Applications/CN=Kelvin DroegemeierKelvin Droegemeier2 /C=US/O=National Center for Supercomputing Applications/CN=Dan WeberDan Weber3 /C=US/O=National Center for Supercomputing Applications/CN=Suresh MarruSuresh Marru

Example – Adding Input Parameters as attributes

What’s the purpose of attributes?• We can’t always remember the name of a file we used

last year. Likewise, we may not remember the name of an experiment that we ran last year or even last month.

• Attributes allow us to find experiments, files, or collections of files based on properties of the data, model, or derived results.

• Attributes can be recorded for any level in the structure of an experiment. Files, collections, experiments, and even projects can have attributes.

• Attributes provide the key to query functionality.– Files can be searched by the attributes and their values– Experiments can be searched by attributes and their values

• Attributes provide better sharing of data, models, and results.

Search Experiment by the model parameters(1)

Search Experiment by the input parameters(2)

Storing WRF input parameters as attributes in myLEAD

• WRF/ARPS input parameters are stored as attributes of experiment.

• Users can search for experiment based on input/model parameters (model run name, Grid etc.).

• Currently we have added the namelist parameters that the MET group identified as the most common, but Kevin is looking at the ARPS namelist file to determine additional parameters that should be recorded.

ARPS & WRF input parameters

• Weather Models use FORTRAN NAMELIST to specify input to the Fortran applications and change the way they execute without having to recompile.

Catagorizing input parameters*

• Category I: Cross cutting - Mandatory user editable -

• Category II: File Handling - Mandatory workflow editable

• Category III: Service specific - Optional (can be defaulted for most cases.) This number changes for different group of users

• Category IV: MPI Parameters - workflow editable• Category V: Never Changing parameters

* Thanks to Suresh

Category I- Description

• Every User has to enter these values at the start of the experiment.

• These input parameters are common among most of the services of ARPS and WRF systems.

• These parameters have to be maintained consisting in all the services.

• These parameters have to parsed by the portal as they determine the magnitude of resources needed.

Category I - Sub-Categories

• Model Run Name

• Model Dimension Parameters

• Model Grid Setup Parameters

• Model Initialization Parameters

• Map Projection Parameters

• Time Integration Control Parameters

Category I Variables List(for quick reference the namelist block which contains these variables in the of arps.input file is mentioned, all these variables are explained

in the following slides)

• runname - &jobname

• nx, ny, nz, - &grid_dims

• dx,dy,dz, ctrlat, ctrlon - &grid

• initime - &initialization

Model Run Name

• Variable runname - this has to be obtained from user and first 6 characters have to be prefixed to intermediate and output logical file names

Model Dimension Parameters

• Number of Grid PointsIdentifies the physical dimension of the model domain Variables nx, ny & nznx and ny are used in inputs to arpstrn, wrfsi, ext2arps, 88d2arps, nids2arps, mci2arps, adas, arps2wrf, wrf, wrf2arps, arpsplt & arpsverifnz used in ext2arps, 88d2arps, nids2arps, adas, arps2wrf, wrf, wrf2arps, arpsplt

(nx and ny determine the number of processors needed and how they need to be split)

Number of Grid Points in X, Y direction

• nx, ny- grid points in Y and Y direction( This input has to be collected from user and

+3 has to be added when setting nx and ny in adas input files and +1 when setting to wrf input files, because over the forecast grid, arps system uses 3 fake points and wrf uses 1 fake point)

Number of Grid Points in Z direction

• nz- grid points in z direction( No fake points need to be added to nz and

also nz need not considered for determining number of processors in mpi runs)

Model Grid Setup Parameters

• Identifies the center of the model domain and grid spacing spacing in x, y and z directions.

• Variables dx, dy, dz, ctrlat, ctrlon • dx and dy must be set in arpstrn, wrfsi, ext2arps,

88d2arps, nids2arps, mci2arps, adas, arps2wrf, wrf & wrf2arps

• dz must be set in ext2arps, 88d2arps, nids2arps, adas, arps2wrf, wrf, & wrf2arps

• ctrlat and ctrlon must be set in arpstrn, wrfsi, ext2arps, 88d2arps, nids2arps, mci2arps

Grid Spacing

• dx, dy, dy denote the grid spacing in X, Y and Z directions respectively.

• dx, dy and dz are specified in meters.

• dx and dy should always be the same

• dz should only be exposed to advanced users, for others this should be set to a value based on nz.

Model Initialization Parameters• initime - the model initilization time

specified as character string in the UTC format - 'yyyy-mn-dd.hh:mm:ss'