www.ifremer.fr

Marine Data Representation systems

T. Loubrieu, G. Maudire, S. Pouliquen, T. Carval, IFREMER,

E. Charpentier, WMO

Scott D. Woodruff, NOAA

on behalf of JCOMM

www.ifremer.fr

Background

Involved in : • European Operational Oceanography

(Mersea/MyOcean, ECOOP) information management system design.

• CORIOLIS operational data management system ocean in-situ observation.

• SeaDataNet, marine data management pan-european framework.

www.ifremer.fr

Data Representation systems• Marine Observation networks (ARGO, OceanSITES, GOSUD,

VOS, GLOSS, GTSPP, Drifting buoys, GRHSST…)

• Integrated systems for marine monitoring and forecasting (Mersea/MyOcean, ECOOP, …)

• Overall framework initiatives for marine data management (SeadataNet, UNIDATA/CDM, CSML).

Current Status regarding GTS• Most of the Marine data are being exchanged in real-time

through the GTS using the recommended formats (SHIP, BUOY, BATHY, TESAC, TRACKOB, WAVEOB, BUFR and CREX)

• Migration to table driven codes has been engaged especially with regards to buoy, Argo, XBT (BUFR) and sea level data (CREX). Works remains for the VOS (BUFR).

www.ifremer.fr

ARGO overview (1/2) More than 3000 profiling floats Measure temperature, salinity, dissolved oxygen. Each delivers every 10 days one vertical profile

from 2000 meters depth to surface 6 regional data assembly centers + 2 global data

centers Harmonized quality assesement procedures, near

real time and delayed mode delivery. Based on ftp repositories and harmonized netCDF

files for data and metadata.

www.ifremer.fr

ARGO overview (2/2)

GOSUD, OceanSITE are

extensions of the DRS

to thermosalinographs

and moorings datasets.

Drifting buoys data are

also managed at ISDM

and CORIOLIS data

center.

www.ifremer.fr

Voluntary Observation Ships

overview

Observation from volontary ships

Transmitted in real time (F-13 SHIP format).

Transmitted in delayed mode (IMMT file format) to

Global Collecting Centers and archived at

ICOADS (IMMA file format).

Data analysis, climate studies provided in

netCDF/CF files.

www.ifremer.fr

Global Temperature and Salinity

Profile Program (GTSPP)

Provide a timely and complete data and

information base of ocean temperature and salinity

profile data of known and documented quality.

www.ifremer.fr

Global Sea Level Observing System

(GLOSS)

A 'Global Core Network' (GCN) of 290 sea level stations around the world for long term climate change and oceanographic sea level monitoring.

A permanent service managing delayed mode data (PSMSL) : ASCII files.

A station directory and link to regional data management facility (non-harmonized).

A fast-mode delivery service (OPeNDAP and ASCII files).

www.ifremer.fr

GHRSST overview

Remote sensing observations of sea surface temperature (SST)

Harmonized quality assesment and file dissemination (L2 products) in real time through data centers (ftp/http).

Analysis, gridded products mixing datasets from different satellites (L4).

Central facilities for indexing the datasets (MMR) and archiving and reanalysing them (LTSRF).

www.ifremer.fr

Mersea/MyOcean overview

Monitors and forecasts the global and regional ocean. Provides a service for application providers (oil spill monitoring, ship routing, …) and policy makers.

Involves 5 modeling and forecasting centers and 3 thematic assembly centers.

Information flows are federated through standards back-end services and central facilities (catalogue, viewing portal, system monitoring)

www.ifremer.fr

SeaDataNet overview

An network infrastructure for managing the large

and diverse data sets collected by the

oceanographic fleets and the automatic

observation systems.

Sets up reference services (vocabularies,

catalogues, users directory…) before setting up a

federating infrastructure for European marine

observation data sharing.

www.ifremer.fr

Suitability criteria

Content

Pictorial display

Operation activity loging (warning, …)

Operational support (real time, …)

Transmission of information to users

Storage of information within systems

Compliance with standards

Metadata availability

Translation of data representation

www.ifremer.fr

Content

For observation networks : datasets are gathered in each DRS according to :– Observing platform/sensor type (profiling floats, moorings, tide

gauges… remote sensing/SST).

– Delivery requirements (quality assesment / delay).

(SeaDataNet focuses on infrastructure, historically deals with marine in-situ delayed mode data)

For integrated systems : datasets are gathered in one DRS according to :– One service requirement (e.g. monitoring and forecasting ocean)

– The processing chain is integrated within the DRS (from observation sub-DRS to modelling and forecasting sub-DRS).

www.ifremer.fr

Pictorial display

Most of time specific GIS web portal or pre-

processed quicklooks with adavanced web

interfaces.

More and more GIS web portal based on

OGC/WMS back-end services and generic web

portal interface (openlayer, mapbuilder).

www.ifremer.fr

Operation activity log

Up to now, very few standardized exchanges for operational activity and results, for example :– Grey list for profiling floats status (ARGO).

– Textual information on web site (e.g. GTSPP, pressure warning)

– Automatic mail report for activity (end of process, status of process) within a production unit (at CORIOLIS).

Standardized monitoring engines, on-going development :– Automatic check of data availability (according to SLA

requirements) : NAGIOS tools verify in real time OPeNDAP services for Mersea/MyOcean.

– Later on, standardized report for activity of MyOcean production units to Service Desk (e-mail or RSS).

www.ifremer.fr

Information dissemination (1/2)

Usually functions divided into :– Discovery

– Viewing

– Download

Discovery disseminates products catalogues or

dataset inventories with ISO19115 and THREDDS

catalogues. CSW is likely to be used for

requesting these catalogues (to be done).

www.ifremer.fr

Information dissemination (2/2) Viewing :

– pre-processed quicklook can be browsed in specific web interfaces

– OGC/WMS is being more widely used for map quicklooks.

– No standard way of dealing with non-map view (vertical profiles, time series, vertical cross section, …)

Download :– Most of time, ftp server with netCDF/CF files with specific extra-convention for

file format (ARGO, OceanSite, …, Mersea/MyOcean)

– ODV ASCII file format for SeaDataNet

– MEDS-ASCII or various ASCII flavors for time series, profiles, …

– IMMT/IMMA binary file format for VOS (ICOADS)

– OPeNDAP interfaces for dynamic subsetting (using THREDDS Data Server) : Mersea/MyOcean.

– OPeNDAP/Dapper interface for profile data at CORIOLIS (DAP4COR).

www.ifremer.fr

Storage facility

Under responsability of production unit (with

Service Level Agreement to be defined so that a

subset of archive can be restored on user

demand).

Or under responsability of NODC (National Ocean

Data Centers)

For GHRSST, GLOSS, ARGO, …, a central

archive and reanalysis facility.

www.ifremer.fr

Standard compliance Metadata : XML/ISO19115 widely used with different profiles depending on

requirements (SeaDataNet, Mersea/MyOcean). THREDDS used for simple metadata index (Mersea/MyOcean).

Images : OGC/WMS (not extended for non-map representation)

Download :

– netCDF file format with CF convention.

– OPeNDAP interfaces (supported and checked by information management team in Mersea)

– OGC/WFS and OGC/WCS emerging (not used yet).

(requires further standardization/convention for non gridded datasets)

References, background (to be used) :

– vocularies provided by SeaDataNet

– guidelines for usage of metadata standard for marine datasets.

– CDM / CSML provide federated data model for ocean/atmosphere dataset.

www.ifremer.fr

Metadata availability

Harmonized metadata available within each project :

By products (SeaDataNet/EDMED, Mersea

catalogue)

By observing platform, numerical model

(SeaDataNet/EDIOS, ARGO/Metadata files, …)

By dataset (SeaDataNet/CDI, Mersea inventory)

www.ifremer.fr

Translation of data representation

To be done :

Rely on standardized data model for

ocean/atmosphere datasets (CDM or CSML) : to be

set up by each production unit.

Rely on external authorities to translate references :– Parameters names with vocabulary services.

– Geographical services for changing map projections or

geospatial reference datum.

www.ifremer.fr

Practical experiment : SeaDataNet

Vocabularies web services : – ontology available on-line. Parameters names,

categories…

– similar to GCMD vocabularies + deprecated flag instead

of complete removal of terms.

– Identification of terms with URN (within ‘sdn’ namespace)

XML/ISO19115 extension :– Usage of URN within the XML metadata description for

embedded controlled vocabularies.

www.ifremer.fr

Practical experiment : download

from Mersea

Download integrated

service architecture :

www.ifremer.fr

Practical experiment : download

from Mersea (2/2)

Real time monitoring of the distributed system for improving the

reliability (status : ok, late delivery, down) : from product and

service catalogue, configuration of NAGIOS monitoring tool.

www.ifremer.fr

JCOMM starting or on-going

activities related to Marine DRS

JCOMM and IODE have engaged in the « Standard

process » (ontologies, temporal and geospatial references,

platforms, quality control…). One meeting has been held in

Ostend, Jan. 2008.

JCOMM Pilot project for WIGOS (2008-2010) :– The Pilot Project will aim at making the appropriate identified data sets

interoperable with the wider WMO and IOC communities. It will develop

and agree on consistent standards to be used across the community (at

metadata and data level).

– Guidelines are expected from ET-ADRS.