WP3:

‘Rapid data delivery at 4 NDACC stations’

Partners:BIRA-IASBINTAUniversitaet BernKarlsruher Institut fuer TechnologieCNRSUniversitaet BremenUniversité de LiègeScience and Technology B.V.

NORS/NDACC UV-VIS meeting, Brussels, 3-4 July 2012 1

WP3 objectives• To implement procedures for operational delivery of

NDACC data to the NORS data server with a delay of maximum 1 month after data acquisition

• Use of a common HDF format compliant with GEOMS (Generic Earth Observation Metadata Standard) guidelines

• Data and stations:

NORS/NDACC UV-VIS meeting, Brussels, 3-4 July 2012 2

WP3 tasks

NORS/NDACC UV-VIS meeting, Brussels, 3-4 July 2012 3

For all WP3 participants:

• Task 3.1: Development of software routines to save data in the defined HDF format compliant with GEOMS

• Task 3.2: Implementation of automatic data processing systems generating the data products, including a QA/QC validation step

• Task 3.3: Set up of data delivery systems providing the data products within 1 month to the NORS data center (MS3, Month 6: April 2012)

• Task 3.4: Test runs and demonstration of operational data delivery system

• Task 3.5: Optimisation of data delivery system

• Task 3.6: Routine data delivery to the NORS data center (D3.1, Month 21: July 2013)

Rapid data delivery system

NORS/NDACC UV-VIS meeting, Brussels, 3-4 July 2012 4

IASB-BIRA rapid data delivery system (UV-VIS data):

Absorption spectra at

station

ftpAbsorption spectra on BIRA-IASB

server

QDOAS analysis DSCDs

OEM profiling

Vertical profile +information content

+error budget

Vertical columns +information content

+error budget

column retrieval

AMF LUTs

AVK LUTs

HDF files

ftp

HDF files on NORS data

server

QA/QC

QA/QC

HDF/GEOMS data format

NORS/NDACC UV-VIS meeting, Brussels, 3-4 July 2012 5

Format description and tools for HDF files creation are available on AVDC at:

http://avdc.gsfc.nasa.gov/index.php?site=1876901039

Important for the NDACC UV-vis WG to move to this data format

Quality Assurance/Quality Control• Regarding the HDF files content and format, verifications

are performed by the idlcr8hdf.sav IDL program, e.g.:

− Variable values outside the specified range

− Uncorrect fill-in values

− Inconsistencies in the time variable (DATETIME)

….

• Additional quality checks to implement (to be discussed):

− Profile retrievals based on OEM: remove retrievals with low DOFS and/or large residuals

− For stratospheric NO2 columns, remove data contaminated by tropospheric pollution (Langley-plot technique)

….NORS/NDACC UV-VIS meeting, Brussels, 3-4 July 2012 6

Planning of rapid data delivery• Month 6 (April 2012): for each partner, set up and

demonstration of their rapid data delivery system

• Months 7-20 (May 2012-June 2013): Optimisation of data delivery systems

• Month 21 (July 2013): Operational data delivery

• Month 22 (August 2013): Preliminary documentation of data delivery systems

• Month 33 (July 2014): Final documentation of data delivery systems

NORS/NDACC UV-VIS meeting, Brussels, 3-4 July 2012 7

After NORS, implementation of a similar data delivery system for all the NDACC UV-vis stations ?