1

WISE SoE TCM data reporting -data processing and QC issues

Event: 2013 Marine & Coastal Eionet Workshop, 24-25 SeptemberAuthor: Jørgen Nørrevang Jensen, ICES

2

What is QA and QC?

Event: 2013 Marine & Coastal Eionet Workshop, 24-25 SeptemberAuthor: Jørgen Nørrevang Jensen, ICES

Quality Assurance

An overallmanagement plan to guarantee theintegrity of data(The “system”)

Quality Control/check

A series of analytical measurements usedto assess thequality of the analytical data(The “tools”)

3

Sources of errors/noise in TCM data

Event: 2013 Marine & Coastal Eionet Workshop, 24-25 SeptemberAuthor: Jørgen Nørrevang Jensen, ICES

Sampling strategy

Main sources of “errors”

Spatial and temporal distribution -Autocorrelation

SampleRepresentative?Use of additional inform.

Chemicalanalysis

Analytical methodAnalytical procedures

Data entry

Datatransfer

Entry errorsLinking of fields

Unit conversions

Size of noise/error

Large

Medium-large

Medium

Small-Medium

Small-medium

QA

QC

4

Deadlines for data submissions

Hazardous substances in biota, sediment and seawater

Oceanographic data (nutrients, chlorophyll, S, T, etc.)

HELCOM 1 September 1 May

OSPAR 1 September 1 September

Eionet 31 October 31 October

Event: 2013 Marine & Coastal Eionet Workshop, 24-25 September Author: Jørgen Nørrevang Jensen

5Event: 2013 Marine & Coastal Eionet Workshop, 24-25 September Author: Jørgen Nørrevang Jensen

Eionet CDRCountry folders

TCM ETC template views

ICES DOME Data on contaminants in biota,

sediment and seawater

TCM data

Collective upload from this year

QC marking andduplicate checking by ETC

QC and feed-back by ETC

Regional/national databases

Overview of the data flow for the TCM SoER dataNovember

ICES TCM

April

ICES Oceano-graphic database

QC screening of data

QC andfeedback by EEA

QC and feed-back by ICES

6

ICES data flow overviewData from OSPAR, HELCOM, AMAP, etc.

Event: 2013 Marine & Coastal Eionet Workshop, 24-25 SeptemberAuthor: Jørgen Nørrevang Jensen, ICES

Contaminantsin biota, sedimentsand seawater

DOME

QC screening of data Data transformation to

ICES format

ICES Oceanographic Database

Oceanographicdata (nutrients, chlorophyll, T, S, etc.)

Fixed format“Free” format

QC screening of data

ETC DD structure

7Event: 2013 Marine & Coastal Eionet Workshop, 24-25 SeptemberAuthor: Jørgen Nørrevang Jensen, ICES

TCM priority data – main QC issues

QC issue Main problem Why is this a problem?

Station issues Stations missing Data useless without station information

Duplicate stations Destroys time series in CSI

Duplicate checking True duplicates can be difficult to find

Important to remove true duplicates in data for the use in CSI

False duplicates occurs due to missing primary key fields (e.g. SampleID and SampleDepth)

Data “degradation” due to aggregation

EEZ Country code versus Data submitter

Country Code might get wrong due to inaccurate positions or EEZ layer

Determinand names Many different synonyms for the determinands

Determinand names have to be unified for the use in CSI

Countries not submitting data

Some countries do not provide data

Spatial coverage needs to be improved for the use in CSI

8

TCM Station ID’s – a big mess

Event: 2013 Marine & Coastal Eionet Workshop, 24-25 SeptemberAuthor: Jørgen Nørrevang Jensen, ICES

Station identification can be difficult. Comparisons of stations within and between station tables have high-lighted these problems:

• Duplicates most likely occurs with the Eionet station table. These are not always easy to confirm due to slightly different coordinates and ID’s• Eionet stations missing – some data refers to a station that is not reported in the station table• The same stations most likely occurs in several station lists (Eionet, WFD, and the ICES station dictionary). As above the coordinates and ID’s are often slightly different• Eionet stations that “refers” to a WFD’s station is not always found in the WFD’s station list

9

KML files with WFD, Eionet, and ICES Station dictionary stations

Event: 2013 Marine & Coastal Eionet Workshop, 24-25 SeptemberAuthor: Jørgen Nørrevang Jensen, ICES

10Event: 2013 Marine & Coastal Eionet Workshop, 24-25 SeptemberAuthor: Jørgen Nørrevang Jensen, ICES

KML files with WFD, Eionet, and ICES Station dictionary stations