EIONET

European Environment Information and Observation Network

http://www.eionet.eu.int/

Version 0.9 2004-11-10

* * *Quality assurance of Eurowaternet data

Presentation for the EIONET Water Workshop

Budapest, 11-12 November 2004

Hermann Peifer, Project Manager EIONET Data Flow

<hermann.peifer@eea.eu.int>

2

Quality: Models, concepts and frameworks

Concerning quality in a broader sense:

• ISO 9000: A family of International Standards for quality management http://www.iso.ch/

• TQM: Total Quality Managementsearch on google.com: 472.000 hitssearch on amazon.com: 5.485 books

• EFQM: Excellence Model of the European Foundation for Quality Management http://www.efqm.org

The word quality might have been overused in the past years.

A too generic definition bears the risk of loosing focus.

3

Data quality frameworks

Concerning data quality in the in sensu strictu:

• ESS: Quality Declaration of the European Statistical System:http://amrads.jrc.it/WPs%20pages/Quality/Documents/LEGsummary.pdf

• IMF: Data Quality Reference Site of the International Monetary Fund (IMF) http://dsbb.imf.org/Applications/web/dqrs/dqrshome/

• MIT: Total Data Quality Management Program (TDQM) http://web.mit.edu/tdqm/www/index.shtml/

• FAO‘s approach to data quality evaluation and monitoringhttp://www.fao.org/faostat/quality_en.asp

• OECD initiative on environmental data qualityhttp://www.oecd.org/

4

What is data quality?

Data quality is a measure of the degree of usefulness of the data for a specific purpose.

Data quality indicators are qualitative or quantitative descriptors of data quality.

5

Data quality management in EIONET:Too little, too late?

• Data quality management is happening in EIONET.

• Much of the QA/QC work is carried out by Topic Centres.

• The level of available QA/QC documentation is improving.

• There is a clearly a lack of visibility for this area of work.

• An overarching framework for EEA/EIONET is missing.

• Annual Management Plan 2005 1.4.2  Maintaining and quality assuring priority data flows“To provide an overview of the quality of the data provided by the member countries by documenting QA/QC processes countryby country, in order to improve the overall quality of data.”

6

7

EEA/EIONET data and information flow

Data collection

AggregationAssessment

Presentation Dissemination

Flow

Reportnet EEA/EIONET

Assessmentprocesses

Informationprocesses

Tools to serveEEA, MS, others

EEA/Users

8

Reports EE

A

NR

C N

RC

NR

C N

RC

NF

P

ET

C E

TC

Data quality: Where it matters most

ReportnetData collection

EEA/EIONETData processing

EEA/UsersData dissemination

Q1 Q2

9

Data quality dimensions

Representativeness

AccuracyPunctuality

10

Which dimension is most important?

Rel

evan

ce

Tim

elin

ess

Com

para

bili

ty

Acc

ura

cy

Acc

essi

bil

ity

Com

ple

ten

ess

Many say: Relevance

Another popular choice:Timeliness

The classical one:Accuracy

A smart answer is:All

My personal preference:...

11

Guiding thoughts on measuring data quality

• You can not measure what is not defined.

• You can not improve what can't be measured.

• Focus on understandable data quality dimensions.

• Easy to measure, preferably via simple automated analysis.

• Providing the right level of information to data providers (Q1) and end-users (Q2)

• The aim: simple, regular, quantitative and user-oriented data quality reporting

13

The aim: regular data quality reporting

0

Punctuality

Timeliness

Representativeness

Comparability

Accuracy

Completeness

Eurowaternet-Rivers Data Quality for Country X

1995

2000

2005

14

EIONETPriority Data Flows:

Annual Progress Reports 1999-2003

Annual monitoring of data deliveries and benchmarking of countries covering:

– Relevance– Punctuality– Timeliness– Completeness

15

Initial work on quality of national data

16

Scoring criteria

for Eurowaternet

data collection

2005

Response to data quality questionnaire

17

Data quality in EIONET: Summary

• Data quality in EIONET needs:– More documentation– Continuous improvement– More visibility

• Topic-specific QA/QC procedureshave to be embedded into a data quality management framework.

• The overall aim is (preferably) quantitative data quality reporting which is regularly updated in order to show trends over time.

• EIONET data quality reporting (mainly targeted at data providers, i.e. Countries) has to be complemented by data quality information targeted at end-users of the data.

EEA Data Quality Framework

QA/QCEurowater

net