Data qualitymonitoring and measurement

in ReportnetTowards meaningful machine-readable

metrics of data quality

Hermann Peifer, EEA

Eionet Priority data flows

18 hand-made data flow reports during the period 1999 – 2015 2016

?

2016

?2016?

Magic?

Data quality

Machinereadable

Meanigfulmetrics

Meaningful machine readable

data qualitymetrics:

6 out of 12= 50 % quality

???

Excursion:Mature Reportnet data flow

using AutomaticQA with BLOCKER

DF1 and DF5: Report on all major roads, railways, … (1)countryName envelopeURI checks points max. result reportingdate bonusMalus dataQuality automaticScore

Bulgaria /bg/eu/noise/df5/envvusoeq 4 12 12 100 26/06/2015 4 104 JJJCroatia /hr/eu/noise/df5/envvzeoqa 5 15 15 100 30/06/2015 0 100 JJJCzech Republic /cz/eu/noise/df5/envvyfs9q 4 12 12 100 22/06/2015 8 108 JJJCzech Republic /cz/eu/noise/df5/envvzivwq 4 12 12 100 30/06/2015 0 100 JJJEstonia /ee/eu/noise/colvzgadg/envvzgbxq 1 1 3 33 29/06/2015 1 34 JFinland /fi/eu/noise/df5/envvxkw6a 4 12 12 100 11/06/2015 19 119 JJJFrance /fr/eu/noise/colt5uwaa/coluckgla/envvzpjpa 1 3 3 100 01/07/2015 -1 99 JJJFrance /fr/eu/noise/colt5uwaa/colunr9ng/envvyqbmg 1 1 3 33 30/06/2015 0 33 JGermany /de/eu/noise/df5/envvyxh7q 44 112 132 85 29/06/2015 1 86 JJHungary /hu/eu/noise/df5/envvyq20q 4 12 12 100 25/06/2015 5 105 JJJIceland /is/eu/noise/df5/envvylw9a 2 6 6 100 15/07/2015 -15 85 JJItaly /it/eu/noise/df5/envvy1xgg 2 6 6 100 01/07/2015 -1 99 JJJItaly /it/eu/noise/df5/envvyqr9g 4 6 12 50 24/07/2015 -24 26 LLatvia /lv/eu/noise/df5/envvzkema 4 6 12 50 30/06/2015 0 50 JLithuania /lt/eu/noise/columjnqq/envvwg7wq 4 12 12 100 23/06/2015 7 107 JJJ

DF1 and DF5: Report on all major roads, railways, … (2)countryName envelopeURI checks points max. result reportingdate bonusMalus dataQuality automaticScore

Luxembourg /lu/eu/noise/df5/envvzdukg 4 12 12 100 29/06/2015 1 101 JJJLuxembourg /lu/eu/noise/df5/envvzer0w 4 12 12 100 29/06/2015 1 101 JJJNetherlands /nl/eu/noise/df5/envvzjzcg 4 8 12 67 01/07/2015 -1 66 JPoland /pl/eu/noise/df5/envvy1jww 1 3 3 100 26/06/2015 4 104 JJJPortugal /pt/eu/noise/df5/envvx7ybg 4 12 12 100 22/06/2015 8 108 JJJSlovakia /sk/eu/noise/df5/envvtefpa 1 3 3 100 24/07/2015 -24 76 JJSlovakia /sk/eu/noise/df5/envvze0bw 2 6 6 100 29/06/2015 1 101 JJJSpain /es/eu/noise/df5/envvzd8lq 4 6 12 50 29/06/2015 1 51 JSweden /se/eu/noise/df5/envvzk23g 4 12 12 100 30/06/2015 0 100 JJJSwitzerland /ch/eu/noise/df5/envvt8rbw 4 12 12 100 09/06/2015 21 121 JJJUnited Kingdom /gb/eu/noise/df5/envvxmsya 4 9 12 75 30/06/2015 0 75 JJUnited Kingdom /gb/eu/noise/df5/envvy0tzw 4 9 12 75 30/06/2015 0 75 JJUnited Kingdom /gb/eu/noise/df5/envvzenya 4 9 12 75 30/06/2015 0 75 JJUnited Kingdom /gb/eu/noise/df5/envvzff3q 4 10 12 83 30/06/2015 0 83 JJUnited Kingdom /gb/eu/noise/df5/envvzke1a 4 8 12 67 30/06/2015 0 67 JJ

ReallyMagic

?

Data quality

Machinereadable

Meanigfulmetrics

Reportnet CDR• changes in presentation layer• feedbackStatus and feedbackMessage added

END deliveries 1 Oct 2014 - 1 Oct 2015: Restart of AutomaticQA250 envelopes delivered and released-30 envelopes (*.pdf, *.doc,*.shp only)-20 envelopes (very! old templates)

---------200 envelopes: AutomaticQA restarted

133 WARNING: COULD BE wrong122 ERROR: This IS wrong432 INFO: No error found

1. Translate into 1-3 points2. Calculate percentage3. Bonus/Malus4. Data quality (%)5. Automatic Score

Reportnet‘sAutomatic

Data checks

?