® sponsored by geoviqua user feedback on data quality 90th ogc technical committee washington, dc...

®

Sponsored by

GeoViQuaGeoViQuaUser Feedback on Data QualityUser Feedback on Data Quality

90th OGC Technical Committee

Washington, DC

Joan Masó

25th March 2014

Copyright © 2014 Open Geospatial ConsortiumCopyright © 2014 Open Geospatial Consortium

OGC®

User feedback in GeoViQuaUser feedback in GeoViQua

GeoViQua provides a set of scientifically developed software components and services that facilitate the creation, search and visualization of quality information on EO data integrated and validated in the GEOSS Common Infrastructure.

Pilot case studies

CC RR OO SS SS

SS BB AA

Communitybuilding

GEO S&T Label

2

OGC®

GeoViQua (265178)GeoViQua (265178)

FP7 Collaborative ProjectEnvironment

10 partners with CREAF coordinating

Duration: 36 months1 February 2011 – 31 January 2014

Budget: 4.024.256,42 €EC Contribution: 3.266.803,98 €

2 reporting periods: 1.Months 1-18

2.Months 19-36

QUAlity aware

VIsualisation for the

Global Earth Observation

system of systems

3

OGC®

Elements that are not in the GEOPortalElements that are not in the GEOPortal

• Documentation phase– No quality focused metadata model. No pixel level quality.

• Discovery phase:– Query

• It is not possible to use a quality indicator as a way to refine the search

• No user feedback consideration– Response

• Quality is not a criteria for ranking results• Quality is not emphasize in the results• Metadata can not be easily compared• N o visual representation of the metadata completeness

• Evaluation phase– No pixel level quality visualization strategies

• Access phase– No data access with pixel level quality

• Interaction phase– No immediate user feedback

4

OGC®

Embedding quality in all phases Embedding quality in all phases of the geospatial data exploitationof the geospatial data exploitation

DAB-Q:Query with

quality information

DAB-Q:Query with

quality information

WMS-Qextension to relate

data with spatialized

quality layers

WMS-Qextension to relate

data with spatialized

quality layers

GEOLabelGraphical

picture that helps to evaluate

data

GEOLabelGraphical

picture that helps to evaluate

data

User feedback

system has been

introduced

User feedback

system has been

introduced

WCS-Q:GMLCov

extension for spatialized

quality layers (in project)

WCS-Q:GMLCov

extension for spatialized

quality layers (in project)

5

GeoViQua Producer and User Quality Model,

UncertML & QualityML

GeoViQua Producer and User Quality Model,

UncertML & QualityML

OGC®

Assessing fitness-for purposeAssessing fitness-for purpose• peer and expert review• better traceability and provenance information• information on citations and usage of a dataset• warnings about problems identified with a dataset and

potential workarounds, • ‘soft knowledge’ from data producers (e.g.

recommendations for use which are not easily encoded using the existing standards)

Chrisman, 1988: asked for ‘user experience’ to be included in data quality specsEpstein et al 1998: disclaimers only cover ‘reasonably foreseeable use’Comber et al., 2006: identify mismatches between producer/user ontologiesDevillers et al., 2007: tools needed to help users understand quality information.Boin & Hunter, 2007: users want a simple summary that they can interrogate for more detail as appropriate.Lush et al., 2011: Interviews identified that these needs are still not being met.

6

OGC®

7

OGC®

8

Reference datasets

8

OGC®

George Mason University Global Agricultural George Mason University Global Agricultural Drought Monitoring and Forecasting System Drought Monitoring and Forecasting System

GEO Week. AIP6. GenevaJanuary 13, 2014

OGC®

User feedback inputUser feedback input

GEO Week. AIP6. GenevaJanuary 13, 2014

OGC®

We needed to define a new model...We needed to define a new model...

Copyright © 2014 Open Geospatial Consortium

OGC®

...that uses known concepts......that uses known concepts...

• Rating• Comment• Usage Report• Quality Overwrite


OGC®

……in an aggregation model…in an aggregation model…

• Based in feedback items that are related to “targets”– Primary– Secondary– Supplementary

– Data – Metadata

• with the idea of feedback collection


OGC®

……and the summary capacity.and the summary capacity.

• This a dynamic element calculated during a query


OGC®

– Mandatory information on user role (application domain, expertise level etc).

– Subject, application domain and keyword tags

– Other optional information such as rating, comments, a quality override (superseding producer quality information), a report of usage or a citation.

– Focus -spatial, temporal or other subsets to which feedback pertains

15

User feedbackUser feedback

15

OGC®

16

Populating the user modelPopulating the user model

16

OGC®

• Feedback Catalog integration– Discover of user feedback items to the Feedback Catalog– Upload of user feedback items to the Feedback Catalog– Merging of discovered metadata records with related user

feedback items • Support to quality-related constraints

– Producer Quality Model constraints (e.g.: misclassification rate, count of data quality info elements, etc…)

– User Quality Model constraints (e.g.: rating score, user domain, etc…)

• WMS-Q integration• Insertion of GEO label link to all discovered records

DAB-Q capabilities

17

OGC®

CSW-Q extensionCSW-Q extension

18

OGC®

Components diagramComponents diagram

19

OGC®

Portal integrationPortal integration


OGC®

EncodingEncoding

<?xml version="1.0" encoding="utf-8"?><gvq:GVQ_FeedbackCollection xmlns:gvq="http://www.geoviqua.org/QualityInformationModel/4.0"

xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gmd19157="http://www.geoviqua.org/gmd19157" xmlns:updated19115="http://www.geoviqua.org/19115_updates" xmlns:gsrC="http://www.w3.org/2001/XMLSchema" xmlns:gsrD="http://www.opengis.net/gml/3.2" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:gco="http://www.isotc211.org/2005/gco" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.geoviqua.org/QualityInformationModel/4.0 http://schemas.geoviqua.org/GVQ/4.0/GeoViQua_PQM_UQM.xsd"><gvq:item>

<gvq:userComment><gvq:comment>When validating the Suitability Map of Golden Apple (link) with field survey evidences, we noticed

that in the area of the “Corn de Borges Blanques, UTM: xxx” the results were not the expected. Instead of having high suitability rates for this particular fruit tree model matching the field reality, the multicriteria model shows low values. This conflict is caused by the GDD layer (an input in the fruit trees suitability multicriteria model). This layer is in turn derived from the mean June temperature, mean July temperature and mean August temperature layers included in the DCAIP (MD_Identifier id). This leads us to discover that the climatic surfaces of these layers in the DCAIP do not reflect appropriately these local climate features ultimately favouring the golden apple suitability (i.e., the climatic model locally in this geographic area used as input in the fruit tree model gives lower values than the high suitability evidences found in the field). This is caused by the lower than expected values of the nearest in-situ meteorological sensors used in the climatic surfaces interpolation models, even though the values series of this sensor are compliant with the time series length specifications. This causes that the residual interpolation structure is not representative of these particular climatic features. </gvq:comment>

<gvq:mime-type>text/plain</gvq:mime-type></gvq:userComment><gvq:rating>

<gvq:score>5</gvq:score><gvq:justification>The data itself is appropriated for this application and the metadata records is beyond the

expectation allowing for error traceability.</gvq:justification></gvq:rating>

</gvq:item></gvq:GVQ_FeedbackCollection>


OGC®

MotionMotion

• Motion to move this idea into the OGC process.• The Metadata.DWG proposes to the TC the creation of the

Geospatial User Feedback Standards Working Group that will adopt the charter document OGC14-020 Geospatial User Feedback Standards Working Group Charter https://portal.opengeospatial.org/files/?artifact_id=57200&version=1


OGC®

GEO labelGEO label

– a quality indicator for GEOSS geospatial data and datasets

– assist in searching - provide users with visual clues of dataset quality / relevance.

– provide accreditation, provenance, monitoring

– increase visibility of EO data– emphasize open access and easy

availability

23

Producer quality model ISO19115

metadataUser Feedback model

OGC®

GEO label meaningGEO label meaning

• Producer of the dataset, e.g., organisation or individual who produced the dataset, their contact information, etc.

• Lineage/provenance information, e.g., processing applied to data and number of process steps.

• Informal comments about the dataset quality as provided by the dataset producer, e.g., any identified problems, suggested use, etc.

• Dataset’s compliance with international standards, e.g., compliance with ISO 19115, Dublin Core, etc.

• Quality measures of the dataset, e.g., uncertainty measures recorded in UncertML, errors, accuracy information, etc.

• Feedback, comments and ratings provided by the users of the dataset, e.g., general comments on dataset quality, identified problems, suggested use for the dataset, etc.

• Domain experts’ comments on dataset quality, e.g., results of formal quality checks, expert suggestions on the dataset applications, etc.

• Citations where the dataset was used and cited, e.g., formal reports on dataset quality checks, journal articles, etc.

24

OGC®

GEO Label : Metadata+feedback summary with drill-downGEO Label : Metadata+feedback summary with drill-down

25

OGC®

GEO Label-based Dataset Intercomparison GEO Label-based Dataset Intercomparison ToolTool

26

OGC®

Initiative to have GEOLabel as an Initiative to have GEOLabel as an IEEE standardIEEE standard

• Dear SiriJodha, Stefano and SCC40,• • I think the GEO Label concept is suitable as a potential

standard to be developed by IEEE and would be willing to support SCC40 in sponsoring the corresponding project.

• The drafting of a Project Approval Request (PAR) would be the opportunity for the GEOSS Standards and Interoperability Forum to consult stakeholders (GEOSS IDIB and IEEE GEO WG/Hans-Peter Plag) to confirm the feasibility of such an approach and hopefully lead to a PAR.


OGC®

www.geoviqua.org

28

OGC®

Some Useful URL’sSome Useful URL’s

• Schemas XSD and UML (ASTON)– http://schemas.geoviqua.org

• User Feedback System (S&T)– https://github.com/GeoViQua/geo-userfeedback– http://geoviqua.stcorp.nl/home.html

• GEO label services, API and code (ASTON+52N)– http:www.geolabel.net/api.html – http://twiki.geoviqua.org/twiki/bin/view/GEO_SIF/SifGeoLabel – https://github.com/lushv/geolabel-service

• Tutorial (ASTON)– http://tutorial.geoviqua.org

• DAB-Q CSW, GI-CAT (CNR) description and code– http://geoviqua.essi-lab.eu/dabq-demo/


® sponsored by geoviqua user feedback on data quality 90th ogc technical committee washington, dc...

Documents