using feedback from data consumers to capture quality information on environmental research data

Session: Approaches to Improved Collection and Dissemination of Earth Science Data Quality InformationAGU 2015 | San Francisco | 14-18 December 2015

MINERAL RESOURCES FLAGSHIP

Anusuriya Devaraju and Jens Klump([email protected])

Using Feedback from Data Consumers to Capture Quality Information on Environmental Research Data

Images: Anett Moritz, bpaquality.wordpress.com

Outline• Definitions (User Feedback, Research Data, Data Quality)• Motivation• Goals & Solutions• Summary

• Feedback refers to information about reactions to a product.• Feedback Types

User Feedback

4 |

User experience (assessment and

usage)General (comment, how-to, suggestion, dissuasion)

RatingRequirements

(feature, content)

Image by Commonwealth Fund

Research Datasets

5 |

Research data are facts, observations or experiences on which an argument, theory or test is based.1

1The University of Melbourne draft policy on the Management of Research Data and Records

Data Quality• The quality of data is often examined based several categories.*• Quality = Fitness for use (Wang & Strong, 1996)

• Appropriate for use or meets user needs • Datasets are often used for a purpose different from the intended one.• Inadequate understanding of the purpose may lead to poor quality of derived

data.

6 | * R.Y. Wang, D.M. Strong, Beyond accuracy: what data quality means to data consumers, 1996.

(Image: https://wq.io/research/quality)

Quality Measures in PracticeData quality descriptions supplied by data providers might be incomplete or may only address specific quality aspects.

• Accessibility, e.g., persistent identification, file format• Completeness, e.g., required metadata• Compliant with community standards• Private and confidentiality concerns • Review code, e.g., check and verify replication code • Link to other research products

Scholarly and data journals may take a role in ensuring data quality, but this mechanism only applies to data sets submitted to the journals.

8 | Reference: http://www.ijdc.net/index.php/ijdc/article/view/9.1.263/358

User Feedback and Data Quality

9 | Image : http://whartonmagazine.com/blogs/women-and-leadership-moving-forward/

Data consumers may complement existing entities to assess and document the quality of published data sets.Data quality information may be gathered via a user feedback approach.

Discovered issues, data application and derived datasets

PROVIDER CONSUMER

Data creation & publication

Existing Feedback Mechanisms

10 |

Research Data Portals Feedback Mechanism

Research Data Australia (RDA) General feedback form, and user contributed tags for datadiscovery

CSIRO Data Access Portal Refer to the email of the data collector in the metadata

TERN Data Discovery Portal General contact form

Australian Ocean Data Network Portal (AODN)

General contact form and portal help forum

Atlas of Living Australia (ALA) UserVoice feedback portal

OzFlux Data Portal Email link (for all inquiries and assistance)

National Marine Mammal Data Portal General feedback form

Urban Research Infrastructure Network Email link for general inquiries, Social media buttons for distribute the link of a data set.

Examples of research data portals and their feedback mechanisms

Why Does Quality Information From Users Matter?

Feedback information from data consumers gives other users and data providers a better insight into application and assessment of published data sets.

11 |

An example of corrected groundwater chemistry data sets provided by the Geological Survey of South Australia and correction notes produced by (Gray

and Bardwell, 2015).

Data providers may use the feedback information to handle erroneous data and improve existing data collection and processing methods.

12 |

Why Does Quality Information From Users Matter?

An issue tracking component installed as part of the Terrestrial Environmental Observatories (TERENO) data portal

GoalsDevelop a systematic and reusable approach to 1. Capture user feedback on the application and assessment of

research datasets (with identifiers)2. Link feedback information to actual data sets3. Support discovery of research data using feedback information.

14 |

Feedback Application Server

Dat

a P

orta

l with

Fe

edba

ck P

lugi

n

Linked Data & SPARQL Clients

Feedback Data Store (MySQL)

RESTFeedback Web Service

RDFSPARQLD2R Server

D2R Engine

JSON RDF

User Feedback System

15 |

Feedback from users may be gathered :• Implicit (automated

tracking of data activities)• Explicit (predefined input

templates)

1 Gather feedback

2 Store feedback

3 Publish feedback

The prototype of the user feedback system

16 |

1. Gather Feedback1

17 |

A relational data model representing key aspects of user feedback:• Feedback types and

contributors• Target data and

context• Supporting

documents

2. Store Feedback2

18 |

3. Publish Feedback3

A high level overview of the W3C PROV model

Image : http://www.w3.org/TR/2013/NOTE-prov-primer-20130430/

19 |

3. Publish Feedback

Feedback published as Linked DataEntities and agent involved in an error report

feedback activity

3

Conclusions• We developed a prototype of the user feedback system to capture

quality information (assessment and application) of research datasets from users.

• The prototype supports retrieval and publication of user feedback information by combining a number of open-source technologies.

• The feedback records are made available as Linked Data to promote integration with other sources on the Web.

• The W3C PROV model is used to represent the provenance of user feedback information.

20 |

What’s Next?• Track data application and assessment in a development

environment

21 |

Thank You…

22 |

IMPORTANT ASPECTS:VALUE, EASY, FAST..

using feedback from data consumers to capture quality information on environmental research data

Software