nash data sharing workshop stuart hamilton, andré bouchard, michael allchin
TRANSCRIPT
NASH data sharing workshop
Stuart Hamilton,André Bouchard, Michael Allchin
Workshop goals
•Community-led initiative on hydrometric
data sharing
•Common Vocabulary
•Advocate for Reciprocal Altruism
•Report to WMO
State of the art in hydrological data sharing•Data sharing involves many complexities
owing to the number of systems, formats, organizations, etc.
•The big tent approach (all-encompassing)
The CUAHSI model
•Consistent, standardised infrastructure, covering
▫Observation methodologies, ▫metadata, ▫data model, ▫Search discovery query engine▫Transmission format ▫Modelling platform
WaterML 2.0
•Universal OGC / WMO standard for hydrometric and hydrometeorological data
•Consistent internal tag-based format, covering▫Observation features / sensors▫Time-variant data▫Metadata▫Associations (eg, rating curves)
To date, hydrologists and water resources professionals have often encountered hurdles in; •Finding data•Assessing fitness for purpose•Data retrieval•Format translation:
•New technologies are at the pointof offering a range of solutions foraddressing and solvingthese challenges
The Panel
•Axel Anderson, FRI/AERD•Wayne Jenkinson, CHC•Alain Pietroniro, WSC•Kevin Shook, USask•Stephanie Smith, BC Hydro•Bruno Tassone, WSC
What are the main barriers to data sharing?• Academic perspective:
• Data management challenges for short term projects• Security concerns: maintaining privacy, strategic
data,• End-user perspective:
• Search and Discovery• Data provider perspective:
• Administrative, technological challenges• Data rescue – important but costly
• Corporate perspective:• Mandate/cost
Audience response• Policy and structural issues • Research data and data management issues.
▫Data often seen as by-products rather than assets• No standards in terms of metadata.
▫Discovery extremely challenging. Metadata directories are essential. Web ‘storefront’ less human
• Balance to be struck between▫rights to data-access▫responsibility for appropriate use
• NASH could foster proper standards or practices.
Policy barriers•Academic Perspective
• No real policy barriers in academia. Rather suffer from a lack of policy. Funding agency driven.
•End-user perspective:•Data Provider perspective:▫No conceptual barriers to providing data▫Administrative and funding constraints (e.g.
CLF)•Corporate perspective:▫IT security
Audience Response
•A push to make data available needs to happen.
•Governance as a barrier (lack of policy)▫Mandate to provide data▫Let people know what is available and
where to get it- suspicion that data are ‘hidden’.
•Requirements on QA/QC▫Data life cycle constraints▫Real-time vs. Archival quality
Other barriers• Academic perspective:
▫data should be valued▫lots of data of varying quality
• End-user perspective:▫we should plan to share data from the onset
and not after the fact. ▫Data are valuable – use as an incentive.
• Data Provider perspective:▫Commitment to long term maintenance of
datasets. Governance issue. Who is the data guardian.
▫Data quality is a big issue.• Corporate perspective:
▫Accountability/liability for key data sets.
Audience Response• Lack of proper data management practices.
▫ File management rather than data management▫ Lack of standard exchange formats▫ Relation between credibility and training /standards
• Communication▫ Insufficient connections between data producers and data consumers. ▫ Increasing synergy happening in the private/public sector
• Lack of incentives to share data.• Use of policies related to public security as an incentive to share
data.• Incentives – lack of distribution leads to mistrust from clients• Need to consider the community monitoring groups in the
discussion.• US
▫ Issue of data rescue and the costs involved.▫ Issue of costs to manage existing and legacy data holdings
• A real opportunity for NASH to set guidelines and standards▫ Technology has evolved▫ Open data standards▫ The move to real-time▫ Interoperability
What makes you most excited about what you see happening ?• Academic perspective:
▫ Are addressing their data management issues and structuring their data.
▫ Interested about connectivities• End-user perspective:
▫ The possibilities of XML as a standard exchange format.▫ Idea of community initiatives to make data available.▫ The technology is progressing rapidly and is expanding the
possibilities. Access to many layers of data through GIS UIs.
• Data Provider perspective:▫ A certain maturity in the hydrometric community. The concept of
« Network of networks ». The notion of data assimilation and remote sensing. Systems are becoming more robust.
• Corporate perspective:▫ Climate – related monitoring program. ▫ Who has what and where ? ▫ Next step is creation of a data portal.
• NASH – role of putting people together.
Audience Response• Cybera – We-Hub (waterenvironmentalhub.ca)
▫Some issues but a good example.• A simple set of questions that a data provider
can fill out related to data quality• Hardest parts is setting the specs. • Concept of the data mart• Web-based & open-source systems. • Need to resolve firewall issues.• OGC family of standards (include security
specs).• Quality flagging – should look at the program done in the EU GeoWOW ! (standards for GEO data). • NASH could provide input
Audience Response (opportunities)• Which data and metadata should be made readily
available…▫ USGS : pretty much all the data that users require in a
timely manner.▫ Look at the community to drive the issue in terms of their
needs.▫ Vision : quality assured real-time flow information on the
web (and in an accessible data source). The move to real-time makes the old data production process an artifact.
• Common data access▫ To generate the change, you need to provide the tools to
facilitate that change. Tools can be costly to develop▫ Example : Water temperature provided along with
hydrometric data to enable users to better understand what is going on.
• Metadata is absolutely crucial. ▫ WMO standards a good place to start. ▫ WaterML 2.0 powerful as well.
Audience Response (opportunities)• Policy improvements ? Program activities ?
▫A suggestion to present the NASH group to the next NAT meeting in St. John’s (NL) – fall 2012
• Water data governance – tools are already available.▫How do you build on those tools with the
community ?▫Council of the federation : knocking on their door
to establish the connections.▫Alberta has a CEO of monitoring.
• Data needs to be treated as a capital asset and not just as a production tool. It should therefore be life-cycle managed.
Summary - Russell•5 themes : •Policy (more practice).•Technology / remote sensing – providing
data in areas where we don’t have data. •Standards•Awareness of technologies and standards.•Willingness to participate.
▫Two words : magic and maturity of hydrometric community. ▫Incentive / cost.
Summary – Paul W.•The issue of trust.•Hard coded push-pull world that we need
to deal with.•Willingness to discuss barriers.•Lack of access to data hurts us all.•Who owns the data ?•Fiscal problem with the management of
long-term data.•The value of data.•Our willingness to help.
Final thoughts•The model for sharing data presented by
Stephanie is interesting and could be explored by NASH.
•Sharing results and synthesis of workshop•Follow up with focussed workshops on
critical issues•A final report will be presented at WMO this
fall.•We should consider other publication options as well.•Culture shift required.
Thanks to ALL!
http://hydrographers.wikispaces.com/