ioos dmac standards process and lessons learned anne ball dmac steering team ocean.us
TRANSCRIPT
![Page 1: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/1.jpg)
IOOS DMAC Standards Process and Lessons Learned
Anne Ball
DMAC Steering Team
Ocean.US
![Page 2: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/2.jpg)
Topics
• Brief overview– IOOS– DMAC
• DMAC standards process
• Lessons learned
• Links to more information
• Discussion
![Page 3: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/3.jpg)
IOOS DMAC Overview
![Page 4: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/4.jpg)
Integrated Ocean Observing System
IOOS
Tracking, predicting, managing, and adapting to changes in our coastal and ocean environment
![Page 5: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/5.jpg)
Integrated Ocean Observing System
SEE
Safety Economy Environment
Oceans Coasts Great Lakes
![Page 6: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/6.jpg)
Integrated Ocean Observing System
• Safety– Earlier and better predictions of severe
weather– Search and rescue– Protect water quality– Homeland security
![Page 7: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/7.jpg)
Integrated Ocean Observing System
• Economy– Optimized shipping routes based on improved
forecasts– More accurate, longer-term forecasts for agriculture– Better planning for coastal construction and zoning– Potential sources for medicines and new technologies– Alternative energy sources– Improvements in offshore drilling practices
![Page 8: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/8.jpg)
Integrated Ocean Observing System
• Environment– More timely and accurate predictions for oil
spills and other pollutants– Reduced vessel groundings– Better ecological protection– Fisheries– Protection and restoration of marine
ecosystems
![Page 9: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/9.jpg)
Integrated Ocean Observing System
• Three subsystems– Observations and Data Telemetry– Data Management and Communications
(DMAC)– Modeling and Analysis
![Page 10: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/10.jpg)
Integrated Ocean Observing System
• 11 Regional Associations
• Federal agencies
• National programs
![Page 11: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/11.jpg)
Data Management and Communications
• DMAC Goal– design the system that efficiently and
effectively links observations to applications by enabling rapid access to diverse data from multiple sources as needed
![Page 12: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/12.jpg)
Data Management and Communications
• Infrastructure– Data discovery, access, transfer, metadata,
archive– identify the technologies, formats, and
protocols that support an infrastructure needed to meet its goal.
– must interact with regional, national, and international systems.
![Page 13: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/13.jpg)
Data Management and Communications
• Usability– Quality assurance, quality control, information
requirements– Data must be accompanied by the information that
supports its proper use and sound results when combined with additional data.
– This information may vary between different types of observations and collection methods.
– DMAC will adopt quality assurance, quality control, and content requirements identified by expert communities and ensure this information is carried within the DMAC infrastructure.
![Page 14: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/14.jpg)
DMAC: One of three IOOS subsystems
ObservingSubsystem
Satellites
Aircraft
Fixed Platforms
Ships
Drifters & Floats
AUVs
Metadata standards
Data discovery
Data transport
Online browse
Data archival
Safe & EfficientMaritime Operations
Homeland Security
Natural Hazards
Climate Change
Public Health
Ecosystem Health
Living MarineResources
DMACSubsystem
Modeling & AnalysisSubsystem
![Page 15: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/15.jpg)
DMAC Implementation
Team approach
• Steering Team
• Expert Teams
• Caucuses• Working groups• Interagency Oversight Working Group
![Page 16: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/16.jpg)
DMAC Steering Team
• Representatives from government, industry, academia, public, and non-profits
• Coordinate and oversee DMAC standards evolution
• Identify and provide recommendations on gaps
• Conduct process in an open, objective, and balanced manner
![Page 17: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/17.jpg)
DMAC Teams
• Expert teams– Archive– Metadata and data discovery– Transport and Access
• Working group– Systems engineering
• Address key issues as defined in DMAC plan• Review and make recommendations on
standards
![Page 18: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/18.jpg)
DMAC Caucuses
• Outreach and community engagement– International– Private sector– Education
• K-12• Professional development
– Modeling– Regional
![Page 19: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/19.jpg)
Interagency Coordination
Interagency Oversight Working Group (IOWG)
• Representatives of federal agencies
• Oversight of DMAC implementation within agencies
![Page 20: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/20.jpg)
Roles
• Teams:– Review standards– Make recommendations– Identify gaps
• Organizations– Implement standards and recommendations– Fill-in gaps
![Page 21: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/21.jpg)
Standards and standard process
![Page 22: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/22.jpg)
Standards
Standards are key to the success of DMAC
• Identify standards needed• Coordinate with other standards processes• DMAC standards mantra:
– Adopt– Adapt– Develop only as a last resort
![Page 23: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/23.jpg)
DMAC Standards Process
• Identifying existing standards is a first step
• Often need to “standardize” the standard– Ex. Use FGDC metadata – not good enough
to describe waves, so must develop a waves profile
• Data providers may be required to use different standards– Crosswalks, interoperability is critical
![Page 24: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/24.jpg)
DMAC Standards Process
Three status levels for standards:• Submitted
– Has no standing within IOOS.
• Proposed– Has status within IOOS and can be used provisionally
by IOOS information system managers for evaluation purposes.
• Recommended– All IOOS data systems should consider supporting
these guidelines wherever applicable
![Page 25: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/25.jpg)
DMAC Standards Process
Originator Submits a Standard
Internal Review
DMAC Chair –Applicable?
Gatekeeper –Complete?
Start
Expert ReviewDMAC ST
Polls
Public Comments
Community Conducts
Tests/Pilots
DMAC ST Polls
End
“SUBMITTED STATUS”
“RECOMMENDED STATUS”
“PROPOSEDSTATUS”
![Page 26: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/26.jpg)
Website: ioosdmac.fedworx.org
![Page 27: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/27.jpg)
Submission Entry
![Page 28: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/28.jpg)
DMAC Standards Process
• Entry fields:– Name– Type (new, existing, adaption)– Data mgmt function (metadata, transport, …)– Abstract– Purpose– Technical description– Statutory requirement or international agreement?
![Page 29: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/29.jpg)
DMAC Standards Process
• Entry fields (continued)– Relationships, dependencies, conflicts– Current usage– Justification– References– Acronyms– Contact information– Supporting parties/members
![Page 30: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/30.jpg)
DMAC Standards Process
• Semi annual review schedule:– Expert review
• 1 month• September and March
– Steering team review• 1 month• October and April• Steering team initial poll (on-line)• Recommendations written
– Steering team final poll• May and November meetings
![Page 31: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/31.jpg)
DMAC Standards Process
• Chair drafts recommendations based on polling and distribute to Steering Team
• Steering Team discusses recommendations at semi-annual meeting– Try to reach consensus– If consensus is not possible (per Terms of Reference):
• Vote is carried by 75% of those members voting “approve” or “disapprove.”
• Quorum consists of 60% of members.
• Support of multiple strategies may be the most prudent approach. As long as the consideration of multiple options does not violate the spirit of IOOS interoperability, the IOOS-ST may move forward without consensus.
![Page 32: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/32.jpg)
DMAC Standards Process
• Steering team may decide:– Move forward (“submitted” to “proposed”)– Request additions or changes– Table and revisit in next round– Reject
![Page 33: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/33.jpg)
Lessons Learned
![Page 34: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/34.jpg)
First round - November 2007
Eight standards submitted:
• IOOS Vocabulary Version 1
• Required and Recommended Vocabularies for IOOS Metadata
• Metadata Catalog Services
• Data Access Protocol – DAP 2.0 (transport)
• Real Time Quality Control Tests for In Situ Ocean Surface Waves
• Quality Control Standards for Real-Time, In Situ Currents Measured by Teledyne RD Instruments
• High Frequency Radar Surface Currents
• Standards Package for the Representation and Transport of Gridded Data: netCDF+CF+OPeNDAP+aggregation
![Page 35: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/35.jpg)
First round
Four moved to “proposed”:
• IOOS Vocabulary Version 1 • Metadata Catalog Services • Data Access Protocol – DAP 2.0 (transport) • Standards Package for the Representation and Transport of
Gridded Data: netCDF+CF+OPeNDAP+aggregation
![Page 36: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/36.jpg)
First round
Four tabled:
• Required and Recommended Vocabularies for IOOS Metadata
• Real Time Quality Control Tests for In Situ Ocean Surface Waves
• Quality Control Standards for Real-Time, In Situ Currents Measured by Teledyne RD Instruments
• High Frequency Radar Surface Currents
![Page 37: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/37.jpg)
First round
• Tabled submissions:– Real Time Quality Control Tests for In Situ Ocean
Surface Waves – Quality Control Standards for Real-Time, In Situ
Currents Measured by Teledyne RD Instruments – High Frequency Radar Surface Currents
• Should DMAC be involved in QA/QC?• If so, how?
![Page 38: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/38.jpg)
First round
Tabled submission:• Required and Recommended Vocabularies for IOOS
Metadata
• Adopt multiple vocabularies?• Use technology/ontologies to map multiple vocabularies?
![Page 39: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/39.jpg)
First round
• Findings– Wide variety of “standards” submitted– No clear definition of “standards” vs “best practices”– How do we handle QA/QC standards?– Need better instructions and/or help for submitters
![Page 40: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/40.jpg)
Additional thoughts
• May need to better define what we’re looking for
• May need to focus more on data/metadata content
![Page 41: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/41.jpg)
May 2008
• Addressed tabled submissions– Additional fields– Tried to refine definitions of fields– Tried to define “standards” vs “best practices”
![Page 42: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/42.jpg)
QA/QC
• “Lessons Learned” discussion:– ST lacks technical expertise to confidently evaluate
QA/QC– Better criteria needed
• To guide ST in evaluation• To assist Originators in preparing “Work Packages”
– Draw on expertise of QA/QC-experienced organizations
• As Originators• During Expert review
– Devote more effort (time) to submission ‘Work Packages’ at front-end
![Page 43: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/43.jpg)
QA/QC
• Is QA/QC a DMAC issue?– Regardless, DMAC must deal with QA/QC to move
forward– Develop supplemental criteria for reviewing QA/QC
submissions
![Page 44: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/44.jpg)
QA/QC Applicability
• Does the submission:– Address an IOOS data type or parameter?– Significant to multiple applications or users?– If specific to a particular instrument, can approach be
extended to similar instruments?– Would it benefit IOOS if implemented more widely?
![Page 45: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/45.jpg)
QA/QC
• Originating organization– Does Originator have recognized expertise in the
topic area?– Will the standard (including, if applicable, all
components of the submission) be maintained over time by a recognized organization?
• Supporting organizations– Are recognized organizations or programs supporting
the adoption of this as a standard?
![Page 46: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/46.jpg)
More information
![Page 47: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/47.jpg)
DMAC Documents
DMAC Plan
•Detailed plans for implementing DMAC •Created with input from federal and state governments, academia, non-profits, and private industry
http://dmac.ocean.us
![Page 48: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/48.jpg)
DMAC Documents
Guide for IOOS Data Providers
• Draft of the IOOS Data Policy
• Guidelines for data and metadata interoperability standards and best practices
http://dmac.ocean.us
![Page 49: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/49.jpg)
DMAC Standards Website
• Submit standards
• Provide feedback on standards (public comments)
![Page 50: IOOS DMAC Standards Process and Lessons Learned Anne Ball DMAC Steering Team Ocean.US](https://reader030.vdocuments.us/reader030/viewer/2022032723/56649d125503460f949e5e99/html5/thumbnails/50.jpg)
Discussion