metadata standards in theory and practice · federal & non-federal exchanges (ex. iarpc)...
TRANSCRIPT
Metadata Standards in Theory and Practice
Lynn Yarmey and Sandy Starkweather!
AGU 2013 discussions
•! We planned to talk about how: !! Metadata is important !! Metadata work is ongoing !! Content is as important (or more important) than
structure
•! I’ve heard a lot of this over the course of the week !! Great! !! Also starting to hear ‘interoperability silos’
•! So let’s reconnect to the big picture
Yarmey and Starkweather– AGU Fall Meeting 2013
Starting with practices
•! IASOA !! Ideal workflow !! Actual workflow
•! ACADIS Arctic Data Explorer !! Ideal workflow !! Actual workflow
•! Lessons Learned •! Recommendations
Yarmey and Starkweather– AGU Fall Meeting 2013
IASOA – Introduction, Consortium Structure
Yarmey and Starkweather– AGU Fall Meeting 2013
!""#$%&'(%"&)*+"#(),"#&)
"-()".)(/*)012)'3"&4)
%&(*#&'(%"&'5)6'47/%8)
'(3"78/*#%9)'9(%:%(%*7)
;"'5)%7)(")$*:*5"8)'&$)
7/'#*)%&(*#"8*#',5*)
$'('7*(7)(")'$$#*77)8'&<
=#9(%9)>-*7(%"&7)
IASOA – Introduction, Consortium Structure
Yarmey and Starkweather– AGU Fall Meeting 2013
038"#('&()(")*38/'7%?*)
(/'()",7*#:'("#%*7)
'#*)!"#$%$"#$"&'()
*+"#$#)@))A*)&**$)(")
'%3)."#)('#4*(7)(/'()
9'&),*)!"#$%$"#$"&'(),%%-.,/0$#)
IASOA DATA ACCESS PORTAL– Ideal workflow
Yarmey and Starkweather– AGU Fall Meeting 2013
U.S. Department of Energy
Canadian Research Group (CANDAC)
…
ACADIS
NOAA Monitoring Programs
BSRN
Donor Archives with Structured Metadata!
IASOA Backend Database
Operationally Ingested into Backend, Differences Mediated w/ Code!
Consistently Merged on the Front End!
Overview Catalog
Search Result Cards
Donor Archives with Structured Metadata
Consistently Merged on the Front End
Global Atmosphere Watch WDC’s
IASOA – Actual workflow •! GAW – designed ISO implementation around single
parameter datasets, puts measurement parameter keyword into file format section!!
•! US DOE vocabulary conflicts with GAW (e.g. aerosols) need experts to create cross walks. Archive attribution high priority.
•! NOAA Monitoring metadata very complete, but narrative, not structured. Hire students!
•! BSRN requires logins, has very confusing navigation and creates a metadata file for each month of observations. We want to extend and amend.
•! CANDAC uses Data Mentor and won’t provide direct links to data. Instrument-based organization. More experts & crosswalks.
Yarmey and Starkweather– AGU Fall Meeting 2013
U.S. Department of Energy
Canadian Research Group (CANDAC)
NOAA Monitoring Programs
BSRN
Global Atmosphere Watch WDC’s
Arctic Data Explorer – Ideal
Yarmey and Starkweather– AGU Fall Meeting 2013
EOL Field Catalogs Met.no
ACADIS Gateway NSIDC
Coming Soon
GICat Metadata Broker
SOLR
Other interfaces or services
!"#$
%%&'
()*+,-"'
()*+,-"'
(./
0&/1234'
()*+,-"'
5&6+*&
('
Metadata !translators!
Query !handling!
Web !Services!
Web portal!
Metadata!Feed!
Repositories!NODC
Challenge – Metadata consistency
EOL Field Catalogs Met.no
ACADIS Gateway NSIDC
Coming Soon
GICat Metadata Broker
SOLR
!"#$
%%&'
()*+,-"'
()*+,-"'
(./
0&/1234'
()*+,-"'
5&6+*&
('
Metadata !translators!
Query !handling!
Web !Services!
Web portal!
Metadata!Feed!
Repositories!
More metadata translators!
NODC
Yarmey and Starkweather– AGU Fall Meeting 2013
Challenge – Metadata consistency
EOL Field Catalogs Met.no
ACADIS Gateway NSIDC
Coming Soon
GICat Metadata Broker
SOLR
!"#$
%%&'
()*+,-"'
()*+,-"'
(./
0&/1234'
()*+,-"'
5&6+*&
('
Metadata !translators!
Query !handling!
Web !Services!
Web portal!
Metadata!Feed!
Repositories!NODC
In development!
Yarmey and Starkweather– AGU Fall Meeting 2013
Lessons Learned/Reminders
•! There are practical reasons for non-standard content (ex. Optimization based on data holdings and local needs)
•! Full standardization isn't realistic
•! Need to gracefully handle non-standard content = human-in-the-loop
Yarmey and Starkweather– AGU Fall Meeting 2013
Recommendations
•! Remember the diversity in the stakeholders, processes, and needs !! We stop when we get to where we need to go !! Others do too
•! Emerging Specialists !! Data curators bridging technologies, processes, needs
•! We need to talk to each other !! Coordination mechanisms and forums at and across
multiple levels are needed o! Example levels: Research team, project, program,
regional, domain, national, international
Yarmey and Starkweather– AGU Fall Meeting 2013
Recommendations
•! We need to talk to each other (continued) !! Sharing current/best practices
o! Through on-line tools (ex. NOAA wiki, ArcticHub) o! Face-to-face meetings (ex. RCN)
"! Implementation and best practices as RCN topics
o! Federal & non-Federal exchanges (ex. IARPC) o! International coordination (ex. SAON)
!! Maturing/progressing conversations o! Informal to formal o! Bottom-up and top down mutually informed o! Iterative, be prepare to adapt and overcome o! Get to know your metadata colleagues
Yarmey and Starkweather– AGU Fall Meeting 2013
Recommendations
•! We need to talk to each other (continued) !! Example - Current Polar Activities
o! Arctic Data Coordination Network (ADCN) o! Arctic/Antarctic Data Coordination Network (A2DCN) o! Many more o! How/when do we bring these together?
!! Governance !! Community-driven consensus !! Success built on collaboration vs. competition
Yarmey and Starkweather– AGU Fall Meeting 2013
Yarmey and Starkweather– AGU Fall Meeting 2013
Thank you!
Lynn Yarmey – [email protected]!Sandy Starkweather – [email protected]!
Comments?
Questions?
!