icos carbon portal progress september 2016bin... · • integration with neon (ringo) and africa...
Post on 30-May-2020
6 Views
Preview:
TRANSCRIPT
ICOS Carbon Portal
Progress September 2016
Alex Vermeulen & ICOS CP team
Carbon Portal “up front”
A website offering users
• Search for data
• Quick view of datasets
• Download of data
• Visualization tools (spatial & temporal)
• Statistics tools
• Synthesis reports
• Educational materials
• Usage summaries
• Community platform for elaborated products
http://www.icos-cp.eu
Carbon Portal principles
• Interoperability
• Internal across themes (air, ecosystem, ocean)
• Other (EU) infrastructures
• Global domain
• Curation, Provenance, Transparancy
• Open linked data
• Scalable and open solutions
• Web standards
• Involvement of and interaction with users
https://github.com/ICOS-Carbon-Portal
ICOS ERIC Carbon Portal, Lund, Sweden | www.icos-cp.eu 3
ICOS Data
• Level 0 – raw sensor output (either mV or physical units)
• Level 1/NRT – calibrated and automatically assured data
• Level 2– final observation data products
• Level 3– elaborated data products, ICOS data
4ICOS ERIC Carbon Portal, Lund, Sweden | www.icos-cp.eu
ICOS Data Policy
• Open and free of charge access to ICOS data products
• License: Creative Commons 4.0 BY
• Attribution required, ICOS requests:– Good scientific practice of use
– Citation needed, ICOS provides suggestion
– Acknowledgement or co-authorship when reasonable
• Redistribution is allowed but not necessary:– ICOS provides free, easy and most up to date access to all data!
– No need for own storage, ICOS provides the best and easiest!
• Users are encouraged to contribute their products and
distribute through ICOS CP through CC 4 BY
5ICOS ERIC Carbon Portal, Lund, Sweden | www.icos-cp.eu
Identification “strategy”
• A “complete” PID & metadata strategy is useful, from
data generation to results publication
• Some initial “cost” (time & effort), but increases
interoperability and reproducibility
• Parallel use of ePIC PIDs and DOIs
– Low-level ePIC for “mass data” & “work flow”; useful in PID collections
– DataCite DOIs for fully citable data with strong preservation policies;
mostly useful at “end” of data life cycle
U.
Sch
ward
mann,
Euro
pean P
ers
iste
nt
Identi
fier
Conso
rtiu
m (
ePIC
), 2
015
ICOS ERIC Carbon Portal, Lund, Sweden | www.icos-cp.eu 6
ICOS and EUDAT common services
Measurement stations
(National networks)
ICOS Carbon Portal
User 1 User 2 User 3
Ecosystem
Thematic
Centre
Atmospheric
Thematic
Centre
Oceanic
Thematic
Centre
Standardized
processing, quality
assurance & control
ICOS
repository
(data,
metadata)
Sensor data
• Data ingestion
• Metadata services
• Data discovery & access
• Usage tracking
• Data management
• Repository
administration
• Preservation planning
• User community support
Diverse user
communities, including
data producers and
other portals
High per-
formance
computing
services
Finalized
data productsMetadata
registry &
catalogue
services
B2SAFE
B2STAGE
PID
PID, AAI
B2FIND
PID
AAI
ICOS data flow, common services
ICOS ERIC Carbon Portal, Lund, Sweden | www.icos-cp.eu 8
Calibration
Labs
Ingestion
Station TCCP
(passthrough)
9ICOS ERIC Carbon Portal, Lund, Sweden | www.icos-cp.eu
StationCP
(passthrough)TC CP Users
NRT;L2L0
L0 L0
B2SAFE
B2SAFE
PID
PID
Users CP Users
B2SAFE
B2SAFE
L3 L3
CC4
BY?
Lice
nse?
CPPID
CP
PID
PID
PID
Progress• Central is Data Upload Service
• Connected to ontology driven persistent versioned metadata store
• Tasks
• Registers metadata of all data objects
• Mints PID’s (ePIC)
• Registers and receives checksummed data objects
• Replicates object to trusted repository (B2SAFE)
• Enriches metadata with ICOS specific ‘static metadata’ for landing page,
minimum Inspire/ISO19115
• Creates (dynamic) landing page/PID end point including evt. citation
• Facilitates machine-machine interaction
• Integrates with AAI (voluntary user registration)
• License information in metadata and landing page
• Enables usage tracking through PID
• Enables attribution/citation through PID
• Meta data registered at other (e.g. GEOSS) portals
• Assures data integrity and transparency
• Supports RDA recommendations for citation of dynamic datasets and
collections: query PID
Motto: Make use simple and render redistribution obsolete and completely useless!
ICOS ERIC Carbon Portal, Lund, Sweden | www.icos-cp.eu 10
Progress…
• Data upload tool for L0, L2 and L3 using REST API
• L0 upload for ETC data
• First two L3 datasets published at CP
• Scalable cloud services, VRE’s (EGI <->EUDAT):– Cloud service tests for footprint provision for atmosphere
– Modeller interactive interface using Jupyter python notebooks:
• Global Carbon project
• Theme 2 ENVRI+: data identification and citation, Use Case (ICOS) and several implementation cases
• Integration with NEON (RINGO) and Africa network (SEACRIFOG)
• CP as central repository for Inversion systems (CP/Eurocom workshop Lund, June 2016)
• Provision of pre-ICOS data (Obspack, link to WMO GAW and IG3IS)
11ICOS ERIC Carbon Portal, Lund, Sweden | www.icos-cp.eu
Danger
Metadata editor: ICOS Station labelling Service
12ICOS ERIC Carbon Portal, Lund, Sweden | www.icos-cp.eu
ICOS Data Service https://data.icos-cp.eu/portal/#search
13ICOS ERIC Carbon Portal, Lund, Sweden | www.icos-cp.eu
ICOS Data Service
14ICOS ERIC Carbon Portal, Lund, Sweden | www.icos-cp.eu
ICOS Data Service
15ICOS ERIC Carbon Portal, Lund, Sweden | www.icos-cp.eu
ICOS Data Service
16ICOS ERIC Carbon Portal, Lund, Sweden | www.icos-cp.eu
Example data object landing page
ICOS ERIC Carbon Portal, Lund, Sweden | www.icos-cp.eu 17
Data license page
https://www.icos-cp.eu/license
18ICOS ERIC Carbon Portal, Lund, Sweden | www.icos-cp.eu
Thank You!
Read more...
• Materials from DataCite & ePIC workshop “Persistent Identifiers: Enabling Services for Data Intensive Research” (Paris, 2015) – see https://blog.datacite.org/recap/
• Deliverable 5.1 from ENVRIplus (contact Alex or Maggie)
• Socha – “Out of cite, out of mind”, Data Science Journal 12 (2013) – see https://www.jstage.jst.go.jp/article/dsj/12/0/12_OSOM13-043/_pdf
• Research Data Alliance (RDA) work groups – see http://rd-alliance.org
ICOS ERIC Carbon Portal, Lund, Sweden | www.icos-cp.eu 20
top related