processing open data using terradue cloud platform
TRANSCRIPT
Session IV: Sustainability and business strategies Processing of Open Data
using Terradue Cloud Pla<orm
Hervé Caumont Open Data Projects cluster mee2ng 07-‐08th September 2015, Brussels
Point of Contact : Hervé Caumont, Program Manager http://www.terradue.com
• Italian SME, created in 2006 – ESA ESRIN spin-off (Rome)
• UK subsidiary, created in 2011 – Space Cluster (Harwell Oxford)
• 12 start-upers – 5 different EU nationalities
Cloud Services for Earth Sciences
Agenda
• A criBcal need to solve
helping Open science to address Societal issues • Three challenges ahead with the exploita5on of Earth observa5on Open Data
• Terradue Cloud Pla<orm’s Business Model: new types of partnerships based on openness
• SupporBng sustainable services: what we have learnt about Open Data
A criBcal need to solve
Open Science addressing environment-‐related societal issues,
globally
Our customers Earth sciences
Mécanique Appliquée et Sciences de l'Environnement
MERIS MKL3 User Guide
Copyright � 2006 ACRI-ST
MERIS
Medium Resolution Imaging Spectrometer
ENVISAT-1 Ground Segment
MERIS Level 3 - MKL3 User Guide
MERIS average water vapour in 2005
Title: MERIS Level-3 MKL3 User Guide
Doc. no: P0-RS-ACR-GS-2001
Issue: 1
Revision: 1
Date: June 14, 2006
Climate Change
Ocean ecosystem
GeoHazards monitoring
EO data is there
…but Earth Science applicaBons
• are s2ll in some early stages. • Only a few opera2onal processes transferred to the industry, mostly within niche markets,
• mainly outside of the “Open Web” and its poten2al to scale them out globally.
• EO datasets remain very technical products, • with no Web-‐na2ve process matured enough, hampering innovaBons that would impact societal challenges globally.
11
The Web as a Pla<orm
Data-‐intensive science is the 4th paradigm
The Internet can do more “You can be reading a paper by someone and then go off and look at their original data. You can even redo their analysis. Or you can be looking at some data and then go off and find out all the literature about this data. Such a capability will increase the ‘informaBon velocity’ of the sciences and will improve the scien2fic produc2vity of researchers” – Jim Gray, The 4th paradigm, 2009, MicrosoV Research
Three challenges ahead
with the exploitaBon of Earth observaBon Open Data
3 proverbs of Pla<orm operaBons
If you can't beat 'em, join 'em
APIs
Build innova2on
without owning all assets
No man is an
island
Pla<orm economy
Seek
partnerships that challenge your idea
If you can't beat 'em, join 'em
APIs
Build innova2on
without owning all assets
When in Rome, do as the Romans do
Network effects
Exploit the
Web at its full scale
If you can't beat 'em, join 'em
APIs
Build innova2on
without owning all assets
No man is an
island
Pla<orm economy
Seek
partnerships that challenge your idea
Maximizing e-‐collaboraBon EC FP7 MELODIES
SenBnel-‐1 Data Access EC FP7 Sen2nels Synergy Framework
Promoting Earth Observation Services
EOP <-‐> SAFE
Data mirroring strategies
OpenSearch protocol
Open Source InSAR Processors ESA Thema2c Plaaorms
hbps://geohazards-‐tep.eo.esa.int/
> > >
Terradue Cloud Pla<orm’s
Business Model
New types of partnerships based on openness
ESA BiDS’14 Conference
hbp://dx.doi.org/10.5281/zenodo.12728
Terradue Cloud Pla<orm Concept of operaBon
29
Hosted CommuniBes
5 Climate Change
Carbon market
Climatology
Oceanography
Ship logis2cs
Land management
Water quality
Deser2fica2on
Biodiversity
Urban accoun2ng
Volcanology
Seismic analysis
Disaster Risk Reduc2on
EC FP7 GEOWOW (2011-‐2013)
EC FP7 SenSyF (2013-‐2015)
EC FP7 MELODIES (2014-‐2016)
ESA Exploita2on Plaaorms (2014 onwards)
8
14
ì
35+
H2020 EcoPoten2al (2015-‐2018)
30+
Campaigns (2013 onwards)
30
Virtual Machines for developers
my RPMs !
31
Cloud deployments ready to scale out
32
Cloud Storage on request
33
Citable applicaBons & datasets
Pla<orm as a Service
• Producer Decks (hybrid)
• Deploy Cloud Appliance clusters on public clouds
• ElasBc Catalogues • S3 data nodes
• Templates (hybrid)
• Community Hubs on private cloud
• Define OneFlow services
• Hadoop Sandboxes on private cloud
• Build ApplicaBons
Developers Integrators
Producers Data Providers
SupporBng sustainable services
What we have learnt about Open Data
We are aiming at
• Rapid prototyping and benchmarking of algorithms
• Seamless data access whatever stage of the process
• Automated data processing for non expert users
37
What is costly ?
• Open Data access is s2ll fragile, it does not inherited yet the resilience of the Web. Linked Open Data is one step ahead in that direc2on.
• The required knowledge stack for valuing Open Data is widely spread, and no single organiza2on can afford the cost to control it all.
• The legal constraints with dissemina2ng informa2on products based on Open Data are s2ll varied, and are 2me-‐consuming to understand well.
What we’ve learnt about Open Data
• Open Data benefits cannot take up without early adopter services, growing the plaaorm with us
• S2ll need to beber communicate on the available Cloud services: PaaS, DaaS, SaaS are suppor2ng the Informa2on-‐as-‐a-‐Service end game
• It’s 2me to spread the word on business models for plaaorm opera2ons : Open Science, e-‐Gov, commercial, …
We can go even further ;)
h`p://melodiesproject.eu h`p://www.terradue.com
@MelodiesProject
@terradue
References • Terradue Cloud Plaaorm – Developer Cloud Sandbox documenta2on
hbp://docs.terradue.com/developer-‐sandbox • EC FP7 SenSyF – Sen2nels Synergy Framework
hbp://www.sensyf.eu/ • ESA Geohazards Exploita2on Plaaorm
hbps://geohazards-‐tep.eo.esa.int/ • ESA Big Data From Space Conference 2014
hbp://congrexprojects.com/2014-‐events/BigDatafromSpace/ • Cantonese proverbs J
hbp://www.dramafever.com/news/81-‐cantonese-‐proverbs-‐explained-‐in-‐one-‐beau2ful-‐poster/
41