snap – the sentinels application...
TRANSCRIPT
![Page 1: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/1.jpg)
SNAP – THE SENTINELS APPLICATION PLATFORM
Carsten Brockmann Norman Fomferra
Earth Observation Open Science 2.0, ESRIN, 12 - 14 October 2015
Brockmann Consult GmbH
![Page 2: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/2.jpg)
What is SNAP?
• SNAP: ESA Sentinel Application Platform • A common software platform and host for the
Sentinel Toolboxes and others • One application, one installation on end-user’s
computer • Synergistic use of various EO data with
common and specific tools • Fully open-source, GPL 3 • Joint, collaborative development of the primes
running the Sentinel Toolbox projects • It’s a ESA SEOM funded activity
![Page 3: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/3.jpg)
Sentinel Toolboxes Consortia
EO Science 2.0, ESRIN, Oct 13, 2015
driven by user requirements, guided by user expertise, implemented by a experienced technical team
![Page 4: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/4.jpg)
Sentinel 1,2,3, ENVISAT, Radarsat, Spot, Modis …
ENVISAT, Radarsat, SPOT, MODIS, …
Toolbox Evolution
EO Science 2.0, ESRIN, Oct 13, 2015
SeaDAS 7
SNAP S1TBX S2TBX S3TBX SeaDAS 8+
BEAM NEST
NASA ESA
NASA ESA
, Radarsat, Spot, Modis, …
![Page 5: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/5.jpg)
Challenges for ESA‘s toolboxes in a new era of Earth Observation
• Sentinel (and other products) come with new features – per pixel uncertainty – multi-resolution – very flexible file format (SAFE like)
• Very large raster size of Sentinel Products – e.g. Sentinel 2: 40.000 x 40.000 pixels and larger
• Big Data Volumes – Sentinels 1+2+3 = Terabytes / day – Petabytes / missions
• Processing Algorithm Complexity – Intense use of spatial & temporal window processing – Working on time series / time series analysis – Iterations & recursion
• Processing where the data is instead of data to processors
• Exploiting cloud services
• Community tools to share data, resources, results, …
• High expectations from users: free, open, extendible, quality (stable, fast, support, …)
EO Science 2.0, ESRIN, Oct 13, 2015
![Page 6: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/6.jpg)
SNAP Architecture
• Dynamic, module-based architecture, with various extension points and extensions
• Install and update of extension modules
• Use from Java and Python programs, extend by Java and Python plugins
• Write a SNAP plugin, use it in all toolboxes
• High-level architecture comprises two subsystems – SNAP Engine, the core and command-line interface – SNAP Desktop, the graphical user interface
EO Science 2.0, ESRIN, Oct 13, 2015
![Page 7: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/7.jpg)
SNAP Architecture
EO Science 2.0, ESRIN, Oct 13, 2015
SNAP Engine
Java SE 8 Platform
NetBeans RCP
SNAP Desktop
Sentinel-3 Toolbox (S3TBX) Sentinel-2 Toolbox (S2TBX) Sentinel-1 Toolbox (S1TBX)
Python
GeoTools JAI NetCDF …
Any combination of toolboxes add-ons is allowed, even none, as SNAP Desktop is a already a useful stand-alone application for EO data exploitation.
Programming language layer
3rd-party library layer
SNAP layer
![Page 8: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/8.jpg)
Main Subsystems
• SNAP Desktop • Modern, intuitive and rich user interface • Fast display of giga-pixel images • Large portfolio of analysis and visualisation functions • Operator interfaces and graph builder for processing
• SNAP Engine • SNAP core code base • EO data model, I/O & operator APIs • Python API allowing to use also numpy, scipy, pandas, etc with SNAP • Common, generic I/O formats: NetCDF, HDF, GeoTIFF, Shapefiles, … • Common, generic functions: reprojection, subset, geo-coding,
collocation, band maths, image filters, masking tools, … • Command-line interface, no GUI • Various uses: library, service implementation,
Cloud services exploitation
EO Science 2.0, ESRIN, Oct 13, 2015
![Page 9: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/9.jpg)
SNAP and the Cloud
• SNAP Engine (and Desktop) is truly platform independent and can be used in various environments
• Distributed Computing using Virtual Machines • Implementation of Web Processing Services • Integrates perfectly with • Apache Yarn/Hadoop, e.g. the
Calvalus processing system
More in Luis Veci’s talk on Sentinel-1 Toolbox (16:30) and his Cloud Exploitation Demoon Amazon EC2, Wednesday afternoon
EO Science 2.0, ESRIN, Oct 13, 2015
![Page 10: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/10.jpg)
SNAP Desktop and the Toolboxes
EO Science 2.0, ESRIN, Oct 13, 2015
![Page 11: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/11.jpg)
Graph Processing Framework
• Majority of SNAP „functions“ are implemented as operators
• Each operator can be invoked from SNAP Desktop and from the command line
• Processing chains („graphs“) are configured in XML files
• Graphical Graph Builder • Graph Processing Tool (gpt) for
executing of graphs (chains) >gpt -help
EO Science 2.0, ESRIN, Oct 13, 2015
![Page 12: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/12.jpg)
SNAP Applicaton Modes
EO Science 2.0, ESRIN, Oct 13, 2015
PC, notebook, tablet
I Interactive Exploration
SNAP gpt:_
SNAP Graph Processing
Framework:_
II
PC, notebook server
bulk / NRT processing
cluster, cloud
III EO data
processing centre
SNAP gpt:_
SNAP Graph Processing
Framework:_
IV Cloud Exploitation
Platform
PC, notebook, tablet
![Page 13: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/13.jpg)
Multi-Sensor Support
• SNAP supports generic NetCDF, GeoTIFF, shapefiles, etc.
• Toolboxes add specific reader plugins for their domain
EO Science 2.0, ESRIN, Oct 13, 2015
![Page 14: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/14.jpg)
Generic Functions and Tools
• Applicable for all toolboxes and wide range of sensors
• Raster data and vector data tools • Visualisation
– Multi-layer displays, layer editors – Image, mask, shapes overlays – Colour management, fast navigation
• Data Analysis – Various statistics and plot types – Spectrum display (optical)
• Data processing – Reprojection, Collocation, Mosaicing – Level-3 processor – Graph processing, ürocessing graph
builder
EO Science 2.0, ESRIN, Oct 13, 2015
![Page 15: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/15.jpg)
Sentinel-3 Toolbox
• Ingesting Sentinel-3 OLCI and SLSTR • Ingesting Envisat MERIS, AATSR, ATSR, MODIS, SeaWiFS, VIIRS, Proba-V, SPOT-VGT,
Landsat TM, etc. • Reuse all of basic functions from SNAP: band maths, spectrum tool, … • Uncertainty information exploitation
– OLCI and SLSTR „error“ variables – Various visualisation modes (blending and overlays) – Propagation build into band-maths (Standard Combined Uncertainty, GUM 95)
• Remote in-situ database access tool (MERMAID for MERIS/OLCI) • Data Processors
– OLCI/SLSTR collocation tool – OLCI IOP, FLH+MCI, NDVI, pixel classification processors – SLSTR LST, SST, cloud screening processors
• Commissioning support tools – OLCI/SLSTR collocation accuracy assessment (L1C product) – SLSTR PDU stitching tool
EO Science 2.0, ESRIN, Oct 13, 2015
![Page 16: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/16.jpg)
STEP
• step.esa.int • Science Toolbox
Exploitation Platform • EO Science collaborative
platform • Technical forum and
community animation • Gathering user feedback
and usage • SNAP tutorials and
documentation • SNAP software download see talk of Yves-Louis Desnos at 16:15
![Page 17: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/17.jpg)
SNAP Download
EO Science 2.0, ESRIN, Oct 13, 2015
1600 beta user Thanks you so much for valuable feedback!
![Page 18: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/18.jpg)
SNAP Roadmap
• SNAP 2.0 beta 8 on Wednesday (before Hackathon) – Feature freeze for S1TBX, S2TBX, S3TBX
• SNAP 2.0 final end of October – Multi-size issue stable, blocking all tools that request single-size – S1TBX, S2TBX, S3TBX – New SMOS Toolbox
• SNAP 3.0 January 2016 – True multi-size, resampling operator – Lots of new features for S1TBX, S2TBX, S3TBX
• SNAP 4.0 March 2016 • SNAP 5.0 June 2016
Project ends in July 2016
EO Science 2.0, ESRIN, Oct 13, 2015
![Page 19: SNAP – THE SENTINELS APPLICATION PLATFORMseom.esa.int/openscience15/docs/default-source/15c12...SNAP – THE SENTINELS APPLICATION PLATFORM Carsten Brockmann Norman Fomferra Earth](https://reader035.vdocuments.us/reader035/viewer/2022062414/5ece7c06048db6121722dc93/html5/thumbnails/19.jpg)
Related Talks and Events
• Today – 16:15 – Sentinel Toolbox Exploitation Platform,
Yves-Louis Desnos, ESA ESRIN – 16:30 - The Sentinel-1 Toolbox, Luis Veci
• Wednesday afternoon: – Demonstration of Cloud services exploitation with
SNAP, Luis Veci – Demonstration of Sentinel-2 Toolbox, Julien Malik
• Thursday / Friday: – SNAP Hackathon, learn from SNAP developers
EO Science 2.0, ESRIN, Oct 13, 2015