CEOS System Engineering Toolset (CSET)

CSET is a Software Framework + Suite of Tools (Apps) that leverages a Common Architecture, Unified Data Model, Common User Experience (App continuity)

Data Management and Data Integration Toolso Space Data Management System (SDMS) - Extensible Data Management framework

Analysis Tools, Data Visualization & Presentation Toolso Rapid Acquisition Tool - Determines when satellites may get acquisitions of regions/locationso Coverage Analyzer - Determines number of actual acquisitions with given cloudiness thresholdo COVE Google Earth visualization tool for satellites - Displays potential and coincidence paths for Earth

Observing Satellites. Data presentation, reporting, and charts

Informational Tools: Mission Instrument Browser, Data Policy portal, Labs…

History: CEOS Visualization Environment (COVE) started in 2009– Web-based suite of tools for searching, analyzing and visualizing actual & potential

satellite sensor coverage– Philosophy: Simple, intuitive, easily accessible– 2014 COVE Stats: ~2500 users, 20K page views, broadly utilized by countries worldwide

Space Data Management System (SDMS)

1

2

3

CSET Application Portal

Analysis &Visualization

Apps

Data Management(SDMS) Apps

www.ceos-tools.orgLive in late November 2014

Space Data Management System (SDMS)

SDMS is a flexible Data Integration & Data Management Framework – Key is the ability to quickly create applications (e.g., country/organization specific applications):

– Common Architecture & Data model, Security model, Project Model– Search & Data Discovery Capability

o Can pull in and search archives by metadata, criteriao Search the entire federated database

– Analysis & Tool Execution Capabilityo Tool Integration framework (Java/JS based tool wrapping)o Existing and custom tools to analyze data, e.g.:

GDAL, Google Earth Engine, Open Foriso Flexibility to work with different workflows/use-cases (mix-and-match)o GIS Tools

– Data Visualization and Presentation Capability– Big Data Framework (Hadoop – Local Big Data Cluster)

• Working toward model for low bandwidth connections– Data housed, fetched, and processed remotely– Tiered solution for different bandwidth levels

• SDMS is not the cure all…

SDMS Framework

Columbia

GFOI

FAO

CommunityCapabilities

SDMS Interface

2

1

3

Global SDMS Data

& Tools

User-specificSDMS Data

& Tools(Analysis Case)

Context specificWork Area

4 Search & Filter

Project (Analysis)… Data… Tools… Output

SDMS Tool Execution (Script Editor)

2

3

User-specificData & Tools

ScriptEditor

4ExecutionConsole

>> Initializing…

1Global

SDMS Data & Tools

SDMS Tool Execution (Terminal Interface)

2

1

3

Global SDMS Data

& Tools

User-specificData & Tools

Terminal Interface

GFOI SDMS Prototype

SDMS Integration - Google Earth Engine

Google Earth Engine Demonstrator:– GEE is powerful satellite data acquisition and processing system– SDMS has JavaScript wrappers to the GEE API (small subset)

o Allows heavy-lifting to be done at Google data centerso Can download result subsets at user specified bounds and resolutions

– Benefits of integration:o Data: Provides additional source of data (SDMS Unified Data Model)o Tools: Additional processing capabilitieso Workflow: Can be a source of data that is then processed by other tools

– Current demonstrator prototypeo Allows fetching of Landsat 7,8 band TIF files and

combined thumbnail datao Focus is on cloud cover removal

SDMS Status

• CSET / SDMS development has been ongoing over the last year • Our initial release is a prototype/demonstrator (Amazon servers)• Alpha release to a small user group in December 2014• Looking for feedback, hoping CEOS SEO tools continue to be of

use to the broader community• SDMS Live Demonstration

1. SDMS Interface Prototype – Concept of Operations

2. Google Earth Engine Prototype – L8 Cloud removal

3. GFOI Prototype - Kenya

POC: Brian Killough, Ph.D. brian.d.killough@nasa.gov Brian Williams, Ph.D b.williams@ama-inc.com Sanjay Gowda, Ph.D. gowda@ama-inc.com

10

Data Services Prototypes ... Kenya Demo

SEO is developing an online version of a generic data services tool for future testing.

A demo of this tool is being worked with FAO to test processing and product development for one Kenya scene.

DEMO Details Location: Kenya (Mt. Kenya National Park) Dataset: Landsat 7, (Path-168, Row-60) Step #1: LEDAPS Processing - TOA reflectance, cloud

screening (ACCA), Atmospheric Correction to SR Step #2: BRDF Correction - Use MODIS data? Step #3: Cloud/Shadow Removal - Tag pixels with a value

or color for statistics and viewing Step #4: SLC Gap Removal - Tag pixels with a value or

color for statistics and viewing Step #5: Forest Classifier (unsupervised, or MODIS

Vegetation Continuous Fields (VCF), or Random Forest) Final Product: Forest Map with Statistics (Clouds, SLC

Gaps, Forest, Non-Forest)