jpl r.r askin / v. z lotnicki 1 ocean esip t ech r eview 6/8/2000 ocean esip improved ocean radar...

OCEAN ESIP TECH REVIEW 6/8/2000JPL R.RASKIN / V. ZLOTNICKI 1

Ocean ESIPImproved Ocean Radar Altimeter

and Scatterometer Data, and Atmosphere-Ocean Simulations,

for Coastal and Global Change Studies

Victor Zlotnicki, JPL, [email protected]

Robert Raskin, JPL, [email protected]


Project Members

• Victor Zlotnicki, JPL• Rob Raskin, JPL• W. Tim Liu, Ernesto Rodriguez, JPL• Michael Stonebraker, UCB• Richard Troy, Science Tools, Inc.• Roberto Mechoso, Y. Chi, UCLA• C.K. Shum, Ohio State• John Ries, U. Texas


Problems to be Solved

• Long-term, global sea level time series require modeling & removing systematic differences between different sat. altimeters

• Derived sea surface wind products with the most physical meaning (wind divergence, wind stress and its curl) are not readily available


Problems to be Solved (cont.)

• A conceptual model of Earth science data processing, and an associated toolset, would help automate high level data generation, ensure data heritage information, and lower the cost of data processing.

• Mechanisms to extract corresponding hyperslices from large and diverse datasets, and deliver them on demand to end-user applications, are not readily available


Scientific Objectives

• Create and distribute improved global sea level and ocean wind products

• Facilitate the joint use of multiple sea surface variables

• Facilitate comparison between satellite data and ocean-atmosphere models


Technology Objectives

• Provide user-defined data and model subsets on demand via the internet

• Refine and implement the BigSur model of database-centric Earth science data processing, distribution and lineage tracking, its schema and toolkit.

• Further develop an application based on BigSur and Web technologies, to handle the processing steps from creation to delivery to end-users


Time Line

11/98 Late start due to lawyer negotiations

1999 2000 2001X

Data TP- SSH SST QuikScat TP Retrack GCMProducts: Output NSCAT TP-Curr ERS-SSH

Data SubsetAccess: GUI Digital Earth DODS Data Miner

BigSur TPGDR Ingest Qscat gridsAutomation: TPResiduals TPGrids

11/98 11/01


Primary Data Products• Altimeter:TOPEX/Poseidon, ERS-1,2

– Sea surface height, wave height, geostrophic current

– Retracked data

• Scatterometer: NSCAT and QuikScat– Sea surface wind, pseudo-stress

– Divergence and curl fields

• AVHRR:– Sea surface temperature (Reynolds )


Innovations• Spatial/temporal subsets selected from GUI, with plots

generated on the fly• Output types:

– Latitude-longitude maps– Animation of time-sequenced maps– Time series plots– Lon-time, Lat-time profiles– Line-time profiles along arbitrary lines

• Key technology ingredients:– Research Systems Inc’ IDL– Java


Example: NSCAT Wind


Example: Along-Track TOPEX Sea Surface Height


Example: Time Series Plot (Sea Surface Height Residual)


Example: Time Series Plot (Sea Surface Temperature Anomaly)


Example: Animation Option


Example: Longitude-Time ProfileEquator


On-line Subsetting Provided Through 3 Interfaces

• GUI (Java Applet)

• Digital Earth - Web Mapping Testbed (WMT)

• DODS (in progress)

• FTP of entire datasets


Digital Earth Interface

• Based upon the interface standard of the OpenGIS Web Mapping Testbed (WMT)

• Provides data access (maps) to other clients or any browser

• Maps have transparency option to facilitate overlay– WMT viewers can overlay Ocean ESIP data

with other WMT-compliant data


Vegetation Index (GLOBE server) & Sea Surface Temperature

(Ocean ESIP Server)


Sample Call to Ocean ESIP Server w/ Digital Earth Interface

• Sea surface height on Oct 25, 1997 over the region: 180W, 20S, 80W, 40N:

http://oceanesip.jpl.nasa.gov/cgi-bin/de.pl

?BBOX=-180.,20.,80.,40.&LAYERS=ssh

&FORMAT=gif&REQUEST=map

&WIDTH=640&HEIGHT=480&SRS=4326

&WMTVER=1.0&DATE=1997102


GIS Compatibility• GeoTIFF output available from our Digital Earth

Interface• GeoTIFF encodes spatial information along with

TIFF– Bounding box (in latitude-longitude)– Map projection (equal angle)

• GeoTIFF can be imported directly into Arc/Info, ArcView, and most other commercial GIS products


DODS Interface

• Part of cluster with the DODS ESIP

• Sea surface height, geostrophic current, sea surface winds available through DODS as of June 00


Gridded

Datasets (TOPEX/Poseidon)

Geolocated

Residual

Parameters

Processing Steps

Lineage

BigSur Schema & Tools:Data and Data Processing Procedures

all Reside Within DBMS

Processing Plans Process Queue

Atmospheric Corrections Ocean Surface Corrections Interpolation Removal of Tides Removal of Time-Mean High-level

Low-level


Potential Advantages•Reliability

–Maintaining data and processing steps within an Informix DBMS insures quality control

•Efficiency–Subsequent processing steps are automatically scheduled and carried out

•Repeatability–Reprocessing is easily carried out by changing parameters and reinitiating processes

•Verifiability–Lineage provides an audit trail showing the processing that led to the creation of the dataset


Ocean ESIP/BigSur Processes

A “process” is a conversion of one dataset into another (higher-level) dataset

Completion of a process automatically triggers new processes. A “process queue” manages computer workload and schedules

new jobs.


Example: Conversion of TOPEX GDR Dataset into “Gridded” Dataset

• Along-track grid pointsTriggered by:

Newcycle of

GDR data available

Triggers:

Createresidual dataset

• Gridded dataset

• GDR dataset

grid_data.f merge_GDR.f


VACS: Visualization, Analysis, and Control System

• Developed at UCLA

• Web-based interface to:

– maintain meta-data and visualize 2D and 3D variables from datasets that are output by model runs


•Web-based User interface to maintain metadata and visualize 2D and 3D variables from model runs.

•Automatic 2-dimensional plot generation using IDL scripts (longitude-latitude and vertical cross-sections)

•Graphical interface to select geographic region of interest•Menu options to customize plots (maps, min and max values, filtering, contour labeling, etc., )

•Output plot delivered in GIF format

VACS - Visualization Subsystem


Web-based User interface to maintain meta-data and analyze variables and fields from datasets that are output by model runs.

•Automatic generation of a 2-dimensional plot of the difference of two output datasets.

VACS - Analysis Subsystem


Web-based User interface to submit local and remote ESM runs.

•Dynamic configuration of command scripts

•Can monitor results from a run on line

•Book-keeping of users and available machines

•Automatic startup of a run on a remote host with rsh or ssh

capabilities

VACS - Control Subsystem


Help Facility http://oceanesip.jpl.nasa.gov/gui

• GUI contains context-sensitive help menus

• User questions and comments will be entered into a DBMS, accessible to all– Creates a dynamic FAQ


Inventory-Level Metadata

• Data products are all created on the fly

• Granules exist, but they are sliced and diced and recombined for delivery

• Is it worth inventorying granules for outside view? (V0 issues)


Metrics• 4 month test period (Feb-May 2000)

• 798 data requests via GUI

• 104 distinct users– Interactive nature of web access encourages multiple data

requests

• 5.9 Gb of data subsets extracted (2 bytes/data point)– Approximately 70Kb per request– Delivered amount varied depending on output type (GIF,

ASCII, averaged fields, etc.)


Metrics (cont.)• No user registration or cookies

• Voluntary customer survey to be implemented this summer

• Only 4 questions submitted to help e-mail

• Home page listed on major search engines– Yahoo, AltaVista, Excite, WebCrawler, Lycos,

InfoSeek, Northern Light


Domain Source of Data Requests

050

100150200250300

edu

fore

ign

net,

com go

v

unkn

own

Total DataRequests

Distinct Hosts


Current Status

• Data subsets of TOPEX/Poseidon, NSCAT, and SST available via 3 different interfaces

• BigSur currently automates steps:T/P GDR ingest, ATGrid, ATResids, VanGrids (~40% of processes)

• VACS replaced ESMDIS.• Retracking of TOPEX/Poseidon data in

progress (Requires 3 months, wall clock time)


To Be Done• Use BigSur (60% of processes)

• Add QuikScat, ERS, TOPEX Retracked• Advertise our data products through PODAAC

and other sources• Keep our data products up-to-date• Implement Customer registration• Use customer surveys• VACS- Extend the Analysis module to include

correlations and other statistics


Lessons Learned (so far)

• Data centers /products must be advertised to be widely used

• Tight labor market for computer expertise is a big problem

• Collaboration with scientists and computer scientists off-Lab is challenging

• Some technologies are not worth implementing


TheEnd

jpl r.r askin / v. z lotnicki 1 ocean esip t ech r eview 6/8/2000 ocean esip improved ocean radar...

Documents