GLAS Standard Data Productsfor Distribution by NSIDC

Polar DAAC User Working Group

PoDAG Meeting XVI

7 - 9 February 2000

Presenters

H. Jay Zwally, NASA/GSFC

Anita C. Brenner, Raytheon ITSS

Product Formats• HDF-EOS swath – level 3

– Geophysical and atmospheric parameters

– Tailored to parameters required for Interdisciplinary/multi-sensor studies

• Customized Science Computer Facility binary stream – level 1 and 2– Includes data on 3 plus all the lower level raw data and

corrections required to produce level 3

– Tailored to mathematical analysis, modeling, detailed surface analysis and comparisons with other altimetry missions and elevation data sets

GLAS Level 3 HDF-EOS swath product contents

• Time• Surface characteristics

– Location (geodetic latitude and longitude)– elevation– Reflectance– Roughness/slope

• Atmospheric parameters– Cloud layer heights– Aerosol layer heights including planetary boundary layer– Thin cloud and aerosol Optical Depth– Polar stratospheric cloud heights/optical depth

Advantages for producing GLAS level 3 product in HDF-EOS

• Level 3 parameters are GLAS science products scientists may require for multi-sensor and interdisciplinary studies with other HDF-EOS data sets.

• Tools such as WebWinds exist to visualize and compare GLAS results with other HDF-EOS data sets

• Files are self descriptive. People used to using HDF will be able to query file contents using tools

GLAS level 1and 2 SCF custom formatted products

Product ID(Identification)

Product Name ProductLevel

GLA00 GLAS Instrument Packet 0GLA01 Altimetry Data 1AGLA02 Atmosphere Data 1AGLA03 Engineering Data 1AGLA04 SRS and GPS Data 1AGLA05 Waveform-based Elevation

Corrections1B

GLA06 Elevation 1BGLA07 Backscatter 1BGLA08 Boundary Layer Height 2GLA09 Cloud Height for Multiple Layers 2GLA10 Aerosol Vertical Structure 2GLA11 Thin Cloud/Aerosol Optical Depth 2GLA12 Ice Sheet Elevation 2GLA13 Sea Ice Roughness 2GLA14 Land/Canopy Elevation 2GLA15 Ocean Elevation 2

Advantages for producing GLAS level 1 and 2 products in custom SCF format

• SCF format is flat integer binary and conforms with IEEE standards

• The polar altimetry user community is used to using data in a format similar to our internal SCF format.

• The JASON (the ocean altimetric mission NASA/CNES ) and ENVISAT( the ESA ERS-2 follow-on) missions are planning on putting their data in flat IEEE binary format.

• It will be compatible with all existing historic altimetry data. No plans exist to put this data in HDF or HDF-EOS.

Advantages for producing GLAS level 1 and 2 products in custom SCF format (cont)• Tools for displaying HDF-EOS swath data display

images and do not give value-added plotting capability to profile data.

• HDF or HDF-EOS adds complexity to accessing the data for researchers – it is a language not a format– requires ECS toolkit or 3rd party software – HDF-EOS routines are complicated to use require days

to program instead of minutes by experienced matlab and IDL programmers.

– All tools access the data one parameter at a time so access time is excessive

– File sizes are twice as large as for custom SCF

Data Access Timing Comparisons 6000 sec of data

Type of access

HDF-EOS_SW

Access time

SCF format access time

Access time factor HDF-EOS time/SCF time

all variables in file

9.6 sec 1.13 sec 7.4

one value of one variable

.640 sec .0046 sec 139

Time to read each additional variable

.026 sec .002 sec

(reads 1 sec

worth of data)

13

Using IDL to access both HDF-EOS_SW file and SCF format file

Advantages for producing GLAS level 1 and 2 products in custom SCF format

(cont)• HDF-EOS long-term usability is a concern

– HDF-EOS is a subset of HDF 4.0– HDF 5.0 is the current version and is NOT backward compatible

with HDF 4.0 – NCSA* promises only to maintain current HDF version– 3rd party tools that now read HDF 4.0 are being updated to read

HDF 5.0. There is no guarantee that they will maintain versions to read HDF 4.0

– HDF-EOS is addressing some of this concern by arranging with GSFC to maintain HDF 4.0

* NCSA – National Center for Supercomputing Applications

GLAS specific HDF-EOS concerns

• Altimeter data is not swath but a single point. We are forced to store it like multi-band images, however each "band" is really a different geophysical parameter.

• The documentation and format is so non-intuitive for profile data that it took months working with NSIDC to produce one sample file in HDF-EOS swath product.

• HDF-EOS Tools do not parse and manipulate data in an effective method for analysis of altimetry data– If data is stored in single dimension, (sec) Arrays with full-rate

parameters are very long and inefficient to access.

– If data is stored in 2-D arrays (40, n sec) the current tools will not properly parse and display the data in a manner appropriate for altimetry uses.

PO-DAAC experience with HDF data

• NSCAT, QuikSCAT and some AVHRR data only available in HDF– PO-DAAC supplies HDF subsetting utility

• Prevalent use is not to subset data, but to output the whole data set in a flat binary format

• Users then use the flat binary format

• GOSTA (Global Ocean Surface Temperature Analysis) – CD released in HDF– Community and international collaborators complained

so much that they were forced to release a new CD with data in flat binary format

PO-DAAC experience with HDF data (cont)

• Survey sent to users asking how they would feel about getting data in net_CDF (an ECS acceptable alternative to HDF)– 4 % positive– 23 % negative

• Most sophisticated users fell in this group• Complained about added complexity of anything other than a

flat binary format

– 24 % neutral– 49 % did not have an opinion