Ed Armstrong, Jorge Vasquez, Andrew Bingham, Sue Heinz, Alex Cervantes, Qui Chau, Tim McKnight, Wendy Zhang

edward.m.armstrong@jpl.nasa.gov

Distribution of Medspiration products from JPL Global Data Assembly Center (GDAC)

GHRSST Medspiration and GlobCOLOUR Users Meeting 4-6 Dec 2006

Villefranche sur mer, France

Introduction - GHRSST data management

GHRSST international and regional task (and cost) sharing model: Built on 3 core components

1) Regional centers of SST expertise produce near realtime data (L2P/L4) for their sensors2) Distribution and coordination from regional centers requires “Clearinghouse” for data and metadata management

• Fulfilled by the JPL Global Data Assembly Center (GDAC)• Built on the 13 year expertise of the NASA Physical

Oceanography DAAC in satellite data management3) Archiving and reanalysis occurs the Longterm Stewardship and Reanalysis Center (NOAA NODC)

The GDAC serves as the link between data users and producers, and between producers and longterm data stewardship (between 1 and 3)

GDAC System Architecture

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Overview of GDAC tasksIngest realtime project data files

L2P, L4, L2P_GRIDDED data and metadata files from RDACs

Stage on public FTP site for 45 daysDeliver to long-term archive (LTSRF; NOAA NODC) after 30 daysProvide a variety data access methods (FTP, OPenDAP, POET)Ingest and quality assure metadata into the Master Metadata Repository. Produce FGDC metadata products for LTSRFProvide web-base interface for querying database and data discoveryAncillary data production for unfilled L2P files Provide USO and applications developmentMODIS L2P (collaboration with the NASA OBPG)

Support and value added services

FGDC metadataCreating and staging FGDC metadata for LTSRF interface

OPenDAPL2/L4 accessible via JPL server http://dods.jpl.nasa.gov

IDL and C L2P software readersRead entire L2P structure, can be used to test integrity of L2P

RDAC assistance with L2P/L4/HRDDS file format and metadata anomaliesL2P subsetting design in progressValue add L2P with ancillary filling

Wind , sea ice, solar insolation, aerosol ( from ECMWF, FNMOC etc.)

Filling for solar insolation

ECMWF Solar insolation

Same day AVHRR L2P filled with solar insolation data

Services continued

Web portal to GHRSST activities and data: http://ghrsst.jpl.nasa.govWebex site for intraproject collaboration on applications and user informationApplication User Services (AUS)

Supporting applications development (S. Heinz) Actively addressing user inquiries

Data products and supported sensors

GHRSST Level 2P European Medspiration AATSR SST GHRSST Level 2P European Medspiration AVHRR16 Global SST GHRSST Level 2P European Medspiration AVHRR16 Local SST GHRSST Level 2P European Medspiration AVHRR17 Global SST GHRSST Level 2P European Medspiration AVHRR17 Local SST GHRSST Level 2P European Medspiration AVHRR18 Global SST GHRSST Level 2P European Medspiration AVHRR18 Local SST GHRSST Level 2P European Medspiration NAR16 SST GHRSST Level 2P European Medspiration NAR17 SST GHRSST Level 2P European Medspiration SEVI RI SST GHRSST Level 2P European Medspiration AMSRE SST GHRSST Level 2P European Medspiration TMI SST GHRSST Level 4 European Medspiration Ultra-High Foundation SST GHRSST Level 2P European Medspiration NAR18 SST GHRSST Level 4 European UKMET OSTIA SST GHRSST Level 2P USA Remote Sensing Systems AMSRE SST GHRSST Level 2P_G RID DED USA Remote Sensing Systems AMSRE SST GHRSST Level 2P USA Remote Sensing Systems TMI SST GHRSST Level 2P_G RID DED USA Remote Sensing Systems TMI SST

GHRSST Level 2P USA NASA MODI S Terra SST GHRSST Level 2P USA NASA MODI S Aqua SST GHRSST Level 2P NAVOCEANO AVHRR18 Local SST GHRSST Level 2P USA OSDPD GOES11 SST GHRSST Level 2P USA OSDPD GOES12 SST GHRSST Level 2P USA NAVOCEANO AVHRR17 Local SST GHRSST Level 2P USA NAVOCEANO AVHRR17 Global SST GHRSST Level 2P USA NAVOCEANO AVHRR18 Global SST

AVHRR

AATSR

MODIS

AMSRE

TMI

SEVIRI

GOES

L4 blends

27 unique L2 and L4 products

MODIS L2P

SST

Bias

Flags

STD

MODIS L2P granule with OCSST

K490

Chlorophyll A

GOES L2P

SST AOD

WindSSI

Data Ingest Statistics -20062006 data ingest volumes

3.5 TB total

0

100

200

300

400

500

600

700

1 2 3 4 5 6 7 8 9 10 11

month (2006)

Volu

me i

n G

B

2006 files ingested327408 files total

0

10000

20000

30000

40000

50000

60000

70000

1 2 3 4 5 6 7 8 9 10 11

month (2006)

Nu

mb

er

of

file

s

Data Distribution Statistics -2006Volume distributed in 2006Total is 3.1 TB, 560K files

0

200

400

600

800

1000

1200

1 2 3 4 5 6 7 8 9 10 11

month (2006)

GB

User breakdown2

1

3

9

531111

8

78

99

Brazil

Canada

Commercial

Denmark

France

JPL

US Defense

NASA

Org

Russia

UK

Unknown

180 153.5 4579548 1044.1 18794153 1806.8 21101840 91.9 117036

Users Vol(GB) #FilesEUR

JPL

NAVO

REMSS

Data uses and impact studies

Data Use and Impact Studies from the Multisensor Improved Sea Surface Temperature Project (MISST)

NAVOCEANO: impact of assimilation of high resolution L2P including AATSR and SEVIRI on NCODA (NRL Coupled Ocean Data Assimilation system) and K10 merged SST product

NOGAPS TC track

Although there was no significant difference in the track forecast errors overall, there were areas where the use of the MISST SST analyses resulted in significantly improved NOGAPS TC track forecasts

Data Use Cases and impact studies

Additional Data Use and Impact StudiesNOAA-RSS: Hurricane intensity impact used blended microwave L4UK Met: L2P for OSTIA L4 merged productsRSS: L4 blend using AMSRE, TMI, MODISNOAA: improved Reynolds OI using TMI and AATSR L2PFNMOC: evaluate AATSR and SEVIRI in operational SST blendCanadian Meteorological Forecast Office: AVHRR L2P

MISST research

MISST researchAerosols

• AATSR used to study impact of aerosols on infrared measurements as compared to AVHRR and MODIS

– Crucial for assigning flag or weight on IR measurements in SST blends

Diurnal warming models for deriving foundation temperature

• SEVIRI measurements have excellent calibration and resolution necessary to resolve the daily diurnal cycle of SST

Skin layer temperature• Used to derive CO2 flux maps (B. Ward)• AATSR has most accurate skin layer measurement available

Data Use Cases continued

US regional users for climate, weather and ecosystem studies at federal or state agencies, non profit organizations, universities coordinated by the GDAC.

• Shorterm prediction and research center (SPORT)

SPORT supports infusion of NASA observations, data assimilation and modeling research into NWS forecast operations and decision making at the regional and local level.Using the OSTIA product for modeling and forecasting activities for the US Gulf and North Atlantic regions.SPORT is generating daily images available at: http://weather.msfc.nasa.gov/sport/sstAnimation/

Data Use Cases

The PDC has pledged their support to prepare and disseminate gridded GHRSST data via the APNHVA.Atlas is published both as Asia Pacific basin-wide and Hawaiian Island region.Support disaster community applications for preparedness, mitigation, response and recovery activities.User include Hawaii State Civil Defense, FEMA, NOAA, UN agencies, Thailand’s National Disaster Warning Center and Vietnam’s Disaster Management Committee. http://www.pdc.org

Pacific Disaster Center

Data Use Cases continued

US regional users for climate, weather and ecosystem studies at federal or state agencies, non profit organizations, universities.

• Shorterm prediction and research center (SPORT)• Pacific Disaster Center

– Asia Pacific Natural Hazards Information Network• NASA Aquarius Salinity Mission• Gulf of Maine Ocean Observing System (GOMOOS)• Southern California Coastal Ocean Observing System

(SCCOOS)• Ecosystem Management

– California State University Humbolt Bay– NOAA Protected Species Management

• Southeast Phytoplankton Monitoring Network (SEPMN)• ECCO assimilation

GHRSST metadata model

Follows the NASA Directory Interchange Format (DIF)Granule records ( File Record [FR] )Dataset record ( Dataset Description [DSD] )XML based

L2P/L4 products only accepted with valid metadataMetadata ingested into a “clearinghouse”: the Master Metadata Repository (MMR)Support for FGDC metadata standard implemented for distribution and archiving at the LTSRFFuture support

ISO19115

Master Metadata Repository (MMR)

MMRmySQL database (ver 4x)

• Fully relational, transactions implementedSeparate multi-table relational databases for DSD and FR records linked with one common fieldDSD metadata records jointly maintained by Met Office and GDAC Over 150K FR records ingested without errorQA checkers/XML parsers for FR and DSD records implementedWeb search interfaceSupports output to FGDC format through XML stylesheet conversions

MMR Search Interface

Interface to the LTSRF

FGDC records created for 30 day (and older data).30 day old data and metadata extracted from an operational FTP site by NOAAGHRSST data products expired from GDAC (PO.DAAC) FTP site after 45-60 days (“rolling store”)

Size of the rolling store can easily be adjusted to accommodate changing disk space resources

To date 100% file integrity between GDAC and LTSRF

DIF to FGDC conversion

Extract FR from MMRFormat to XML

Extract andformat DSD to XML

Apply XSL

FGDC formatted record

GDAC future activities

Integrated into the PO.DAAC in 2008ISO19115 support

Routine Ancillary L2P fillingDelivery of L4 and L2P products via the POET visualizer and subsetter

Google Earth browse capability

Additional L2P, L2P_GRIDDED, and L4 productsNetCDF 4 supportIngestion of metadata into ECHO

GDAC team

Jorge Vazquez - Project managerEd Armstrong - GDAC Cognizant EngineerAndy Bingham - Data management and Oceanids developerSue Heinz - Applications and User Services lead (sheinz@geologics.com)Alex Cervantes - Software engineer for Oceanids development and L2P ancillary fillQui Chau - Software engineer for L2P ancillary fill, MODIS L2P, L2 subsetting developmentWendy Zhang - Software engineer on MMRTim McKnight - Operations engineer (txm@seastar.jpl.nasa.gov) Pat Liggett - PO.DAAC team and management

Backup - Data Access

ftp://podaac.jpl.nasa.gov/GHRSSTOrganized by:datatype / sensor / RDAC / year / DOY

OPenDAP:dods.jpl.nasa.gov

User support:Email ghrsst@podaac.jpl.nasa.gov

OCEANIDS OVERVIEW

CATALOGUE

INGEST / QC

DISTRIBUTE

REPO

RTS

& AC

COUN

TABI

LTY

DATA

STORE

DATA PROV IDERS

USERS