Passive Microwave Data Set Management Workshop:

Preliminary ResultsA review of the passive microwave radiometer

data sets at Earth Science Data Centers supported by NASA

Workshop Committee Members:Ed Armstrong, Helen Conover, Michael Goodman, Brian Krupp,

Zhong Liu, John Moses, Rama Ramapriyan, Donna Scott, Deborah Smith, Ron Weaver

29 June 2011 1AMSR-E Science Team Meeting Asheville, NC

Summary of Workshop GoalsA Passive Microwave Data Set Workshop was held 17-19 May 2011 at the Global Hydrology Resource Center, sponsored by the NASA Earth Science Data and Information System (ESDIS) Project. Data producers and data distributors met in order to organize and document passive microwave radiometer data sets available at NASA Earth Science Data Centers (ESDCs) with the purpose of better informing users of data set choices. We reviewed all primary (Level 1 – Level 3) passive microwave data sets from NASA and non-NASA sensors held by ESDCs or NASA-funded data producers.

Primary goals of the workshop were:•Determine if products are duplicative and, if so, why each specific product is important to a given user community.•Discuss possible changes to levels of service for any products without an active user community. For example, certain data sets may be maintained because they were inputs for generating a historical geophysical product.•Discuss ways in which we can more clearly describe and document the passive microwave data sets held by the ESDCs and other Data Providers funded by NASA. The intent is to harmonize semantics, formats, and documentation to the benefit of users.

29 June 2011 2AMSR-E Science Team Meeting Asheville, NC

Attendees

Anderson Mark U. NebraskaConway Dawn AMSR-EFishbein Evan JPLForsythe John Colo St / STCShie Chung-Lin UMBC / GESTSmith Deborah RSS

Krupp BrianMoses JohnRamapriyan* Hampapuram

Data Centers

Data Producers

ESDIS

* Participating remotely

Armstrong* Ed PO.DAAC Conover Helen GHRC Drewry Marilyn GHRC Goodman Michael GHRC Hardin Danny GHRC Harrison Sherry GHRC Leon Amanda NSIDC Liu Zhong GES DISC Meier Walt NSIDC Rinsland* Pamela ASDC Scott* Donna NSIDCWeaver Ron NSIDC

29 June 2011 3AMSR-E Science Team Meeting Asheville, NC

Data Providers Considered

• The various NASA Earth Science Data Centers (ESDCs):– Atmospheric Science Data Center (ASDC) – primarily focusing on atmospheric chemistry

products, but also holding some heritage passive microwave data– Global Hydrology and Climate Center (GHRC) – focusing on the atmospheric components

of the hydrologic cycle– Goddard Earth Sciences Data and Information Services Center (GES DISC) – archive of

record for the TRMM Microwave Imager products, among others – National Snow and Ice Data Center (NSIDC) – focusing primarily on cryospheric products,

AMSR-E archive of record– Physical Oceanography DAAC (PO.DAAC) – extensive holdings of sea surface temperature

and other ocean products• Other NASA-funded data production / distribution facilities:

– Precipitation Processing System (PPS) – processes, analyzes and archives data from the upcoming GPM and the current TRMM mission

– Remote Sensing Systems (RSS) - currently reprocessing all microwave satellite radiometers to a common calibration standard using their newest radiative transfer model

– Precipitation Research Group (PRG) at Colorado State University – currently developing a Fundamental Climate Data Record (FCDR) of SSM/I and SSMIS brightness temperatures.

29 June 2011 4AMSR-E Science Team Meeting Asheville, NC

Data Disciplines and Services Considered

Disciplines• Brightness Temperature Data Sets• Atmospheric Data Suites (Wind, Vapor, Cloud, Rain)

– Additional Precipitation Data Sets– Additional Wind Data Sets– Ocean Flux Data Sets

• Sea Surface Temperature Data Sets• Cryosphere / Ice Data Sets• Soil Moisture and Land Data Sets

Services• Subsetting, Data Extraction, Other Data Services

29 June 2011 5AMSR-E Science Team Meeting Asheville, NC

Summary of Data Holdings

Data Center

TypeBrightness Temps Atmos Rain SST

Cryo / Ice

Soil Moist / Land

ASDC ESDC X X

GHRC ESDC X X X

GES DISC ESDC X X X

NSIDC ESDC X X X X X

PO.DAAC ESDC X X

PPS SIPS X X X

RSS MEaSUREs (X) X X X

CSU PRG MEaSUREs X X

(X) Not publicly available

29 June 2011 6AMSR-E Science Team Meeting Asheville, NC

Tuesday, 17 May 8:30a Welcome and Introductions (M. Goodman) 9:00a Overview of Passive Microwave Radiometer products at each data center (RSS, GHRC, NSIDC, PO.DAAC, GES

DISC, ASDC) 1:00p Introduce data review process for workshop (M. Goodman) 1:15p Review data group 1: Atmosphere over Oceans (moderator D. Smith; scribe R. Weaver) 3:00p Review data groups 2-4: Other atmosphere and/or ocean products Wind (moderator H. Conover; scribe J. Moses) SST (moderator H. Conover; scribe B. Krupp) Precipitation (moderator M. Goodman; scribe B. Krupp)Wednesday, 18 May 8:30a Review data group 5: Brightness Temperatures (moderator R. Weaver; scribe H. Conover)

10:15a Review data groups 6 and 7: Cryosphere products, Land products (moderator W. Meier; scribe D. Smith)12:45p Other passive microwave data sources (moderator B. Krupp; scribe M. Goodman) NASA Passive Microwave data not previously covered NOAA Passive Microwave data holdings International Passive Microwave data holdings 1:30p Subsets, dynamic products, tools (moderator J. Moses; scribe M. Anderson) 3:00p Additional resources and data stewardship activities Data identifiers (A. Leon) Provenance and Context Content standard for archive (J. Moses) 3:45p Discussion - recommendations / future work (moderator M. Goodman; scribe H. Conover) Major conclusions, Recommendations overall and for each data group Future joint activities

Workshop Agenda

29 June 2011 7AMSR-E Science Team Meeting Asheville, NC

Summary of Meeting Outcomes:Duplication of data across data providers

• We only found a few duplicate data sets; however we did find lack of documented differences between some data sets which were therefore construed to be duplicative.

• Some passive microwave products are available from both the data producer and the archive of record. Generally, data producers make data available on a temporary basis to a smaller community and the archive of record provides longer-term archival, broader community distribution, and may provide higher levels of service. Examples include:– TRMM Microwave Imager products at PPS and GES DISC– MEaSUREs products from PI’s and the respective ESDCs

Draft Recommendation: Provide clear documentation and cross-referencing between these data sets. Digital Object Identifiers (DOIs) can help clarify whether data sets are identical.

29 June 2011 8AMSR-E Science Team Meeting Asheville, NC

Summary of Meeting Outcomes:Identification of source data and version

• We found several instances in which information about the source data set from which a product was derived and/or the algorithm and version used, was absent or not clear. Examples include:– Time series for which input data source or version may have changed

over time (e.g., gridded SSM/I Tb from NSIDC)– Imagery or analysis products for which input data sources are not

obvious (e.g., imagery produced by services like Giovanni) Draft Recommendation: For all NASA data products, lineage should

be clearly documented. In particular, Versions of both the data product and the algorithm(s) used to

generate it need to be verified, documented, and made more visible as necessary.

Source and version of data set(s) used as input to the product need to be identified, noting when the source data were published, and especially when the version of the source data has changed.

29 June 2011 9AMSR-E Science Team Meeting Asheville, NC

Summary of Meeting Outcomes:Documentation requirements

• We found a variety of levels of documentation among different PM data sets, especially between the EOS missions (AMSR-E, TMI) and some of the others (e.g., SSM/I).

• While ESDIS provides guidelines for data set documentation, they are not consistently applied. Existing guidelines include:

– Metadata requirements including GCMD DIF, ECHO entries, Guide documents– Templates for ESDC data product reviews– Emerging Provenance and Context Content Standard

Draft Recommendation: Compile a checklist of required documentation for data sets, based on existing requirements and guidelines. Review and update documentation for PM data sets, coordinating across ESDCs holding similar data. Consider tying documentation requirements to levels of service Review documentation for high value data sets first Maintain and update regularly

29 June 2011 10AMSR-E Science Team Meeting Asheville, NC

Summary of Meeting Outcomes:Old or out-dated products

• We found several data sets which have been or will be superseded by newer products at the same data center or another data center. Examples include:

– SSM/I Tb and atmospheric suite products from GHRC are based on the RSS v4 algorithm. RSS is now finalizing v7.– SSM/I ocean wind speed from DMSP-F8 available from JPL is a Pathfinder product, superseded by current data

sets from RSS.– SSM/I ocean wind vector products at JPL are superseded by Cross-Calibrated Multi-Platform (CCMP) Ocean

Surface Wind Components product currently being generated Draft Recommendation: In consultation with their User Working Groups, data centers should

consider common practices to handle data set retirement or sequestering. Users should be referred to the replacement product.

29 June 2011 11AMSR-E Science Team Meeting Asheville, NC

Status and Plans

• Workshop Committee is compiling a detailed report to include– Findings specific to each data discipline discussed (listed in

slide 5)– Specific recommendations for consideration by each ESDC

User Working Group

• All participants – ESDCs, MEaSUREs, ESDIS – found this exercise to be valuable and we recommend that other instrument /discipline groups host similar workshops

29 June 2011 12AMSR-E Science Team Meeting Asheville, NC

Backup Slides

1329 June 2011 AMSR-E Science Team Meeting Asheville, NC

Summary of Data HoldingsOrganization Type Sensors held Data Products

Atmospheric Sciences Data Center (ASDC)

NASA ESDC SMMR, SSM/I atmospheric products, ice/snow

Global Hydrology Research Center (GHRC)

NASA ESDC SSM/I, AMSR-E Tb, atmospheric products

Goddard Earth Sciences Data and Information Services Center (GES DISC)

NASA ESDCTMI, SSM/I, AMSR-E

Tb, precipitation, atmospheric products

National Snow and Ice Data Center Distributed Active Archive Center (NSIDC)

NASA ESDCSSM/I, SSMIS, AMSR-E, ESMR, SMMR

Tb, Cryosphere, land/soil moisture, precipitation, atmospheric products

Physical Ocean Distributed Active Archive Center (PO.DAAC)

NASA ESDCSMMR, SSM/I, AMSR-E, TMI

Sea surface temps, atmospheric products

Precipitation Processing System (PPS)

NASA SIPS TMITb, precipitation, atmospheric products

Remote Sensing Systems (RSS)

MEaSUREsSSM/I, SSMIS, AMSR, AMSR-E, TMI WINDSAT

Tb*, atmospheric products, sea surface temps

Colorado State University, Precipitation Research Group (CSU PRG)

MEaSUREsSSM/I, TMI, AMSR-E

Tb, precipitation

* Not publicly available

29 June 2011 14AMSR-E Science Team Meeting Asheville, NC