CDIAC AmeriFlux Data Management

Tom Boden (Bai Yang)Carbon Dioxide Information Analysis Center (CDIAC)

Oak Ridge National Laboratory (ORNL)

Presented September 27, 2010 at theCDIAC User Working Group Meeting

Oak Ridge National Laboratory

OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY

2

AmeriFlux Network ~140 sites in 5 countries; 92 active

sites, 48 inactive sites

Participation Requirements

Make year-round core measurements using the eddy-covariance technique

Submit data to CDIAC within 1 year of collection

Participate in AmeriFlux Science Meetings and synthesis/modeling activities

DOE supports the AmeriFlux infrastructure including the chairperson, calibration laboratory, and a permanent data archive (CDIAC since 1997).

Architecture of Global / Regional Flux Networks

Tower Sites

Regional Networks

CARBOEURO- FLUX

Americas Europe

AmeriFluxCanadian CP

AmeriFluxCanadian CP AsiaFluxAsiaFlux OzNetOzNet

Asia - Japan Oceania

Non-network

sites

Global Network

F L U X N E T- a network of networks

F L U X N E T- a network of networks

Primary FY2010 CDIAC AmeriFlux Team

Misha Krassovski – data system - (0.9 FTE)

Bai Yang – micrometeorologist [QA/QC] - (0.8 FTE)

Barbara Jackson – SAS programming - (0.7 FTE)

Ben Norton – web & DB maintenance - (0.5 FTE)

Tom Boden – coordination - (0.3 FTE)

Lianhong Gu – A-Ci analysis – (0.04 FTE)

Flux/MetData

BiologicalData

30 min Averages

BADM

Quality FlagsGap-filledNEP/Re

AmeriFlux Data Processing & Products

Level 0 Level 1 Level 2 Level 3 & 4

Site Investigator & Team CDIAC FLUXNET

CADM

Network-wide DatabaseStandard Files

QA/QCSiteProc.

SiteProc.

6

Level 1 data received

from AmeriFlux site PIs

Perform QA/QC

Flux, Meteorological, and Ecological Data Files • Stored “as received” from investigator • File format, parameter names, and units unchanged

Flux, Meteorological, and Ecological Data Files • Stored “as received” from investigator • File format, parameter names, and units unchanged

Calculateadditional

parameters, add to network

database

Incorporate into Web data interface & produce standardized data files & metadata

reports

Incorporate into network-wide database• Convert into standard AmeriFlux units, reporting intervals, and naming

conventions• Generate core parameters (e.g., VPD) where needed

Incorporate into network-wide database• Convert into standard AmeriFlux units, reporting intervals, and naming

conventions• Generate core parameters (e.g., VPD) where needed

Web access to network-wide database• Querying capabilities – time, measurement parameter, and site • Graphics generation and data retrieval capabilities

FTP access to standard files• Standard files with 40 core variables (csv, NetCDF, tar)• Metadata files (html & txt formats)

Web access to network-wide database• Querying capabilities – time, measurement parameter, and site • Graphics generation and data retrieval capabilities

FTP access to standard files• Standard files with 40 core variables (csv, NetCDF, tar)• Metadata files (html & txt formats)

Add value through automated QA/QC • Range checks, missing values, check solar elevation and time stamp• Basic Statistics (mean, variance, range)• Gap Analysis: determine frequency and duration of gaps in record• Spike detection, stationarity, diurnal and seasonal pattern• Cross checks/correlations (Rg vs. PAR)• Graphical Analysis: time series, property vs property, frequency

Add value through automated QA/QC • Range checks, missing values, check solar elevation and time stamp• Basic Statistics (mean, variance, range)• Gap Analysis: determine frequency and duration of gaps in record• Spike detection, stationarity, diurnal and seasonal pattern• Cross checks/correlations (Rg vs. PAR)• Graphical Analysis: time series, property vs property, frequency

Level 1 to Level 2 Data Processing SchemeL1 files available from FTP area

L2 data products available via Web interface and from FTP area

| |

5 days

| |

Progress in gap-filling meteorological data

Completed. We have completed a prototype data set of gap-filled meteorological data for the majority of active AmeriFlux sites.

Ongoing. This data set has been evaluated against other data sets filled by different techniques. The evaluation will continue if necessary and the sensitivity of ecosystem models to the gap-filled data sets will be estimated.

Ongoing. We are improving our gap-filling methods by including reanalysis data sets (NARR, DayMet, etc) and radar precipitation data. Present focus on filling long gaps and improving data reliability for precipitation, radiation, soil moisture and soil temperature.

Goal. We hope to finalize the gap-filled met-data sets before the next AmeriFlux annual meeting (Feb. 2011).

8

AmeriFlux Data System

Site & PI Information, Pubs,

& Instruments12/2008

Reported Variables

Flux & Met/Bio1/2009 3/2009

Flux & Meteorological

DataCommenced 8/2009 Leaf

Aci

3/2010

BiologicalData

BADM11/2009

Metadata(e.g., Fc corrections, instrument calibrations FY 2011

Users WorldwidePHP Interface

ORNL AmeriFlux & FLUXNET

Data Team PHP Interface

CDIACMS SQL 2005

RDBMS

Gu EDO methodology & codeWeb interface/Sharepoint for data submission and dissemination of results

http://ameriflux.ornl.gov http://ameriflux.ornl.gov/edit

FY2010 Priorities Data extraction/e-mail notification Load remaining L2 data, begin loading L4 data Web migration Inclusion of L2 & L4 data in the Earth System Grid Biological data editing functionality

http://leafweb.ornl.gov

Examples of CDIAC AmeriFlux Contributions to Synthesis & Modeling Activities

La Thuile, Italy workshop and resulting data sets

NACP Interim Site Synthesis

C-LAMP

La Thuile, Italy FLUXNET WorkshopFebruary 18-22, 2007

Participation and data response– ~60 participants, good representation from regional

networks and major programs (LBA, TCOS) – 921 site yrs of data from 240 sites worldwide (33

countries)– 318 site yrs of data from 77 AmeriFlux sites

Resulted in ~60 proposed papers– http://www.fluxdata.org

Workshop highlighted the importance of biological/ecological data and consistent ancillary information (ecosystem classifications)

Mandatory data:NEE/FCCO2/SFCU*/TAURg/PARTaH2O/RH/VPD

C-LAMP The Carbon-Land Model Intercomparison

Project (C-LAMP) consists of an experimental protocol, model evaluation metrics, a prototype diagnostics package, model output standards, and a database of model simulation results on the Earth System Grid (ESG). See http://www.climatemodeling.org/c-lamp

In the first set of runs, over 16ky of simulation and 50TB of output were generated using the Climate Science End Station INCITE allocation at ORNL.

CLM3.1 (Community Land Model) combined with CASA´ (Carnegie-Ames-Stanford Approach) and CN (carbon & nitrogen) biogeochemistry modules were evaluated against best-available satellite- and ground-based measurements, and new runs are being used to evaluate CLM4 performance.

C-LAMP will serve as a benchmarking prototype for Intergovernmental Panel on Climate Change (IPCC) Fifth Assessment Report (AR5) in conjunction with ESG.

Randerson, J. T., F. M. Hoffman, P. E. Thornton, N. M. Mahowald, K. Lindsay, Y. H. Lee, C. D. Nevison, S. C. Doney, G. B. Bonan, R. Stöckli, C. C. Covey, S. W. Running, and I. Y. Fung. “Systematic Assessment of Terrestrial Biogeochemistry in Coupled Climate-Carbon Models.” Global Change Biology, in press.

AmeriFlux Data Are An Important Component of C-LAMP

C-LAMP diagnostics include model comparisons with MODIS (Moderate Resolution Imaging Spectroradiometer) leaf area index (LAI) and net primary production (NPP), Globalview amplitude and phase, AmeriFlux Level 4 energy and carbon fluxes, Free Air CO2 Enrichment (FACE) site measurements, and various estimates of carbon stocks and transient dynamics.

Shown at right is a time series comparison of net ecosystem exchange (NEE), incoming shortwave, latent and sensible heat, gross primary production (GPP), and ecosystem respiration from CLM3.1-CN against measurements from the Morgan Monroe site.

C-LAMP diagnostics must be run before any model changes to the CLM biogeophysics or biogeochemistry can be made.

13

Future Directions & Emphasis?

Gap-filling meteorological data Biological data

QA/QC of existing submissions Improved parameterizations for models Expanding LeafWeb Amassing key measurements (e.g., soil respiration)

Flux re-analysis product within the ORNL SFA framework FY 2011 DOE BER “AmeriFlux Center” solicitation Collaborations with NEON