r. reichle 1 * and c. draper 1,2 with contributions from:
DESCRIPTION
AMSR-E Technical Interchange Meeting Oxnard, CA Sep 4-5, 2013 Soil moisture retrievals from AMSR-E and ASCAT: Error estimates and data assimilation. R. Reichle 1 * and C. Draper 1,2 With contributions from: G. De Lannoy, R. de Jeu, Q. Liu, V. Naeimi, R. Parinussa, W. Wagner. - PowerPoint PPT PresentationTRANSCRIPT
R. Reichle1* and C. Draper1,2
With contributions from: G. De Lannoy, R. de Jeu, Q. Liu, V. Naeimi, R. Parinussa, W. Wagner
AMSR-E Technical Interchange MeetingOxnard, CA
Sep 4-5, 2013
Soil moisture retrievals from AMSR-E and ASCAT:
Error estimates and data assimilation
1NASA/GSFC, Greenbelt, MD, USA2Universities Space Research Association, Columbia, MD, USA
*Email: [email protected]
Root Mean Square Error (RMSE) needed • to validate against mission target accuracies, and • to specify observation error for data assimilation.
Difficulty: No recognized soil moisture truth at large scales.
Motivation
• Point-scale (e.g., SCAN): Representativity errors when comparing against satellite estimates.
• Footprint-scale (e.g., USDA ARS): Limited spatial coverage.
RMSE traditionally from validation against in situ observations:
Error estimates for soil moisture retrievalsTriple collocationError propagation
Soil moisture data assimilation
Outline
How can we estimate RMSE in satellite soil moisture retrievals?
Approach
Two methods:• Triple collocation• Error propagation through remote sensing retrieval algorithms
Two remotely sensed soil moisture datasets:• AMSR-E (X-band) Land Parameter Retrieval Model (de Jeu and Owe, 2003)• ASCAT C-band TUWien empirical change detection (Wagner et al., 1999)
Experiment details:• Jan 2007 - Oct 2011• North American domain
Triple Collocation (TC) is a method to estimate RMS errors.
TC requires three independent estimates.
NOTE:1.) TC cannot provide absolute RMS errors because it does not
estimate the bias.2.) For soil moisture, we found TC to work only for “anomalies”
from the seasonal cycle.3.) TC is sensitive to the climatology of the chosen reference
data set.
Triple Collocation
Triple Collocation
Surface soil moisture datasets: (converted to anomalies from the mean seasonal cycle)• θ(A): ASCAT• θ(L): AMSR-E LPRM• θ(C): Catchment model
Additive random error model (θ = truth, < ϵ(X) >= 0)θ(A) = α( θ + ϵ(A))θ(L) = λ(θ + ϵ(L))θ(C) = γ(θ + ϵ(C))
α, λ, γ are calibration constants (rescale to account for systematic differences in variability between datasets)
Stoffelen (1998), Scipal et al. (2008), Miralles et al. (2010)
Triple Collocation
Set θ(A) as the reference dataset (α=1) subscript “A” for estimated calibration constants and estimated errors.
Solve for calibration:• λA ≈ < θ(L) θ(C) > / < θ(A) θ(C) >
• γA ≈ < θ(L) θ(C) > / < θ(A) θ(L) >
Solve for ϵ2 = (RMSETC)2:• ϵA
2(A) ≈ < (θ(A) – θ(L)/λA) · (θ(A) – θ(C)/γA) >
• ϵA2(L) ≈ < (θ(L)/λA – θ(A) ) · (θ(L)/λA – θ(C)/γA) >
• ϵA2(C) ≈ < (θ(C)/γA – θ(A) ) · (θ(C)/γA – θ(L)/λA) >
Assumption: Errors are not correlated with each other, nor with the true state.
RMSE(X)2 = var(X) + var(true) – 2 R sqrt( var(X) var(true)) + bias2
Interpretation of large-scale soil moisture RMSE
• “Because vartrue differs from place to place […] a single global RMSE target may not be appropriate.” (Entekhabi et al., 2010).
• At the global scale, we do not know vartrue (or even the mean):
Information on agreement is in time series correlation coefficient R.
Comparing soil moisture data sets using TC requires rescaling them to match the climatology of a reference data set.
RMSETC estimates will depend on the climatology of the chosen reference data set.
Variability (anomalies)
ASCAT
AMSR-E/LPRM
GEOS-5
RMSE of ASCAT (anom)
[Ref dataset = ASCAT]
[Ref dataset = AMSR-E]
[Ref dataset = GEOS-5]
Triple Collocation
RMSE and Fractional RMSE (fRMSE)
Spatial differences in variability introduce nonlinearities: the ratio (and ranking!) of the domain-average RMSETC in m3m-3 depends on the selected reference:
Solution: Report errors as fractional RMSE:
fRMSE = RMSEX(X) / var(X)
Variability (anomalies)
ASCAT
AMSR-E/LPRM
GEOS-5
RMSE of ASCAT (anom)
[Ref dataset = ASCAT]
[Ref dataset = AMSR-E]
[Ref dataset = GEOS-5]
Fractional RMSE
ASCAT
AMSR-E/LPRM
Draper et al. 2013, RSE, in press
Estimated RMSE reflects variability of reference product.Use fractional RMSE instead.
Triple Collocation
Uncertainty estimates indicate where fRMSETC is reliable.
ASCAT fRMSETC AMSR-E fRMSETC
Width of 90% confidence interval for ASCAT fRMSETC
Width of 90% confidence interval for AMSR-E fRMSETC
Triple Collocation
Draper et al. 2013, RSE, in press
fRMSETC estimates can provide insights into relative skill.Consistent with expectations.
Triple Collocation
Draper et al. 2013, RSE, in press
• fRMSETC is reasonably insensitive to choice of data triplet.• Residual differences between fRMSETC estimates are related
to error correlations between data products.
Triple Collocation Assumptions
Draper et al. 2013, RSE, in press
In situ “errors” are not small! (scale mismatch, sensor issues,...)
Assumption: Errors not correlated across datasets and with the truth.Check by adding a 4th dataset (in situ).
Previous finding holds only if using anomalies from the mean seasonal cycle!
Triple Collocation Assumptions
Error estimates for soil moisture retrievalsTriple collocationError propagation
Soil moisture data assimilation
Outline
Error propagationPropagate expected errors in input observations and parameters through the soil moisture retrieval model:• ASCAT (Naeimi et al. 2007)• LPRM (Parinussa et al. 2011)
Used average of the error time series at each location. Assumed to represent errors in the soil moisture anomalies.
TC and EP yield reasonably consistent fRMSE patterns.
ASCAT fRMSETC AMSR-E fRMSETC
Triple Collocation (TC) and Error Propagation (EP)
Draper et al. 2013, RSE, in press
ASCAT fRMSEEP AMSR-E fRMSEEP
Triple Collocation
fRMSETC is consistent with:
• fRMSE from error propagation
• expectations (vegetation classes)
Draper et al. 2013, RSE, in press
Error estimates for soil moisture retrievalsTriple collocationError propagation
Soil moisture data assimilation
Outline
Improvements from data assimilation
Validated with in situ data
Anomalies ≡ mean seasonal cycle removed
Skill increases significantly through data assimilation. Similar improvements from AMSR-E/LPRM and ASCAT.
Metric: Anom. time series corr. coeff.
Draper et al. (2012), GRL, doi:10.1029/2011GL050655.
Improvements from data assimilation
Skill increases significantly through data assimilation. Similar improvements from AMSR-E/LPRM and ASCAT.
Draper et al. (2012), GRL, doi:10.1029/2011GL050655.
Consistent with similar skill levels for the AMSR-E and ASCAT retrievals themselves….…except for terrain with high topographic complexity (TC>10%).
Root zone soil moisture skill improvement from assimilation over model (ΔR)
As long as (obs skill − model skill) > −0.2, assimilation improved the model skill.
Draper et al. (2012), GRL, doi:10.1029/2011GL050655.
Reichle et al. (2008), GRL, doi:10.1029/2007GL031986.
Improvements from data assimilation
Synthetic experiment
Both triple collocation and error propagation can estimate spatial patterns in the fRMSE.• Good agreement with each other, and with expectations (based on
vegetation, known problems)• Triple collocation is believed to provide correct magnitude, and method is
robust (dependent on careful selection of data sets, use of anomalies from the seasonal mean)
Conclusions (1/4)
How should we evaluate remotely sensed soil moisture globally?
• Substantial spatial variation in RMSE and fRMSE across North America• Evaluation based on handful of in situ sites may not represent global
accuracy• Triple collocation or error propagation could provide a useful complement
• It is unclear how current RMSE target accuracies can be interpreted globally• Selection of reference determines magnitude of RMSE (and domain-
average RMSE has non-linear dependence on reference)• Uniform RMSE target over large domain not sensible• Need alternative metrics (fRMSE, or see Crow et al. (2010), Entekhabi et
al. (2010))
Conclusions (2/4)
Data assimilation:• Assimilation of soil moisture retrievals improves model estimates of surface and root zone soil moisture.
• AMSR-E and ASCAT retrievals have comparable skill and yield similar improvements after data assimilation.
• Improvements can be realized even when the retrievals are (somewhat) less skillful than the model estimates.
Conclusions (3/4)
How should we specify soil moisture observation error variances for data assimilation?• Usually specified as a
constant in the observation climatology.
• Uniform error variance not sensible, and resulting spatial patterns do not resemble other estimates.
• Use a uniform fRMSE, or spatial maps from triple collocation or error propagation (latter gives temporal variability).
Conclusions (4/4)
Thank you for your attention.
Questions?
References:Draper, C. S., R. H. Reichle, R. de Jeu, V. Naeimi, R. Parinussa, and W. Wagner (2013), Estimating root mean square errors in remotely sensed soil moisture over continental scale domains, Remote Sensing of Environment, in press.
Draper, C. S., R. H. Reichle, G. J. M. De Lannoy, and Q. Liu (2012), Assimilation of passive and active microwave soil moisture retrievals, Geophysical Research Letters, 39, L04401, doi:10.1029/2011GL050655.
Back-Up Slides