page 1 cost/esf school: utls, cargese, 3-15 october 2005 da 12: evaluation of future missions...
TRANSCRIPT
Page 1COST/ESF School: UTLS, Cargese, 3-15 October 2005
DA 12: Evaluation of future missions
Author: W.A. Lahoz
Data Assimilation Research Centre, University of Reading RG6 6BB, UK
Page 2COST/ESF School: UTLS, Cargese, 3-15 October 2005
•Why is it important to evaluate expensive future missions such as ESA’s Envisat
Quantify added value from new observations in comparison to Global Observing System
•Use of data assimilation:
Different approaches to evaluating future missions. Observing System Simulation Experiments (OSSEs) and Observing System Experiments (OSEs)
•Example of an OSSE: The proposed SWIFT instrument, measuring stratospheric winds and ozone
Topics:
Page 3COST/ESF School: UTLS, Cargese, 3-15 October 2005
Importance of evaluating future EO missions
Page 4COST/ESF School: UTLS, Cargese, 3-15 October 2005
Envisat: http://envisat.esa.int
Page 5COST/ESF School: UTLS, Cargese, 3-15 October 2005
You are given 2.3 BEuros for Envisat to observe the Earth System…
Hooray!
BUT…
How can you check if this is a good use of money?
How can you quantify value?
Page 6COST/ESF School: UTLS, Cargese, 3-15 October 2005
What do you need to consider?
NOT the value of Envisat
BUT added value of Envisat above what else will be available
-> INCREMENTAL VALUE
THIS IS TRUE FOR ANY ADDITION TO GOS
Page 7COST/ESF School: UTLS, Cargese, 3-15 October 2005
So…
What will the GOS be like?
Existing & planned satellite missions
What type of observations do we include?
Conventional: ground-based, sondes, aircraft
Satellites: operational, research
Page 8COST/ESF School: UTLS, Cargese, 3-15 October 2005
850 KM
3 5 8 0 0 K m SUBSATELLITE POINT
GOMS (Russian Federation)
76E
MSG
(EUMETSAT) 63 E
MTSAT (J apan)
140E
FY-2 (China)
105E
GOES-E (USA) 75W
NPOESS (USA)
GOES-W (USA) 135W
G E O S T
A T I O N A R Y
O R B I T
Global Earth Observing system for 2008-2010
Page 9COST/ESF School: UTLS, Cargese, 3-15 October 2005
Observation types used by Met Office for NWP
Page 10COST/ESF School: UTLS, Cargese, 3-15 October 2005
GOS:
Observing characteristics
Viewing geometries (limb/nadir; LEO/GEO)
Sonde & ground-based observations distribution
Aircraft corridors
Observation errors: random, systematic, representativeness
See DA 11
Page 11COST/ESF School: UTLS, Cargese, 3-15 October 2005
We have a description of future GOS
BUT…
How do we get values of a future instrument: obs & errors?
Need to sample the “truth” (nature)
-> Get from a model run, or analyses
Page 12COST/ESF School: UTLS, Cargese, 3-15 October 2005
• Lack of global observations of stratospheric winds in the current operational meteorological system:
• No sondes above 10 hPa (no global coverage anyway)• AMVs from satellites in troposphere• Wind information from temperature nadir sounders in extra-tropics
(troposphere/stratosphere)• But, thermal wind relation breaks down in tropics
• We have no good current estimates of state of the tropical stratosphere:
• Variability in the quasi-biennial oscillation (QBO) is underestimated• “Balanced” winds problematic for estimating varability of QBO Randel et al. (2002)
A current concern about GOS are winds
Page 13COST/ESF School: UTLS, Cargese, 3-15 October 2005
“Realistic” quasi-biennial oscillations in the MO Unified Model
MO observational analyses of equatorial winds for Nov 1992 - Jan 2000
Page 14COST/ESF School: UTLS, Cargese, 3-15 October 2005
• Recent past:
• UARS WINDII: mesospheric winds
• UARS HRDI: stratospheric winds, but impact marginal as observed winds not accurate enough compared to forecasts (Boorman et al. 2000)
• Future:
• ESA ADM-Aeolus: launch October 2007
• CSA SWIFT: launch 2010?
Missions measuring winds
Page 15COST/ESF School: UTLS, Cargese, 3-15 October 2005
ADM-Aeolus
Doppler Wind Lidar (DWL)
1 component global wind profiles up to ~30 kmN.B. need DA to get 2 components
•Better information to predict weather•Global wind profiles for the entire planet, including remote areas lacking any g-based weather station
Main objective:•Correct major deficiency in winds in current GOS
•Increased skill in NWP •Data needed to address WCRP key concerns:
Quantification of climate variabilityValidation & improvement of climate modelsProcess studies relevant to climate change
OSSEs done for ADM-Aeolus (e.g. Stoffelen)
Page 16COST/ESF School: UTLS, Cargese, 3-15 October 2005
Use of data assimilation
Page 17COST/ESF School: UTLS, Cargese, 3-15 October 2005
Philosophy:
Nature, Truth (T) -> sample to get all observations
Test impact of one observation (e.g. Envisat-like)
Control experiment (C): all observations less that of interest
Perturbation experiment (P): all observations
Compare C & P against T
QUANTIFY!
Page 18COST/ESF School: UTLS, Cargese, 3-15 October 2005
Space agencies (ESA, NASA, JAXA, CSA) invest a lot of money
on missions (e.g. ESA’s Envisat has cost 2.3BEuros) Important to evaluate beforehand possible benefits of
future missions, especially those involving satellites
Techniques for evaluating future missions:
A technique often used by the space agencies is the OSSE Information content (e.g. Prunet et al. QJ 98, IASI) Ensembles (e.g. Andersson et al. WMO 2005, ADM-Aeolus) OSSEs tend not be used as much by the met agencies
This is due to the shortcomings of OSSEs (see later) -> less attractive to the met agencies
Future EO missions
Page 19COST/ESF School: UTLS, Cargese, 3-15 October 2005
A technique often used by the met agencies to evaluate components of an existing observing system is the “Observing System Experiment” (OSE)
An OSE studies the impact of one observation type by removing it from the system under study
- test impact of satellite data for NH/SH - test impact of nadir/limb geometries
Note:
Page 20COST/ESF School: UTLS, Cargese, 3-15 October 2005
OSSE goal: evaluate if the difference P-T (measured objectively)is significantly smaller than the difference C-T
Simulated atmosphere (“truth”; T): using a model Simulated observations of instruments appropriate to the
study, including errors: using T Assimilation system: using a model Control experiment C: all observations except those under
study Perturbation experiment P: all observations
Structure of an OSSE
Page 21COST/ESF School: UTLS, Cargese, 3-15 October 2005
Note shortcomings of an OSSE:
Expensive (cost ~ assimilation system) -> alleviate problem: “reduced OSSE” (e.g. profiles instead of radiances)
Note: “reduced OSSE” generally only useful when observation of interest has relatively high impact (e.g. stratospheric winds)
Difficult interpretation (model dependence) -> alleviate problem: conservative errors, several methods to investigate impact
Incest -> alleviate problem: different models to construct “truth” & perform assimilation (BUT there could be bias between models)
Despite shortcomings, high cost of EO missions meansthat OSSEs often make sense to space agencies
Page 22COST/ESF School: UTLS, Cargese, 3-15 October 2005
Example of an OSSE
Page 23COST/ESF School: UTLS, Cargese, 3-15 October 2005
SWIFT: see Lahoz et al. QJ 2005
Based on UARS WINDII principle (Doppler effect)
2 wind components using 2 measurements at ~90o
Thermal emission (mid-IR) of ozone (1133 cm-1)
Technology difficult to implement
Global measurements of wind and ozone profiles (~20-40 km)
OSSE: evaluate proposed SWIFT instrument
Page 24COST/ESF School: UTLS, Cargese, 3-15 October 2005
1. Current observing system:
No operational observations of winds for levels above those of radiosondes (~10 hPa)
Note: indirect information on winds can be obtained from nadir soundings of temperature (thermal wind; but this breaks down in the tropics)
2. Science:
Help build climatologies of tropical winds
Transport studies (e.g. ozone fluxes)
Use assimilation to obtain 4-d quality-controlled datasets for scientific studies (e.g. climate change and its attribution)
Why SWIFT?
Page 25COST/ESF School: UTLS, Cargese, 3-15 October 2005
Establish basis for assimilating SWIFT observations (u, v; ozone) Investigate scientific merits of SWIFT observations
Models used: “Truth” (ECMWF directly, or forcing a CTM) Assimilation system (Met Office) (cf. incest)
Simulated observations:Operational: C {MetOP, MSG, sondes, balloons, aircraft,
surface}Temperature, winds, humidity, ozone
SWIFT; C+SWIFT = POzone, winds (stratosphere, conservative errors)
Several assimilation experiments; analyses evaluated. Qualitative & quantitative tests
Design of SWIFT OSSE
Page 26COST/ESF School: UTLS, Cargese, 3-15 October 2005
SWIFT: N - and S - observations (87°N-53°S, 53°N-87°S): non sun-synchronous orbit
- winds 16-50km, every 2km approximately
- ozone 16-44km, every 2km approximately
Errors: conservative; random; representativeness error considered to be relatively unimportant
SWIFT wind component error
0
10
20
30
40
50
60
0 5 10 15 20 25
error (m/s)
hei
gh
t (k
m)
SWIFT ozone error
10
20
30
40
50
0 5 10 15 20
error (as a percentage of the observation)
hei
gh
t (k
m)
SWIFT characteristics
Page 27COST/ESF School: UTLS, Cargese, 3-15 October 2005
Several tests -> robustness (cf. interpretation of the OSSE)
Qualitative (histograms, monthly means)
Quantitative (RMS statistics, significance tests)
Assumption:
We can discount the bias between the ECMWF and Met Office models because it is removed when comparing P-T and C-T
Same bias in both P-T and C-T differences which are compared
Evaluation of SWIFT impact
Page 28COST/ESF School: UTLS, Cargese, 3-15 October 2005
Y=Abs(C-T) -Abs(P-T); Zonal-wind (m/s); January 2000; Shaded:95% C.L. & Y>0. Similar results for April 2000.
10 hPa 1 hPa
Significance tests Areas > 5%
N.B. Some areas of -ve impact (new obs can degrade DA system) - not significant
Page 29COST/ESF School: UTLS, Cargese, 3-15 October 2005
SWIFT winds Significant impact in tropical stratosphere EXCEPT lowermost levels Can have significant impact in extra-tropics when:
SWIFT observations available Flow regime is variable (relatively fast changing)
Have scientific merit in that they improve: Information on tropical winds Wintertime variability
Useful for forecasting & producing analyses to help study climate change & its attribution:
better models, better initial conditions, model evaluation
SWIFT ozone Significant impact at 100 hPa & 10 hPa -> regions of relatively high vertical gradient
Conclusions
Page 30COST/ESF School: UTLS, Cargese, 3-15 October 2005
“Reduced OSSE”: Radiances used for AMSU-A, IASI at time of SWIFT launch
1. Expectation is that impact in tropics will not change2. Impact in the extra-tropics may remain unchanged (impact in
flow regimes which change relatively fast)
Thermal wind relationship does not hold for 1, and is not accurate in 2
Caveats
Page 31COST/ESF School: UTLS, Cargese, 3-15 October 2005
Higher horizontal resolution-> Less “thinning” of satellite data (reduce number of obs): AMSU-A, IASI
Would impact stratospheric wind analyses in the extra-tropics
Conclusions in tropics should be robust
Conclusions for ozone analyses (100 & 10 hPa) should not change
NOTE: the SWIFT OSSE did not evaluate forecasts (only analyses) and did not calibrate the OSSE by, e.g., removing simulated sonde data & replacing it with real sonde data
Caveats
Page 32COST/ESF School: UTLS, Cargese, 3-15 October 2005
You might think OSSEs are a load of… (fill in words)
If so help is at hand!
• Information content
• Ensembles
Alternatives
Page 33COST/ESF School: UTLS, Cargese, 3-15 October 2005
Aim: evaluate the potential benefit of future sensors compared to other available sensors
Prunet et al. (1998) used approach to quantify impact of information content in simulated IASI radiances vs information content in TOVS radiances.
Impact of IASI radiances estimated by comparison of analysed errors (which include TOVS or IASI data) vs those of a background field (from model forecast excluding both TOVS and IASI data).
If observation type of interest has positive impact, analysed errors should be smaller than background errors.
By comparing errors of analyses including TOVS or IASI data, relative information content in these data can be evaluated, and assessment made of their relative benefit.
Information content
Page 34COST/ESF School: UTLS, Cargese, 3-15 October 2005
In principle, information content approach is simpler & less expensive to apply than an OSSE.
However, information content approach requires a realistic characterization of background & observation errors, which could be difficult to achieve.
Furthermore, it could be argued that OSSE approach provides a more complete test of the future sensors.
Information content
Page 35COST/ESF School: UTLS, Cargese, 3-15 October 2005
Ensembles Courtesy Andersson et al.ECMWF
Page 36COST/ESF School: UTLS, Cargese, 3-15 October 2005
Ensembles Courtesy Andersson et al.ECMWF
Page 37COST/ESF School: UTLS, Cargese, 3-15 October 2005
Ensembles Courtesy Andersson et al.ECMWF
According to Andersson et al:
Need “truth” for simulated data
Page 38COST/ESF School: UTLS, Cargese, 3-15 October 2005
Important to quantify value of future missions
-> participation of all actors: multi-disciplinary
-> quantify benefits: OSSEs & other methods
-> caveats: set up experiments carefully
Should there be a dedicated OSSE facility?
Way forward: