estimation of surface pressure & wind observation errors from era-20c observation feedback

ESTIMATION OF SURFACE PRESSURE & WIND

OBSERVATION ERRORS FROM

ERA-20C OBSERVATION FEEDBACK

Paul Poli

Poli, Observation errors in ERA-20C, EMS 2013

Outline

• Why we care about observation errors• Handling of– Gross errors– Systematic errors– Random errors– And lessons learnt from ERA-20C

• Conclusions


Reanalysis aims to objectively reconstruct a history of the past

“Model only” integration

“Observations-only” climatology

Reanalysis

Continuous, as realistic as the model and forcings

1) Immediate: improve upon estimates made at the time, by bringing together & recovering measurements from various sources, benefit from state-of-the-art analysis methods2) Longer-term: expose discrepancies = issues to be resolved, to improve the agreement between observations and models, understand uncertainties

“outliers”

Benefits


Most global (re)analysis methods rely (more or less) on least-squares statistical linear estimation

𝐱𝒂=𝐀 𝐲𝟎

𝐲𝟎=𝚪 𝐱𝒕+𝜻

E (𝜻 )=𝟎 E (𝜻 𝜻 𝐓 )=𝚺

𝐀=[𝚪𝐓𝚺 −𝟏𝚪 ]−𝟏𝚪𝐓𝚺 −𝟏

Observations

True stateObservation matrix

Observation errors

Observations are unbiased Observations error covariances are known

(e.g., Talagrand, 1997)

Mapping the information from the observations to estimate the system state happens through the observation matrix and the observation error covariance matrix


In practice• We use sequential or variational schemes, which assume a prior estimate

– Obtained by propagating / forwarding in time with a (imperfect) forecast model another prior (imperfect) estimate typically valid at an earlier time

– Makes the problem solvable (whole atmospheric state >>106 variables) with limited observations

– This apparent simplification further complicates things… – … because we then need to estimate errors in these first-guess states

• These errors aren’t separable analytically in their sources (between observation, model, forcings) because the information propagates in time and space and across the geophysical variables

• So… the theory very quickly becomes much more complicated

• In any case, whatever the analysis method, the presence of S (sometimes written R) in the earlier slide always remain:

Uncertainties of past (atmospheric) observations are essential when deriving quantitative estimates of past (atmospheric) states.


Illustration with actual data: surface pressure observations on 4-5 June 1900


Errors, mistakes, biases…• Mistakes and errors are not the same in our jargon

– Mistakes or obvious outliers (whatever this means) are nicely referred to as “gross errors”

• Another, also somehow philosophical separation, is also adopted between– The systematic parts of the errors

• Generally called observation biases• Although what is systematic in some restricted population may appear as random in a

larger population– The random parts of the errors

• The standard deviations of the observation errors being abusively called ‘observation errors’

• We will make no attempt to estimate the observation error distributions– Although we will generally assume that they are ~Gaussian


ERA-20C summary slide• ECMWF reanalysis, covering years 1899-2009• Assimilating ISPD 3.2.6 and ICOADS 2.5.1 observations

– surface pressure (ocean, land)– near-surface winds (ocean)

• Data assimilation system– Ensemble of 10 members– 24-hour 4DVAR– Self-adjusting background errors over time and space– Products 3-hourly, 91 model levels, ~125 km horiz. resolution– Forced by HadISST2.1.0.0 ensemble

• Data to be copied on public data server within a few months,along with observation feedback (containing obs. input)

[hPa]


4 Quality Control steps to (hopefully) remove all occurrences of gross errors in ERA-20C

1. Blacklist quality control; rejects observations:– From sources other than ISPD 3.2.6 or ICOADS 2.5.1– Reporting variables other than surface pressure (or geopotential) and near-surface wind– Reported exactly at location exactly 0◦N 0◦E (except for buoy observations because there is a buoy from the Pirata network there)

– Of wind above the land surfaces or near the coastlines (such observations could be assigned land surface roughness)

– Of wind reported too far above the model surface height (such observations are then located in closed seas in mountainous regions)

– Of surface pressure reported too much below the model surface are rejected2. Redundancy quality control; rejects observations:

– Of surface geopotential if there is a corresponding surface pressure observation– Of pressure reported at sea-level if there is a corresponding report at station level

3. Background or first-guess quality control; rejects observations:– With departures from the first-guess (+3 to +27-hour forecast initialised from previous analysis)

larger than a threshold: 18 * ( so2 + sb

2 ) 1/2

– Since sb (background error) varies over time and space, the rejection threshold also varies– For surface pressure: regional maxima (minima) of 310 hPa (20 hPa) in 1901, 175 hPa (17 hPa) in 2008.– For wind observations, the rejection thresholds are also extremely permissive, but an added criterion

removes observations reporting or in regions of wind speed greater than 35 m/s.– Overall, less than 1% of the observations are rejected by the first-guess check.

4. Variational quality control (Andersson and Järvinen, 1999), using the Huber norm (Tavolato and Isaksen, 2010); rejects observations:– That cannot be fitted within reasonable limits by the first minimisation– About 2% (14%) of the surface pressure (wind) observations are typically rejected, more in 1900, less in

2000. This rate increases as the observation time is located towards the end of the analysis window


Impact of observation quality controls

Hardly any rejections for ‘other causes’ (first-guess check nearly always passed)


End March 1954, Southern Hemisphere30 March 1954, 00 UTC 31 March 1954, 00 UTC 31 March 1954, 03 UTC

30 March 1954, 06 UTC 31 March 1954, 09 UTC 31 March 1954, 12 UTC

Southern Hemisphere


Hourly data from this station indicate surface pressures ~800 hPa, at 51m above msl

Date time obsvalue (Pa) O-B (Pa) O-A (Pa)(…)

19540330 10000 82040.000000 -18152.878906 -18208.525391 19540330 20000 82010.000000 -18265.691406 -18324.767578 19540330 30000 81950.000000 -18398.414062 -18456.244141 19540330 40000 81930.000000 -18483.009766 -18535.792969 19540330 50000 81880.000000 -18590.810547 -18646.716797 19540330 60000 81850.000000 -18675.292969 -18737.505859 19540330 70000 81860.000000 -18713.806641 -18790.753906 19540330 80000 81790.000000 -18828.126953 -18927.904297 19540330 90000 81760.000000 -18906.607422 -19023.998047

19540330 100000 81700.000000 -19157.095703 -19037.751953 19540330 110000 81710.000000 -19189.607422 -19114.912109 19540330 120000 81640.000000 -19249.113281 -19148.837891 19540330 130000 81610.000000 -19230.953125 -19113.726562 19540330 140000 81470.000000 -19317.041016 -19218.109375 19540330 150000 81470.000000 -19261.933594 -19192.337891 19540330 160000 81410.000000 -19259.660156 -19195.234375 19540330 170000 81340.000000 -19253.533203 -19209.138672 19540330 180000 81300.000000 -19224.179688 -19227.476562 19540330 190000 81210.000000 -19272.863281 -19283.312500 19540330 200000 81250.000000 -19214.863281 -19180.957031 19540330 210000 81240.000000 -19199.351562 -19124.273438 19540330 220000 81100.000000 -19295.736328 -19209.656250

(…)


Some information about this island

Information and photo from http://www.sthelena.se

“In 1961 a dramatic volcanic eruption forced the evacuation of the entire island. They were taken to what we glibly refer to as 'civilisation'. Almost all chose

to return to the island when the eruption was over.”


Observation biasesIn ERA-20C, we use a variational bias correction method for surface pressure.

The observation biases are estimated within the analysis, using prior bias estimates:

Background constraint Observation constraint

Estimate the 3D atmospheric fields and the bias parameters

Observation operator and observation bias model simulate the observations

For each analysis, construct a cost function and find its minimum:

(z)hyR(z)hyz)(zBz)(zJ(z) 1Tb

Tb

~~minmin 1 z

βxbxhzh ,~

TTT βxz

zB RBackground error covariances Observation error covariances


Maps of surface pressure observation bias estimates

1906

2006


Time-series of surface pressure observation bias estimates

Reminder: the number of observations increases by a factor ~50 in the time-frame above


Overall observation bias estimates become closer to zero over time: interpretation

• Assuming these estimates are correct– Are the observations now better calibrated, with more

recurrent procedures?– In particular, for the ships, are the methods to correct for the

height of the barometer more accurate?

• If these estimates of observation bias contain in fact model bias:– Is the model bias (constant over the years, but could be time

and space-dependent) more corrected by a greater number of observations in the assimilation?


Model bias problem in ERA-20C

• Model time-step of 60 minutes was employed instead of 30 minutes (CPU cost constraints)

• This time-step is insufficient to produce the right phase in the representation of atmospheric tides

• As a result, systematic increments are produced at the same time of day (which can be seen here by looking here around the equator (10S-10N) as a function of longitude)

Only in the region 10S-10N

Mean surface pressure analysis incrementsIn 1900

[hPa]


Estimating the observation error (standard deviation)

• From Desroziers et al. (QJRMS, 2005)

– Under hypotheses of optimality in the analysis and separation of scales between background and observation

– Also, if we haven’t fully removed the observation biases, they may creep into the results

• There are other methods– Usually also assuming separation of scales

Tba hhE xyxyR 00


Observations in the ERA-20C feedback

Hereafter, we ignore report types with fewer than 100 obs. per year


Surface pressure observation error estimates


Interpretation of surface pressure observation error estimates

• Land stations:– Observation quality appears to improve

over the century, from 1.6 hPa in 1900 to about 0.8 hPa in the years 2000s (assumed estimate was 1.1 hPa)

– This improvement is in two steps:• 1.6 1.0 hPa pre-World War II,• 1.0 hPa 0.8 hPa in the years 2000s

– Possibly a mixture of factors: generally improved instrumentation and site installations, more regular calibration practices and facilities, and also probably more accurate reporting of the observation time and automatic reporting with fewer errors in reading the instruments (?).

• Buoys:– Of worse quality in the early years of

introduction (in the 1970s); the location error may play a large role in this total error estimate.

– In the recent times, of similar quality as from land stations

• Ship:– Observation quality appears to improve over

the century, from 2 hPa in 1900 to about 1.2 hPa in recent years (assumed estimate 1.5 hPa)

• Ocean bottle and Conductivity Temperature Depth (CTD), eXpandable CTD (XCTD), Mechanical or digital or micro Bathythermograph (MBT), Expandable Bathythermograph (XBT):– Estimates based on small yearly samples – Nevertheless, estimates are substantially larger

than for other report types and would need to be understood in future repeats of similar reanalysis.

• Tropical cyclone bogus– About near-constant estimates around 4–5 hPa– Could reflect representativeness error & the

poor horizontal resolution of ERA-20C.• Meteorological vessels:

– In the 1980s, their quality approaches that of land stations

– Better known position of the ship, more regular observing times closer to the schedule time, more regularly recalibrated instrumentation, more trained onboard personnel (?)


Automatic stations (from the ERA-Interim feedback)

• Land SYNOP • Automatic land SYNOP only

Estimated from ERA-Interim

Assumed in ERA-Interim

*

Pressure observation error estimates with respect to station altitude

• Estimates for values reported at station level show increased errors in altitudes– Representativeness issues in mountains

• Estimates for values reported at sea level (using some formula*) show much increased height dependence– Reduction to sea-level introduces errors

• Pressure reduction formulae have been discussed for a while (WMO, 1954; 1964; 1968).• As of early 2013, the World Meteorological Organisation recommends a single practice only for stations

below 750 m altitude (WMO Commission for Instruments and Methods of Observation Expert Team on Standardisation, 2012).

• For stations located at altitudes above 750 m, the reduction methods are still a matter of regional choices. The resulting differences reach a few hPa for high-altitude stations.


4000m

3000m

2000m

1000m

0m


Wind component observation error estimates


Interpretation of wind component observation error estimates

• Ship:– Around 2.2 m/s per component– Quality appears to increase between WWI & II– Quality appears to degrade from the 1970s– Could this be related to the greater size of the ships? (i.e. not

measuring what we think we are using – not ’10-meter’ wind anymore)

• Buoys:– Would appear to have the ‘best’ quality– Could be because their measurement height is the closest to

expected?


Interpretation of wind component observation error estimates

• General remark about observation representativeness error– Above ocean, the error structures caused by non-representation of fine-scale

wind, even if transient in principle, may look the same for all the ships and buoys because the background is produced by the same (low-resolution) model

– Consequently may the diagnostic used here assign representativeness part of the observation error to the background error?

– In such case, the estimate computed here would thus be an under-estimation

• Some ideas to go further– Estimate wind speed and wind direction errors, instead of zonal and

meridional component estimates– Retrace how wind estimation methods evolved over the years


How reliable are those estimates?

• Quick answer: We can’t really tell right away!• Let’s look at regional disparities

For ship reporting observations of mean sea level pressure, from ICOADS 2.5.1, after assimilation in ERA-20C

Poli, ERA-20C Data Assimilation System, EMS 2013

29

How useful are these revised (larger) observation error estimates?

Surface pressure at Montreal, QuebecObservations from ISPD 3.2.6, collection #3004 (Canadian Stations Environment Canada )

Background forecast, with uncertainties in the model and its forcings (HadISST2.1.0.0 ensemble)

Observationswith uncertainties(some could not be fitted – they are VARQC rejected)

Analysis, with uncertainties

ERA-20C

Same observationsbut with LARGER uncertainties No obs. rejected

Background forecast, Initial state (and subsequent ones) presents LARGER uncertaintiesAnalysis, presents LARGER uncertainties

Increased sigo


Conclusions• Nothing is perfect…

… The observations… Our procedures to detect gross errors… Our estimations of observation biases… Our assumptions about observation errors

• A reanalysis provides the means to realize all of this, in 2 steps:1. Creation of a unique archive of collocated

observations and model data (DONE)2. Exploitation of the observation feedback

archive so as to improve procedures, estimates, understanding (TODO)

• Today we saw a few examples; this talk shall motivate further investigations from researchers and users

estimation of surface pressure & wind observation errors from era-20c observation feedback

Documents