Ensemble Predictions: Understanding Uncertainties

Lars LandbergWind Energy Department Risø National Laboratory DTUDenmark

Gregor Giebel, Jake Badger, Risø National Laboratory DTUHenrik Aalborg Nielsen, Torben Skov Nielsen, Henrik Madsen, Pierre Pinson, IMM DTU

Kai Sattler, Henrik Feddersen, Henrik Vedel, Danish Meteorological Institute

Contents

• Where do we come from?

• Two new things:

• Ensemble forecasts

• More than one NWP

• Conclusions

About forecasts

Good to know: the expected production

Even better to know: the uncertainty

NWP

Obs

ModelOutput:

ProductionUncertainty

In the ”old” days (late 90’ties!)

The idea of ensembles / Spaghetti plot

An ensemble of multiple forecasts, done from different initial conditions, or different numerical models / model runs, should give a measure of forecast uncertainty

• Assumption: There is a connection between spread and skill

Image Peter Houtekamer, Environment Canada

Ensemble predictions

• Ensembles try to catch more of the variety of the weather

• “Proper” ensembles – deliver plume of possible futures

• Multi-model ensembles: can increase accuracy

• Connection between spread & skill?

• Might be better use of computer resources

Source: NCEP/NCAR

2 “proper” ensembles

• ECMWF:

51 members, up to 10 days

ahead, global domain, made by

adding singular vectors

• NCEP/NCAR:

12 members, up to 84 hours

ahead, global domain, made

through bred modes

• resolution (both): 80 km

• Variables: wind (speed and dir)

@10m

PSO-Ensemble project

• 3-year Danish national project (now finished)

• Uses ECMWF and NCEP ensembles

• Risø, IMM, DMI + Danish utilities

• Ensembles = same meteo model, many slightly different runs -> finds many probable futures -> uncertainty bands

• Important result: The quantiles as coming from the ensembles are not directly applicable as power quantiles!

• Demo ran 1 year (using ECMWF) and counting, used for

• trading over weekend,

• weekly fuel demand forecasts and for

• maintenance / power plant repair scheduling

Visualisation: All members

• Shows all the 51 ECMWF members without transformation

• The black line is the control run (the best guess of ECMWF)

Visualisation: most quantiles

• Showing most derived and transformed quantiles between 5% and 95%

• Too much spread to be useful at a glance (outer quantiles are doubtful)

Visualisation: only 25 and 75 %

• Only central quantiles

PSO-Ensemble, unadjusted quantiles!!

PSO-Ensemble, adjusted quantiles

How to use a quantile forecast

Informal

• Is the forecast uncertain or not?

• Can we be “sure” to have more than 50% of installed capacity?

• If we need to take out a conventional plant for revision within the next week when should we do that?

Formal

• Given up- and down-regulation costs the quantile cup/(cup + cdown) should be used as the bid on the spot market.

• If we have many quantiles (the full p.d.f.) the optimal bid can be derived from any cost function.

Are the quantiles reliable?

• E.g. is the actual production below the 25% quantile in 25% of the cases?

• Can be checked by grouping the data (here: by horizon).

Accuracy of quantiles

Deviations up to5%, but oftenless.Some curvature;can presumablybe removed bytuning of themodel.

More than one NWP forecast

Doubling the number of NWP• Used DMI and DWD for six test cases in Denmark

• Result: the combination of inputs is better

SyltholmFjaldene

MiddelgrundenKlim

Hagesholm

Tunø Knob

G. Giebel, A. Boone: A Comparison of DMI-Hirlam and DWD-Lokalmodell for Short-Term Forecasting. Poster on the EWEC, London, Nov 2004

Conclusions

• Ensemble predictions can not be used directly as a measure of the error

• Errors are modelled much better if ensemble predictions are used

• Two NWP forecasts improve the predictions by more than 1 m/s!

• In general: we are getting much better af predicting the power output