The Accuracy of Wind and Solar Energy Forecasts and the Prospects for Improvement

Has Wind Energy Forecasting Solved the Challenge Posed by Intermittency? Evidence from the United KingdomKevin F. ForbesUSAEE Distinguished LecturerAssociate Professor of EconomicsThe Catholic University of AmericaForbes@CUA.edu

Ernest M. ZampelliProfessor of EconomicsThe Catholic University of AmericaZampelli@CUA.edu

USAEE North American ConferencePittsburgh, Pennsylvania27 October 2015

1The Organization of this Talk1)Why is Forecasting Important?

2) The Literature on Wind Energy Forecast Accuracy

3) What is the level of forecast skill ? Specifically, what does the Mean Squared Error Skill Score (MSESS) indicate about the solar and wind energy forecasts? How does this level of accuracy compare to the accuracy of the load forecasts?

4)From the point of view of a system operator, how does wind energy compare with conventional forms of generation?

5)What are the prospects for improving the accuracy of the wind, solar, and load forecasts? 21)Why is Forecasting Important?The stability of the power grid is enhanced when forecasts are more accurate. This is important because blackouts have very high societal costs

Some forms of balancing technologies such as open-cycle gas turbines can be very expensive to deploy and also have above average emissions factors. 32) The Literature on Forecast Accuracy Some researchers calculate a root-mean-squared error of the forecasts and then weight it by the capacity of the equipment used to produce the energy. The reported capacity weighted root mean squared errors (CWRMSE) are usually less than 10 percent. Adherents of this approach include Lange, et al. (2006, 2007), Cali et al. (2006), Krauss, et al. (2006), Holttinen, et al. (2006), Kariniotakis, et al. (2006), and even NERC (2010, p. 9).

In a publication entitled, Wind Power Myths Debunked, Milligan, et al. (2009) draw on research from Germany to argue that it is a fiction that wind energy is difficult to forecast. In their words: In other research conducted in Germany, typical wind forecast errors for a single wind project are 10% to 15% root mean-squared error (RMSE) of installed wind capacity (emphasis added) but drop to 5% to 7% for all of Germany. (Milligan, et al. 2009, p. 93)

The UKs Royal Academy of Engineering (2014, p. 33) has noted that wind energys capacity weighted forecast error of about five percent is evidence that that the wind energy forecasts are highly accurate.

A report by the IPCC ( 2012 p, 623) on renewable energy indicates that wind energy is moderately predictable as evidenced by a capacity weighted RMS forecast error that is less than 10%. Solar energy is reported to be even more accurate.

4The Literature on Forecast Accuracy (Continued)NREL (2013) implicitly endorses capacity weighted RMSEs for wind energy but makes use of energy weighted RMSEs when discussing the accuracy of load forecasts.

In contrast, Forbes et. al. (2012) calculate a root-mean-squared forecast error for wind energy and then weight it by the mean level of wind energy that is produced. The reported energy weighted root mean squared errors (EWRMSE) are in excess of 20 %.

5 3) Using The Mean-Squared-Error Skill Score (MSESS) to Assess Forecast Accuracy6How accurate are the forecasts?MSESS were computed for the following zones and/or control areas:Bonneville Power AdministrationCAISO: SP15 and NP15MISOPJM50Hertz in GermanyAmprion in GermanyElia in BelgiumRTE in FranceNational Grid in Great BritainFinlandSwedenNorwayEastern DenmarkWestern Denmark When possible the MSESS are reported for Wind, Solar, and Load7Mean Squared Error Skill Scores (MSESS) with a Persistence Forecast as ReferenceControl Area/ZoneForecast TypeSample PeriodObservationsGranularityMSESS50Hertz (Germany)Day-Ahead Load1Jan2011 31Dec2013104,590Quarter-Hour-62.7486Day-Ahead Wind1Jan2011 31Dec2013104,590Quarter-Hour-31.3501Day-Ahead Solar1Jan2011 31Dec201354,545Quarter-Hour-5.26831Amprion (Germany)Day-Ahead Load1Jan2011 31Dec2013103,326Quarter-Hour-12.3308Day-Ahead Wind1Jan2011 31Dec2013103,326Quarter-Hour-14.5887Day-Ahead Solar1Jan2011 31Dec201355,498Quarter-Hour-11.206918Mean Squared Error Skill Scores (MSESS) with a Persistence Forecast as Reference (Continued)Control Area/ZoneForecast TypeSample PeriodObservationsGranularityMSESSCalifornia ISODay-Ahead Load1Jan2013 31Dec20138,760Hourly0.6026NP15Day-Ahead Wind1Jan2013 31Dec20138,704Hourly-6.1401NP15Hour-Ahead Wind1Jan2013 31Dec20138,704Hourly-2.3605NP15Day-Ahead Solar1Jan2013 31Dec20138,666Hourly-3.2002NP15Hour-Ahead Solar1Jan2013 31Dec20138,666Hourly-2.4846SP15Day-Ahead Wind1Jan2013 31Dec20138,752Hourly-4.8210SP15Hour-Ahead Wind1Jan2013 31Dec20138,752Hourly-2.1894SP15Day-Ahead Solar1Jan2013 31Dec20138,752Hourly0.7050SP15Hour-Ahead Solar1Jan2013 31Dec20138,752Hourly0.79729Mean Squared Error Skill Scores (MSESS) with a Persistence Forecast as Reference (Continued)Control Area/ZoneForecast TypeSample PeriodObservationsGranularityMSESSBelgium Day-Ahead Solar1Jan2013 31Dec201317,921Quarter-Hour-12.2621Intra-Day Solar1Jan2013 31Dec201311,278Quarter-Hour-9.7931FranceDay-Ahead Load1Jan2012 31Dec201335,088Half-Hourly0.3842Day-Ahead Wind1Jan2012 31Dec201317,349Hourly-5.7375Hour 1 Same Day, Wind1Jan2012 31Dec201315,109Hourly-5.2889NorwayDay-Ahead Load1Jan2011 31Dec201326,160Hourly0.1870SwedenDay-Ahead Load1Jan2011 31Dec201326,160Hourly0.2008FinlandDay-Ahead Load1Jan2011 31Dec201326,159Hourly0.0486Eastern DenmarkDay-Ahead Load1Jan2011 31Dec201326,160Hourly0.3953Day-Ahead Wind1Jan2011 31Dec201326,107Hourly-2.7507Western DenmarkDay-Ahead Load1Jan2011 31Dec201326,160Hourly0.6560Day-Ahead Wind1Jan2011 31Dec201326,105Hourly-3.674910Mean Squared Error Skill Scores (MSESS) with a Persistence Forecast as Reference (Continued)Control Area/ZoneForecast TypeSample PeriodObservationsGranularityMSESSMISODay-Ahead Wind Energy1Jan2011 31Dec201326,303Hourly-4.3873PJMDay-Ahead Load1Jan2011 31Dec201326,160Hourly0.4727New York CityDay-Ahead Load1Jan2011 31Dec201325,675Hourly0.1703Bonneville PowerFive Minute-Ahead Wind1Jan2012 31Dec2013206,477Five minutes-36.25762Hour-Ahead Wind1Jan2012 31Dec201316,847Hourly-63.57602Great BritainDay-Ahead Load1Jan2012 31Dec201330,477Half-Hourly 0.62Day-Ahead Wind1Jan2012 31Dec201330,477Half-Hourly-19.032

1 Daylight portion of the sample period

2MSESS calculation excludes periods in which wind energy production was curtailed by the system operator.114)From the point of view of a system operator, how does wind energy compare with conventional forms of generation? Evidence from Great BritainIn Great Britain, each generating station informs the system operator of its intended level of generation one hour prior to real-time. This value is known as the final physical notification (FPN).

Generators also submit bids (a proposal to reduce generation) and offers (a proposal in increase generation) to provide balancing services

During real-time, the system operator accepts the bids and offers based on system conditions. In short, the revised generation schedule equals the FPN plus the level of balancing services volume requested by the system operator.

Failure to follow the revised generation schedule gives rise to an electricity market imbalance that needs to be resolved by other generators.

12The Revised Generation Schedules vs Actual Generation: The Case of Coal in Great Britain

EWRMSE = 2.5 %13The Revised Generation Schedules vs Actual Generation: The Case of Combined Cycle Gas Turbines in Great Britain

EWRMSE = 5.6%14Actual vs. Scheduled Generation: The Case of Nuclear Energy in Great Britain

EWRMSE = 7.4 %15The Revised Generation Schedules vs Actual Generation: The Case of Wind Energy in Great Britain, 1 Jan 2012 31 2013

EWRMSE= 18 %16Average Imbalances by Fuel in Great Britain, 1 Jan 2012- 31 December 2013

175) The Prospects for Improving the ForecastsSignificant improvements in day-ahead forecasts will probably require major advances in meteorological research. One obvious place to begin is to note that the heat trapping properties of Greenhouse gases most likely have implications for wind speeds.

Significant improvements in very short run forecasts (e.g. one or two hours ahead) are possible by exploiting the systematic nature of the existing forecast errors. 18The Systematic Nature of the Existing Day-Ahead Forecast Errors for Wind Energy: Evidence from Great Britain

19Out of Sample Results for Wind Energy in Great Britain, 1 Jan 2014 30 June 2014Forecast TypeNumber of ObservationsMSESSEWRMSEDay-Ahead Wind Forecast8,571

-35.0531.9 %Forecast equal to the levels of generation declared by operators one hour prior to real-time8,571

-19.7124.2 %Modified Forecast: available to system operator 30 min prior to real-time8,571 -1.959.1 %20Further AnalysisWhat about other Power Systems ?

What about Solar Energy ?

21Hour-Ahead Forecasted and Actual Wind Energy in SP15 in CAISO, 1 January 30 September 2015EWRMSE = 37.1 % MSESS = -2.62

22Actual Wind Energy and a Revised Hour-Ahead Wind Energy Forecast for SP15 in CAISO), 1 January 30 September 2015EWRMSE = 15.8 % MSESS = 0.34

23Actual Solar Energy in 50Hertz in Germany and an Out-of-Sample Econometrically Modified Solar Energy Forecast, 1 July 2013 3 March 2014For the daylight period: EWRMSE = 4.8 %

MSESS = 0.768

24Summary and ConclusionsWith few exceptions, the load forecasts examined in this study have positive skill scores relative to a persistence load forecast.

With few exceptions, the solar and wind forecasts examined in this study have negative skill scores relative to the corresponding persistence forecasts.

Evidence has been presented that the forecast errors have a systematic component

Evidence has also been presented that econometric modelling of this systematic component can yield short-run solar and wind energy forecasts that are significantly more accurate. This does not resolve the challenge of intermittency but may mitigate matters.

The modelling approach can also be applied to improve load forecasts. See http://dialogue.usaee.org/index.php/day-ahead-market-prices-of-electricity-and-economic-fundamentals-preliminary-evidence-from-new-york-city

25ReferencesGodfrey Boyle, 2010. Renewable energy technologies for electricity generation, in Harnessing Renewable Energy in Electric Power Systems, Boaz Moselle, Jorge Padilla, and Richard Schmalenese (eds.), RFF Press, Washington, DC, 2010, at 7-29. California Independent System Operator, ISO New England, Midwest Independent Transmission System Operator, New York Independent System Operator , PJM Interconnection, and Southwest Power Pool, 2010. 2010 ISO/RTO Metrics Report. At http://www.isorto.org/atf/cf/%7B5B4E85C6-7EAC-40A0-8DC3-003829518EBD%7D/2010%20ISO-RTO%20Metrics%20Report.pdf mit Cali, Bernhard Lange, Rene Jursa, Kai Biermann, 2006. Short-term prediction of distributed generation Recent advances and future challenges, Elftes Kasseler Symposium Energie-Systemtechnik. At http://www.iset.uni-kassel.de/public/kss2006/KSES_2006.pdf Mark A. Delucchi and Mark Z. Jacobson, 2011. Providing all global energy with wind, water, and solar power, Part II: Reliability, system and transmission costs, and policies. Energy Policy, 39, at 1170-1190.European Wind Energy Association, 2007. Debunking the Myths. At http://www.ewea.org/fileadmin/ewea_documents/documents/publications/wind_benefits/Windpower_is_unreliable.pdf Kevin Forbes, Marco Stampini, and Ernest M. Zampelli, 2012a. Are Policies to Encourage Wind Energy Predicated on a Misleading Statistic?, The Electricity Journal, Volume 25, Issue 3, pp. 42-54Kevin Forbes, Marco Stampini, and Ernest M. Zampelli, 2012b. Do Policies to Encourage Wind Energy Inadvertently Pose Challenges to Electric Power Reliability? Evidence from the 50Hertz Control Area in Germany, The Electricity Journal, November 2012, Volume 25, Issue 9, pp. 37-42GE Energy, 2010. Western Wind and Solar Integration Study, NREL/SR-550-47434, National Renewable Energy Laboratory, Golden, Colorado, May. At http://www.nrel.gov/wind/systemsintegration/pdfs/2010/wwsis_final_report.pdf Gregor Giebel, Richard Brownsword, George Kariniotakis, Michael Denhard, and Caroline Draxl, 2011. The State-Of-The-Art in Short-Term Prediction of Wind Power A Literature Overview, 2nd Edition. Project report for the Anemos.plus and SafeWind projects. 109 pp. Ris, Roskilde, Denmark. Available at http://130.226.56.153/zephyr/publ/GGiebelEtAl-StateOfTheArtInShortTermPrediction_ANEMOSplus_2011.pdf Hannale Holttinen, Peter Meibom, Antje Orths, Frans van Hulle, Bernhard Lange, Mark OMalley, Jan Pierik, Bart Ummels, John Olav Tande,Ana Estanqueiro, Manuel Matos, Emilio Gomez, Lennart Sder, Goran Strbac, Anser Shakoor, Joao Ricardo, J. Charles Smith, Michael Milligan, and Erik Ela, 2009. IEA WIND Task 25: Design and operation of power systems with large amounts of wind power. At http://www.vtt.fi/inf/pdf/tiedotteet/2009/T2493.pdf 26References (Continued)Hannale Holttinen, Pirkko Saarikivi, Sami Repo, Jussi Ikheimo, Goran Koreneff, 2006. Prediction Errors and Balancing Costs for Wind Power Production in Finland. Global Wind Power Conference, AdelaideIntergovernmental Panel on Climate Change, 2012, Renewable Energy Sources and Climate Change Mitigation Special Report of the Intergovernmental Panel on Climate Change. At http://srren.ipcc-wg3.de/report/IPCC_SRREN_Full_Report.pdfGeorge Kariniotakis, 2006. State of the art in wind power forecasting, 2nd International Conference on Integration of Renewable Energies and Distributed Energy Resources, Napa, California/USA, 4-8 December.Mattias Lange and Ulrich Focken, 2005. State-of-the-Art in Wind Power Prediction in Germany and International Developments. Prediction of Wind Power and Reducing the Uncertainty for Grid Operators, Second Workshop of International Feed-In Cooperation, Berlin (DE) http://www.energymeteo.de/media/fic_eeg_article.pdf Bernhard Lange, Kurt Rohrig, Bernhard Ernst, Florian Schlgl, Umit Cali, Rene Jursa, and Javad Moradi, 2006. Wind power prediction in Germany Recent advances and future challenges. European Wind Energy Conference and Exhibition, Athens (GR).Bernhard Lange, Kurt Rohrig, Florian Schlgl, Umit Cali, and Rene Jursa,2006. Wind Power Forecasting. in: Boyle, G.(Ed.), Renewable Electricity and the Grid. Earthscan, London, England, at 95-120.Bernhard Lange, Arne Wessel, Jan Dobschinski, and Kurt Rohrig, 2009. Role of Wind Power Forecasts in Grid Integration Kasseler Symposium Energie-Systemtechnik, at 118-130 http://www.iset.uni-kassel.de/public/kss2009/2009_KSES_Tagungsband.pdf Bernhard Lange, Kurt Rohrig, Bernhard Ernst, Florian Schlgl, Umit Cali, Rene Jursa, and Javad Moradi, 2006. Wind power prediction in Germany Recent advances and future challenges, Zeitschrift fr Energiewirtschaft, vol. 30, no 2, at115-120. At http://www.iset.uni-kassel.de/abt/FB-I/publication/Lange-et-al_2006_EWEC_paper.pdf 27References (Continued)Bernhard Lange, Kurt Rohrig, Florian Schlgl, Umit Cali,and Rene Jursa, 2007. Wind Power Forecasting, in Renewable Electricity and the Grid, Godfrey Boyle, Ed. Sterling,VA: Earthscan, London, at 95-120.David Milborrow, 2007. Wind Power on the Grid, in Renewable Electricity and the Grid, Godfrey Boyle, Ed. Sterling,VA: Earthscan, London, at 31-54 Michael Milligan,Kevin Porter, Edgar DeMeo, Paul Denholm, Hannele Holttinen, Brendan Kirby, Nicholas Miller, Andrew Mills, Mark OMalley, Matthew Schuerger, and Lennart Soder , 2009. Wind Power Myths Debunked, IEEE Power and Energy, November/December vol 7 no 6, at 89-99.National Grid, 2009. Operating the Electricity Transmission Networks in 2020: Initial Consultation. At http://www.nationalgrid.com/NR/rdonlyres/32879A26-D6F2-4D82-9441-40FB2B0E2E0C/39517/Operatingin2020Consulation1.pdf North American Electric Reliability Corporation, 2009b. Accommodating High Levels of Variable Generation, April. At http://www.nerc.com/files/IVGTF_Report_041609.pdf NERC, 2010. IVGTF Task 2.1 Report: Variable Generation Power Forecasting for Operations. At http://www.nerc.com/files/Varialbe%20Generationn%20Power%20Forecasting%20for%20Operations.pdf Jennifer Rodgers and Kevin Porter, 2009. Central Wind Power Forecasting Programs in North America by Regional Transmission Organizations and Electric Utilities, NREL/SR-550-46763. Available at http://www.nrel.gov/docs/fy10osti/46763.pdf

28References (Continued)National Grid, 2009. Operating the Electricity Transmission Networks in 2020: Initial Consultation. At http://www.nationalgrid.com/NR/rdonlyres/32879A26-D6F2-4D82-9441-40FB2B0E2E0C/39517/Operatingin2020Consulation1.pdf North American Electric Reliability Corporation, 2009b. Accommodating High Levels of Variable Generation, April. At http://www.nerc.com/files/IVGTF_Report_041609.pdf NERC, 2010. IVGTF Task 2.1 Report: Variable Generation Power Forecasting for Operations. At http://www.nerc.com/files/Varialbe%20Generationn%20Power%20Forecasting%20for%20Operations.pdf Jennifer Rodgers and Kevin Porter, 2009. Central Wind Power Forecasting Programs in North America by Regional Transmission Organizations and Electric Utilities.Royal Academy of Engineering, 2014, Wind Energy : Implications of Large-Scale Deployment on the GB Electricity System http://www.raeng.org.uk/publications/reports/wind-energy-implications-of-large-scale-deployment

29