challenges in forecasting peak electricity demandchallenges in forecasting peak electricity demand...
TRANSCRIPT
Challenges in forecastingpeak electricity demand
Rob J Hyndman
Challenges in forecasting peak electricity demand 1
Part 2
Challenges
1 How to evaluate forecast distributions
2 How to select the best forecasting method
3 How to account for off-grid generation
4 How to use smart metre data and networkdata when forecasting
5 How to improve forecasts
Challenges in forecasting peak electricity demand 2
Forecast accuracy measures
MAE: Mean absolute errorMSE: Mean squared errorMAPE: Mean absolute percentage error
å Good when forecasting a typical future value(e.g., the mean or median).
å Useless for evaluating forecast percentiles(probability of exceedance values) and forecastdistributions.
Challenges in forecasting peak electricity demand How to evaluate forecast distributions 3
Evaluating forecast distributions
Challenges in forecasting peak electricity demand How to evaluate forecast distributions 4
PoE (annual interpretation)
Year
PoE
Dem
and
2.0
2.5
3.0
3.5
4.0
98/99 00/01 02/03 04/05 06/07 08/09 10/11
10 %50 %90 %
●
●
●
●
●
●
●●
● ●
●
●
●
●
10 out of 13 above 90% PoE
5 out of 13 above 50% PoE
0 out of 13 above 10% PoE
Evaluating forecast distributions
Qt(p) = PoE of yt, to be exceeded with per-centage p.
G(p) = percentage of times yt greater thanQt(p) in the historical data.
Challenges in forecasting peak electricity demand How to evaluate forecast distributions 5
If Qt(p) is accurate, then G(p) ≈ p
Evaluating forecast distributions
Challenges in forecasting peak electricity demand How to evaluate forecast distributions 6
020
4060
8010
0
p = Forecast percentage exceedance
G(p
) =
Act
ual p
erce
ntag
e ex
ceed
ance
0 10010% 50% 90%
5/13
10/13
Evaluating forecast distributions
Challenges in forecasting peak electricity demand How to evaluate forecast distributions 7
0 20 40 60 80 100
020
4060
8010
0
p = Forecast percentage exceedance
G(p
) =
Act
ual p
erce
ntag
e ex
ceed
ance
KS
KS = maxp |G(p)− p|
KS = Kolmogorov-Smirnov statistic= largest difference between G(p) and p.
Evaluating forecast distributions
Challenges in forecasting peak electricity demand How to evaluate forecast distributions 8
0 20 40 60 80 100
020
4060
8010
0
p = Forecast percentage exceedance
G(p
) =
Act
ual p
erce
ntag
e ex
ceed
ance
Area = MAEP: Mean Absolute Excess Probability
MAEP =∫ 10 |G(p)− p|dp
Evaluating forecast distributions
MAEP more sensitive and less variable than KS.
Weekly or monthly maximums are betterbecause there are more of them to evaluate.
What is a good value of KS or MAEP?
We could restrict the range of p to “interesting”values such as p > 0.5.
These only measure whether the PoEs wereexceeded, not how much they were exceeded.
Challenges in forecasting peak electricity demand How to evaluate forecast distributions 9
Forecast scoring
Challenges in forecasting peak electricity demand How to evaluate forecast distributions 10
0 1 2 3 4 5 6
Demand distribution
Demand (GWh)
50%
50% PoE
Score for 50% PoEEquivalent toabsolute error
Forecast scoring
Challenges in forecasting peak electricity demand How to evaluate forecast distributions 11
0 1 2 3 4 5 6
Demand distribution
Demand (GWh)
10%
10% PoE
Score for 10% PoE
Forecast scoring
Challenges in forecasting peak electricity demand How to evaluate forecast distributions 12
0 1 2 3 4 5 6
Demand distribution
Demand (GWh)
75%
75% PoE
Score for 75% PoE
Forecast scoring
Let Qt(1), . . . ,Qt(99) be the PoEs of the forecastdistribution for probabilities 1%,. . . ,99%. Then thescore for observation y is
S(Qt(i), yt) =
{1
100 i(Qt(i)− yt) if yt < Qt(i)1
100(100− i)(yt − Qt(i)) if yt ≥ Qt(i)
Scores are averaged over all observed data foreach i to measure the accuracy of the forecastsfor each percentile.Average score over all percentiles gives thebest distribution forecast.Takes account of how far PoEs are exceeded.
Challenges in forecasting peak electricity demand How to evaluate forecast distributions 13
Forecasting the past
Traditional evaluation
Time series cross-validation
GEFCOM 2012 competition
Challenges in forecasting peak electricity demand How to select the best forecasting method 14
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● timeTraining data Test data
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
Also known as “Evaluation ona rolling forecast origin”
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● time
GEFCom2012
Challenges in forecasting peak electricity demand How to select the best forecasting method 15
Accuracy measuredby weighted root MSE.
Methodspublishedin IJF,April 2014
GEFCom2014
Probabilistic forecasting of demand, price,wind, and solar.
Forecasts to be submitted in the form ofpercentiles of future distributions.
Accuracy measured by scoring.
Rolling forecasts with incremental data updateon a weekly basis.
Prizes for student teams, and for best methods.
Winning methods to be published in the IJF.
Challenges in forecasting peak electricity demand How to select the best forecasting method 16
How to account for off-grid generation?
Locally generated power may not be recorded.But forecasts better if they are total demand.Need a model for PV generation that is linked tothe model for demand.
Challenges in forecasting peak electricity demand How to account for off-grid generation 17
Better: measure the off-grid generation viasmart metres.
How to use smart metre data?
Challenges in forecasting peak electricity demand How to use smart metre data 18
Smart metre data allow predictionof usage at household level basedon household characteristics:number of occupants, ages, etc.
So we could build a model forindividual usage, and scale it upfor the entire network.
How to allow for demandresponse?
Need aggregate information onhousehold characteristics for thenetwork.
Combine network and smartforecasts to improve accuracy
Ten steps to improving your forecasts
1 Look after your data
2 Understand how your forecasts will be used
3 Find the right forecasting tools
4 Use appropriate accuracy measures
5 Do not set targets
6 Do not adjust dishonestly
7 Keep it simple, stupid
8 Combine forecasts
9 Share ideas and mix with other forecasters
10 Adopt a process of continuous improvement
Challenges in forecasting peak electricity demand How to improve forecasts 19
Some resourcesBlogs
robjhyndman.com/hyndsight/blog.drhongtao.com/
OrganizationsInternational Institute of Forecasters:forecasters.orgIEEE Working Group on Energy Forecasting:linkedin.com/groups/IEEE-Working-Group-on-Energy-4148276
BooksDickey and Hong (2014) Electric loadforecasting: fundamentals and best practices,OTexts. www.otexts.org/book/elf
Challenges in forecasting peak electricity demand How to improve forecasts 20