wind power prediction using artificial neural networks and evolu8th semesterfinal thesis

8/12/2019 Wind Power Prediction Using Artificial Neural Networks and Evolu8th SemesterFinal Thesis

http://slidepdf.com/reader/full/wind-power-prediction-using-artificial-neural-networks-and-evolu8th-semesterfinal 1/43

基

于Xilinx

系统生成

器和Simulink

联合

仿真的

［

单击此处键入论文文题目］

阿力夫

北科技

大学



论文题目：使用人工神经网络和进化算法预测

风力发电

学号： _________________________

作者： _________________________

专业名称： _________________________

2013年 05月 05日

Muhammad Arif Mughal

信息通信工程

s20111983



使用人工神经网络和进化算法预测风力发电

Modeling telemetry system for sensor monitoring

based on co-simulation of Xilinx System Generator

and SIMULINK

研究生姓名：阿瑞斯

指导教师姓名：杨裕亮

北京科技大学计算机与通信工程学院

北京 100083，中国

Master Degree Candidate： Haris Anwaar

Supervisor ： Prof. Yang YuLiang

School of Computer and Communication Engineering

University of Science and Technology Beijing

30 Xueyuan Road，Haidian District

Beijing 100083，P.R.CHINA



分类号： ____________ 密级： ______________

ＵＤＣ： ____________ 单位代码： ______________

北京科技大学博士学位论文

论文题目基于 Xilinx系统生成器和 Simulink 联合仿

真的传感器遥测系统建模

作者 _________________________

指导教师单位

指导小组成员单位

单位

论文提交日期 2013年 5月 5日

学位授单位北京科技大学

杨裕亮北京科技大学

北京科技大学

阿瑞斯

１０００８



北京科技大学硕士学位论文

- I -

致谢

I am heartly pleased to express my sincere gratitude to my supervisor Prof.

Yang YuLiang for his guidance, encouragement and constant support throughout

my research work. I commend him giving me freedom for independent research.

My words are inadequate to express my heartful thanks to Prof. Yang YuLiang for

his enthusiastic support in all efforts during my research period. I would like to

express my gratitude to his sincerity, dignity and dedication towards work. He has

been a constant inspiration for me and I feel privileged to have been associated

with him.

I wish to thank my group mate and friend LiuWei for his great cooperation,

his timely and generous help during my research period.I pay special thanks to all of my classmates, lab mates and country mates for

their help and encouragement for my research work.

Moreover, I would like to express my deep appreciation and love to my

entire family especially my mother and father for their support, prayers, patience

and encouragement during all of my studies living away from them. In the end, I

am deeply thankful to God to guide me and provide me strength to complete my

research work.



Table of Contents

Abstract ................................................................................................................ v

Chapter 1 : I ntroduction ........................................................................................ 1

1.1 Purpose of Project ................................................................................................. 1

1.2 Scope of Research.................................................................................................. 1

1.3 Applications .......................................................................................................... 1

1.4 Overview of Thesis ................................................................................................ 2

Chapter 2 : L iterature Survey ................................................................................ 3

2.1 Long Term Wind Energy Prediction ..................................................................... 3

2.1.1 ANN ....................................................................................................................................... 3

2.2 Short Term Wind Power Prediction ...................................................................... 4

2.2.1 Irrelevancy Filter .................................................................................................................... 4

2.2.2 Redundancy Filter .................................................................................................................. 6

2.2.3 RBFNN .................................................................................................................................. 6

2.2.4 EPSO...................................................................................................................................... 7

2.3 SVR ....................................................................................................................... 8

2.4 GRNN ................................................................................................................... 9

Chapter 3 : Implemented Methodologies .............................................................. 11

3.1 Long Term Wind Energy Prediction ................................................................... 11

3.1.1 Data Set ................................................................................................................................ 11

3.1.2 Why Use These Values ........................................................................................................ 11

3.1.3 Grassi Model Implementation .............................................................................................. 12

3.1.4 Improved Models ................................................................................................................. 13



ii

3.2 Short Term Wind Power Prediction .................................................................... 14

3.2.1 Two Stage Feature Selection ............................................................................................... 14

3.2.2 Forecast Engine.................................................................................................................... 15

3.2.3 Data Set ................................................................................................................................ 17

3.3 SVR ..................................................................................................................... 18

3.4 GRNN ................................................................................................................. 18

Chapter 4 : Resul ts and Discussion ...................................................................... 19

4.1 Long Term Wind Energy Prediction ................................................................... 19

4.1.1 Grassi Model Implementation .............................................................................................. 19

4.1.2 Three ANN Model ............................................................................................................... 20

4.1.3 Three ANN with EPSO Model ............................................................................................ 22

4.1.4 Comparison of Three Models .............................................................................................. 23

4.2 Short Term Wind Power Prediction .................................................................... 24

4.3 GRNN ................................................................................................................. 25

4.4 SVR ..................................................................................................................... 26

Chapter 5 : Conclusion and Futur e Work ............................................................ 28

5.1 Conclusion .......................................................................................................... 28

5.2 Future Work ....................................................................................................... 28

References .......................................................................................................... 30



iii

List of Figures

Figure 2-1 ANN Architecture [4] ............................................................................... ........................... 3

Figure 2-2 MI [3] .................................................................................................................................... 5

Figure 2-3 RBFNN [7] ................................................................ ............................................................ 6

Figure 2-4 PSO (particles converges to best position) [4] ................. .................................................. 7

Figure 2-5 Mapping of input into high dimension [9] ......................................................................... 8

Figure 2-6 SVR [4] ........................................................... ................................................................. ...... 8

Figure 2-7 GRNN Architecture [11] ........................................ ........................................................... 10

Figure 3-1 ANN architecture [2] .............................................. ........................................................... 12

Figure 3-2 Three ANN Model [4] ........................................................ ................................................ 13

Figure 3-3 ANN architecture used in three ANN model [2] ......................... .................................... 13

Figure 3-4 Three ANN with EPSO Model [4] .................................................................... ................ 14

Figure 3-5 MHNN and EPSO hybrid model [4] ..................... ........................................................... 15

Figure 3-6 Forecast engine [4] ................................................................................................ ............. 16

Figure 3-7 ANN Architecture used in MHNN [4] ................................................................ .............. 17

Figure 4-1 Testing plot of Grassi model implementation ................................................................ .. 19

Figure 4-2 Training plot of Grassi model implementation ............................................... ................ 20

Figure 4-3 Training and testing plot given in the paper [2] .......................................................... .... 21

Figure 4-4 Testing plot of three ANN implementation .................................................................. .... 21

Figure 4-5 Training plot of three ANN implementation ...................................................... ............. 22

Figure 4-6 Training plot of three ANN with EPSO implementation ............................................... 23

Figure 4-7 Testing plot of three ANN with EPSO implementation .................................................. 23

Figure 4-8 Testing plot of short term wind power prediction .......................................................... 25

Figure 4-9 Testing plot of GRNN ........................................................ ................................................ 26

Figure 4-10 Testing plot of SVR .......................................................... ................................................ 27

http://c/Users/john/Desktop/THESIS.docx%23_Toc330394443






































iv

List of Tables

Table 4-1 Results on Grassi model Implementation ............................................................. ............. 19

Table 4-2 Results of three ANN Model ........................................................... .................................... 20

Table 4-3 Results of three ANN with EPSO Model ........................................................................... 22

Table 4-4 Comparison of models on long term wind energy prediction paper ............................... 24

Table 4-5 Results of short term wind power prediction ................................................................ .... 24

Table 4-6 Results on GRNN ................................................................................................................ 25

Table 4-7 Results on SVR .................................................................................................................... 26



v

Abstract

Wind power prediction system is essential component in power distribution system.Wind is a fluctuating source of energy which has raised the issue of reliability of

power distribution system. To ensure reliability, wind power must be predicted

accurately in advance. The purpose of the project is to exp lore the learning

capability of both artificial neural network (ANN) and evolutionary algorithms (EA)

for wind power prediction and then develop a hybrid system of ANN and EA to

improve the efficiency of wind power prediction. Wind power prediction systems

using ANN, general regression neural network (GRNN), support vector regression

(SVR), combination of different types of ANNs and hybrid model of ANNs along

with enhanced particle swarm optimization (EPSO) are explained. It is observed that

hybrid model of ANNs along with EPSO and SVR are showing good results in terms

of accuracy.



1

Chapter 1 : Introduction

One of the most important challenges faced by mankind now days is the issue ofenergy. The issue of energy crisis has led the researchers to think and industrialist to

invest in the alternate energy resources. One of alternative source is wind, and it is

gaining popularity because it has the potential to produce power on commercial scale.

But there are some problems regarding wind, wind is a fluctuating source of energy

which has raised the issue of reliability of power distribution system. To ensure

reliability, wind power must be predicted accurately in advance. Wind power can be

estimated on long and short term basis. Short term wind power prediction period varies

from minutes to a day, whereas long term wind power prediction period varies from

months to years. Here both short and long term wind power prediction are considered.

1.1 Purpose of Project

The purpose of the project was to explore the learning capability of both artificial

neural network (ANN) and evolutionary algorithms (EAs) for wind power prediction.

Also to develop a hybrid system of ANN and EAs to improve the efficiency of wind

power prediction.

1.2 Scope of Research

The work presented in thesis is enough to develop a system that can do wind power

forecasting. Different techniques of ANNs, EAs and hybrid system of ANN with EA

along with general regression neural network (GRNN) and support vector regression

(SVR) are studied and also implemented. Efficiency of the wind power prediction was

improved by changing the parameters of the system.

1.3 Applications

Wind power prediction has greater application in electricity supply system, because

without wind power prediction system grid operators are unable to schedule the

economically efficient generation of electricity and also cannot ensures system

reliability of the electricity supply system. This is because of fluctuating and

intermittent behavior of wind power generation. Therefore wind power forecasting is

an important factor of power systems [1].



3

Chapter 2 : Literature Survey

This chapter will build the understanding of the different techniques and componentsthat are used in the implementation of the papers [2], [3] along with SVR and GRNN.

First of all, techniques and components of paper [2] will be discussed then techniques

and components of paper [3] will be discussed and after that SVR and GRNN will be

discussed.

2.1 Long Term Wind Energy Prediction

Paper [2], which is based on long term wind energy, uses ANN as wind energy

prediction system. Therefore only ANN will be discussed in detailed.

2.1.1 ANN

An ANN is a computational model that is inspired by the structure of human brain

consisting of neurons act as biological neural networks. A neural network consists of

layers of artificial neurons that are connected to each other through weights as shown

in Figure2-1.

Then the neural network is trained for a problem on the data relevant to the problem,

which will assign values to the weights (connection between two artificial neurons).

Now if the trained neural network is subjected to unknown data of same problem it

will give predicted values against the input test data [4].

Input Layer Hidden Layer Output Layer

Figure 2-1 ANN Architecture [4]



4

On basis of the architecture of neural networks there are many algorithms designed to

train the neural network but mostly widely used and efficient algorithm is back

propagation neural network (BPNN). In BPNN error is calculated at each step of

training at output nodes and then back propagates it to adjust the weights [4]. Inputs used for wind power prediction are mainly, wind speed, humidity, air pressure,

temperature, wind direction, generations hours e.tc. Different inputs will be tested on

experimental basis and those will be selected having greater influence on wind power

prediction [5].

BPNN will be used to implement wind power prediction system. The performance

parameters mainly in BPNN are mainly

Number of hidden layers along with number of neurons

Transfer functions used

Number of input units, and

Which inputs are to use

2.2 Short Term Wind Power Prediction

Wind power prediction system proposed in the paper [3] is composed of feature

selection and forecast engine shown. Feature selection comprises of irrelevancy filterand redundancy filter and forecast engine contains modified hybrid neural network

(MHNN) with enhanced particle swarm optimization (EPSO). First irrelevancy filter

then redundancy filter will be discussed. But MHNN is combination of ANNs and

radial basis function neural network (RBFNN), so only RBFNN will be discussed. In

the last EPSO will be discussed.

2.2.1 Irrelevancy Filter

The basic idea about this filter is that, it calculates mutual information (MI) value

between the target variable and all the input features one by one, MI basically shows

the common information between the two variables as shown in the Figure 2-2, after

calculating MI values it is decided that which variables should be used in next step on

basis of MI values, variables having MI values greater than specific thresh hold are

selected and others are discarded [6]. The performance of the filter will mainly

depend on the threshold that is used. MI values were calculated by first linearly

normalizing all the candidate features and the output target in the range [0-1].Then

median for each variable is calculated. Values of variable greater than its median were



5

rounded to 1 and values less than its median were rounded to 0.Then MI values were

calculated by using the formula given in equation (2.1)

mm m 2

m

mm 2

m

mm 2

m

P(X=0,Y =0)MI(X,Y )=P(X=0,Y =0)×log

P(X=0)XP(Y =0)

P(X=1,Y =0) +P(X=1,Y =0)×log

P(X=1)XP(Y =0)

P(X=0,Y =1) +P(X=0,Y =1)×log

P(X=0)XP(Y =1)

mm 2

m

P(X=1,Y =1)+P(X=1,Y =1)×log

P(X=1)XP(Y =1)

(2.1)

Where,

X=Target

Ym=Input candidate features

m m

mm

mm

m

m

mm

m m

m m

m mm

m mm

U =2*Y +X

U0P(X=0,Y =0)=

L

U2P(X=0,Y =1)=

L

U1

P(X=1,Y =0)= L

U3P(X=1,Y =1)=

L

(U0 +U2 )P(X=0)=

L

(U1 +U3 )P(X=1)=

L

(U0 +U1 )P(Y =0)=

L

(U2 +U3 )P(Y =1)=L

Figure 2-2 MI [3]

MI(X,Y)



6

2.2.2 Redundancy Filter

The basic idea behind this filter is that higher value of mutual information between

two selected candidates means more common information among them, so they are

redundant candidates. Therefore redundancy of each selected feature with the other

candidate inputs are calculated according to formula given in equation (2.2).

x m(t) S1(t) - x k(t)RC xk(t) =max MI xk(t),xm(t) (2.2)

Then redundancy criterion (RC) values were observed among the input features that

were greater than the thresh hold and discarded one of the input feature on basis of MI

values, candidate having smaller value of MI was discarded [3]. The performance of

the filter will mainly depend on the threshold that is used.

2.2.3 RBFNN

RBFNN have three layers an input layer, hidden layer with nonlinear radial basis

function (RBF) as activation function and a linear output layer as shown in Figure 2-

3. The output of the RBFNN is represented by equation (2.3).

N

i i

i=1

φ(x)= a ρ x-c (2.3)

Where, N is the number of neurons in the hidden layer, c i is the center of each neuron

in the hidden layer. The norm is taken as Euclidian distance and the basis function is

taken as Gaussian [7].

Figure 2-3 RBFNN [7]



8

Equation (2.4) is the velocity formula which moves the particle on basis of local best,

global best and personal-not best components. Equation (2.5) is the formula for

updating the particle position in the space. Equation (2.6) and (2.7) are formulas for

updating the personal best of the particle. Equation (2.8) and (2.9) are formulas forupdating the personal-not best of the particle.

The performance parameters in EPSO case will be factors that derive the

mathematical formula for changing the position of the particle in the swarm.

2.3 SVR

Support vector regressions are used due to many reasons; some of these are usage of

kernels, absence of local minima, sparseness of the solution and capacity controlobtained by acting on the margin etc. SVR follows these points to create a prediction

model [8].First of all SVR transform the input vectors into other dimension by using

kernels as shown in the Figure 2-5.

Figure 2-5 Mapping of input into high dimension [9]

The input vectors in the high dimension mostly act as linear due to sparseness created

by the kernel. Then linear hyper plane is fitted on the data in the other dimension.

Hyper plane is fitted in such a way to get maximum margins on its both sides which

helps in correct prediction as shown in the Figure 2-6. Then with help of the hyper

plane we predict the target value for the input test values.

ε i

ε

Error

Loss

Support Vectors

iε

ε

Figure 2-6 SVR [4]



9

2.4 GRNN

GRNN is one pass learning algorithm and is basically probability based prediction

neural network. Its work can be explained by a simple example given below.

There is a fund in which three peoples are contributing. First one contributed $2,

second one $3 and the third one contributed $100. Then three numbers are calculated,

sum of dollar amount times the person number (1*$2+2*$3+3*$100=$306), sum of

the $ amount ($2+$3+$100=$105), then divide the first number by second (306 / 105

~ 3).This shows the top contributor person in the fund i.e. person number 3. This is

exactly what GRNN does; the pattern that is very close to training data will get heavy

contribution while the other will get very smaller in calculating the output predicted

value [10], [11]. The architecture of GRNN is shown in the Figure 2-7.

Input layer: The input neurons feed their values into each node of hidden

layer. There is one neuron for each predictor variable in the input layer.

Hidden layer: In this layer for each value in the training data set one neuron

exists. In each neuron one of the training input vectors along with the target

value is captured or saved.

Summation layer: There are two neurons in the summation layer. One neuron

is the denominator summation unit and the other is the numerator summation

unit. The denominator summation unit just adds up the weight values coming

from each of the hidden neurons. The numerator summation unit for each

hidden neuron adds up the weight values multiplied by the actual target value

that is saved in the neurons of the hidden layer.

Decision layer: The decision layer divides the numerator summation unit by

the denominator summation unit and the result is presented as predicted target

value.

The actual working of the GRNN can be explained such as; first the input test vector

is given to input layer which transfer it to each neuron in the hidden layer. In each

neuron of the hidden layer, the distance between the input test vector and the vector

that is already saved in that corresponding neuron is calculated. The rbf activation

function is then applied on the resultant, which gives maximum value if the resultant

is small and minimum value otherwise. The output of the hidden neuron is called

weight. Then the weights are provided to summation layer and then at last to thedecision layer for final prediction.



10

Input Hidden Summation Decision

layer layer layer layer

Figure 2-7 GRNN Architecture [11]

X1

X2

X3



11

Chapter 3 : Implemented Methodologies

This chapter will briefly describe the implementation details of the papers [2], [3]along with SVR and GRNN. First of all paper [2] will be discussed then paper [3] will

be discussed and after that SVR and GRNN will be discussed.


Here implementation of paper, “Wind energy prediction using a two-hidden layer

neural network” [2], is discussed in which two hidden layer neural network is used to

make wind energy prediction using back propagation learning algorithm. The network

was trained on two years of data and then tested on one year data.

3.1.1 Data Set

The data set used, is given in the paper [2] in tabular form. The input parameters used

as input for neural network are

Wind speed

Relative humidity

Temperature

Generation hours

Maintenance hours

And the output is wind energy. Total 3 years of data was available, each input

parameter and output have total 12 values for year, because their values are taken as

average over the month. The data have been collected from 43 wind turbines over

period from January 2005 to December 2007.

3.1.2 Why Use These Values

Here the mentioned parameters in the data set section are briefly discussed, which

mainly affect the generation of wind power. It is known that the electrical power

generated by a wind turbine is given by equation (3.1)

3P=0.5ρAw (3.1)

Where,

ρ =Air density

w =Wind speed



12

P = Wind Power

A =Area swept by the turbine

From equation (3.1) it is clear that wind speed has a major influence on the power

output, because power depends on the cubic value of wind speed w. And also, thewind power depends on the air density ρ, which is in turn affected by the relative

humidity and the temperature. By using generation and maintenance hours as inputs,

the performance of the system is improved.

First, author’s Grassi model implementation is discussed then the improved model

implementations are discussed. In improved models, three ANN and three ANN with

EPSO models are discussed.

3.1.3 Grassi Model Implementation

In the paper [2] a two hidden layer neural network is used to predict the wind power

generation of three wind farms. Three years of proper experimental data is used to

train and test the neural network with back propagation learning algorithm. First two

years data was used for training and last year data was used for testing.

a. Structure of ANN

The neural network used is shown in the Figure 3-1. It have 5 input neurons and 1

output neuron with two hidden layers, where hyperbolic tangent is used as activation

function in the first hidden layer and the logarithmic sigmoid as second hidden layer

activation function and linear activation function at the output layer. Three neurons

are used in the hidden layers.

Figure 3-1 ANN architecture [2]

Humidity

Temperature

Generation hours

Maintenance hours

Wind energy

Wind speed



13

3.1.4 Improved Models

The results of data set of paper [2] were poor on the Grassi implementation so it was

improved by using a different model explained below.

a. Using Three ANNs

From model of paper [3], a part of the model was used to improve the accuracy,

shown in Figure 3-2.

Explanation of how the model works is in coming sections. The ANN used in this

case has the structure shown in Figure 3-3. Five inputs, seven first hidden layer

neurons, five second layer hidden layer neurons are used. Hyperbolic tangent is used

as activation function in the first hidden layer and the logarithmic sigmoid as second

hidden layer activation function and linear activation function at the output layer.

Figure 3-3 ANN architecture used in three ANN model [2]

Humidity

Temperature

Generation hours

Maintenance hours

Wind speed

Wind Energy

Figure 3-2 Three ANN Model [4]



14

b. Using Three ANNs with EPSO

The three ANN model was further modified and the results were further improved.

The modified model is given in Figure 3-4. The model shown in the Figure 3-4 is

same as used in paper [3] except that the feature selection component and RBFNN are

not used. The ANN used here is same as used in three ANN model shown in Figure 3-

3, i.e. having five inputs, seven first hidden layer, five second hidden layer neurons

with hyperbolic tangent as first hidden layer, logarithmic sigmoid as second hidden

and linear as output layer activation functions.


Here implementation of paper [3] is discussed briefly. Wind prediction system

proposed in the paper is composed of feature selection and forecast engine shown in

Figure 3-5. Feature selection comprises of irrelevancy filter and redundancy filter and

forecast engine contains MHNN with EPSO.

3.2.1 Two Stage Feature Selection

Wind power can be seen as a nonlinear mapping function of its past values and the

meteorological variables, and these variables are available in wind farm. Set of

candidate features used as input for prediction system is given in equation (3.2)

wp

ws

wd

H

T

S t ={ WP t-1 , WP t-2 ,... WP t-N ,

WS t ,WS t-1 ,...,WS t-N ,

WD t ,WD t-1 ,...,WS t-N ,

H t ,H t-1 , H t-2 ,... H t-N ,

T t ,T t-1 , T t-2 ,... T t-N }

(3.2)

Figure 3-4 Three ANN with EPSO Model [4]



15

Where,WP=Wind Power

WD=Wind Direction

WS=Wind Speed

H=Humidity

T=Temperature

Nwp=Nws=Nwd=NT=NH=Number of past values to be considered

The input candidate feature set is first pass through irrelevancy filter which filter out

those features that are irrelevant to the output target or not help much in predicting the

output wind power. Then the output of the irrelevancy filter is given to the

redundancy filter which finds and discards those features that are redundant. Working

of both filters has been discussed in literature survey chapter in detail.

3.2.2 Forecast Engine

After applying the two stage feature selection to the candidate feature set, the filtered

set is supplied to the forecast engine to train itself; forecast engine is shown in Figure

3-6. Forecast engine mainly comprises of MHNN and EPSO.

First the data set is given to RBFNN, which gives initial forecast for target variables

after training. Then the initial forecast from RBFNN plus the filtered candidate input

feature set is given to artificial neural network 1 (NN1). NN1 after training, further

tuned its weights through EPSO, then the forecast for target plus the final weights of

NN1 are passed to artificial neural network 2 (NN2). NN2 also train itself on the

filtered input candidate feature set plus the forecast of target provided by NN1,

provided with initial weights that of passed by NN1. NN2 weights are then further

Fi ure 3-5 MHNN and EPSO h brid model 4



16

tuned through EPSO. Then the forecast for target plus the final weights of NN2 are

passed to artificial neural network 3 (NN3). NN3 also train itself on the filtered input

candidate feature set plus the forecast of target provided by NN2, provided with initial

weights that of passed by NN2, the NN3 also further tuned its weights with help ofEPSO. In the testing phase we will need only the weights of NN1, NN2 and NN3. The

selected input feature set will be given first to RBFNN, giving target prediction

values. Then NN1 will be given the selected features along with predicted target

values of RBFNN, giving predicted target values. Similarly NN2 and NN3 will give

target predicted values. Target predicted values of the NN3 will be considered final.

a. ANNs in MHNN

From Figure 3-6 it is clear that each ANN transfers two kinds of results to next ANN.

The first set of results transferred i.e. weights and bias values are actually the

knowledge of the ANN which it learnt, and the next ANN begin its learning process

from the point that the previous ANN terminated. Here all ANNs of the MHNN have

the same number of output, input and hidden neuron so that next ANN increases the

obtained knowledge of its previous NN. The structure of ANN used is given in Figure

3-7. Activation function used for hidden layer was tangent sigmoid.

Figure 3-6 Forecast engine [4]



17

Three different ANNs have been selected in MHNN because by suitable selection of

training algorithms of the ANNs the MHNN can learn more than a single ANN.

Levenberg-Marquardt (LM) is selected as first ANN because it is a fast learning

algorithm and at the beginning of the training phase it learns quickly about the

problem and its training error quickly decreases. The Broyden-Fletcher -Goldfarb-

Shanno (BFGS) is selected as second ANN because it performs a better search of the

solution space with condition that it starts from a suitable initial point i.e. provided

with good initial weights. The Bayesian Regularization (BR) is selected the third

ANN because it generalizes the problem well [12].

b. EPSO

EPSO is used because when if any ANN is trapped in a local minimum then neither

that ANN nor the next one may be able to escape from the local minimum. The EPSO

is used after each ANN of the MHNN to avoid the explained situation i.e. trapping of

the ANN in local minima.

3.2.3 Data Set

The data set is taken from the website of Morrisville State College [13]. Data set

comprises of data of year 2008,2009,2010,2011 and two months of year 2007. The

data set have average daily values of wind speed, relative humidity, temperature and

wind power. So modified input candidate input feature set is used i.e. without wind

direction, instead of the candidate input feature set that is represented by equation

(3.2). Data from 2007 to 2010 was used for training purpose and 2011 data was used

for testing purpose.

Figure 3-7 ANN Architecture used in MHNN [4]

http://en.wikipedia.org/wiki/Charles_George_Broyden

http://en.wikipedia.org/wiki/Roger_Fletcher_(mathematician)

http://www.columbia.edu/~goldfarb/

http://rutcor.rutgers.edu/~shanno/

http://www.math.vu.nl/~stochduo/talks/ICM.pdf

http://www.math.vu.nl/~stochduo/talks/ICM.pdf

http://rutcor.rutgers.edu/~shanno/

http://www.columbia.edu/~goldfarb/

http://en.wikipedia.org/wiki/Roger_Fletcher_(mathematician)

http://en.wikipedia.org/wiki/Charles_George_Broyden



18

3.3 SVR

SVR was applied on data set taken from Morrisville State College [13]. The data set

was first applied to the irrelevancy filter then to redundancy filter as explained in the

feature selection of paper [3], after that the filtered set was given to the SVR for

training. 25% of the data set was used for testing and 75% for training.

3.4 GRNN

GRNN was applied on data set taken from Morrisville State College [13]. The data set

was first applied to the irrelevancy filter then to redundancy filter as explained in the

feature selection of paper [3], after that the filtered set was given to the GRNN for

training. 25% of the data set was used for testing and 75% for training.



19

Chapter 4 : Results and Discussion


4.1.1 Grassi Model Implementation

For training and testing the data set was divided into two parts, first two years data

was used for training purpose and the last year was used for testing purpose. For

training, the data was first normalized between 0 and 1, total 500 epochs were run to

train the neural network and the performance function used was MSE (Mean Square

Error).After training the ANN, it was tested on the last year data and the resultsobtained are shown in Table 4-1.

Table 4-1 Results on Grassi model Implementation

Training Testing

MAE 0.1351 0.1020

The testing and training plots are shown graphically in Figure 4-1 and Figure 4-2respectively.

Figure 4-1 Testing plot of Grassi model implementation



20

Figure 4-2 Training plot of Grassi model implementation

In the training plot shown in Figure 4-2 the predicted plot is not completely matching

the actual plot but the predicted shows the general trend. This is because of the

generalization behaviour of the neural network. In the testing plot shown in Figure 4-1

predicted values are close for months 2,3,5,8,10,11 but other values are at some

distance from the original one.The plot given in the paper is shown in Figure 4-3,

comparison of our plot and the plot given in the paper shows that our implementation

results are not good. Reason is that author have not discussed the implementation

completely, we have implemented the paper on some assumptions.

4.1.2 Three ANN Model

Similarly for three ANN model first two years data was used for training purpose and

the last year was used for testing purpose. The results obtained are given in Table 4-2

Table 4-2 Results of three ANN Model

Training Testing

MAE 0.0114 0. 0138

The testing and training plots are shown graphically in Figure 4-4 and Figure 4-5

respectively.



21

Figure 4-3 Training and testing plot given in the paper [2]

Figure 4-4 Testing plot of three ANN implementation

In the the testing plot shown in Figure 4-4 predicted values are close for all months

except the 4 one. In the training plot shown in Figure 4-5 the predicted plot is not the

exact match of the actual plot but the predicted shows the general trend. If the

predicted plot in Figure 4-5 exactly matches the original one then on testing data the

ANN wiil show great error.



22

Figure 4-5 Training plot of three ANN implementation

4.1.3 Three ANN with EPSO Model

Similarly for three ANN model with EPSO first two years data was used for training

purpose and the last year was used for testing purpose. The results obtained are givenin Table 4-3

Table 4-3 Results of three ANN with EPSO Model

Training Testing

MAE 0.0127 0. 0118

The training and testing plots are shown graphically in Figure 4-6 and Figure 4-7

respectively .In the the training plot shown in Figure 4-6 the predicted plot is not the

exact match of the actual plot but the predicted shows the general trend. If the

predicted plot in Figure 4-6 exactly matches the original one then on testing data the

NN wiil show great error because of over fitting problem. In the testing plot shown in

Figure 4-7 predicted values are close for all months except the 4 one. This means the

error value in case of testing is mainly due to the value of the 4 month. The

comparison of these plots with the one’s given in the paper have close resemblence,this is explained in the coming section.



23

Figure 4-6 Training plot of three ANN with EPSO implementation

Figure 4-7 Testing plot of three ANN with EPSO implementation

4.1.4 Comparison of Three Models

Comparison of the results of three models mentioned previously is given in Table 4-4.

From the Table 4-4 it is clear that the results of three ANN model and three ANN

with EPSO model have better results in case of testing than the other two.



24

Table 4-4 Comparison of models on long term wind energy prediction paper

MAEGrassi Model

Implementation

Three ANN

model

Implementation

Three ANN

with EPSO

model

Implementation

Author’s

error

(mentioned

in paper)

Testing 0.1020 0.0138 0. 0118 0.0156

Training 0.1351 0.0114 0.0127 0.0109

From Table 4-4 it can be seen that among the mentioned models the training error is

minimum for the author’s model that is mentioned in its paper . It can be seen that the

training error for three ANN model is smaller than that of three ANN with EPSO

model but the testing error of the three ANN with EPSO model is smaller than the

three ANN model error, it is because of the generalization. The three ANN with

EPSO model is showing greater generalization then the three ANN model because it

has not memorized the training data, instead it has learned only the trend from the

data set.


Ninety nine input candidate feature set was given to two stage feature selection, in

which 25 features were of wind speed, 25 features were of relative humidity, 25

features were of temperature and 24 features were of wind power. Five features were

passed from two stage feature selection to the forecast engine. The forecast engine

trained itself on the training data and then the forecast engine was tested for whichresult is shown in Table 4-5

Table 4-5 Results of short term wind power prediction

MSE Accuracy %

Testing 0.0072 91.75

The testing plot is shown in the Figure 4-8. From the plot it can be seen that the actual

and the predicted plots are close on the whole region. But if the plot is seen in the



25

between 45-110 (x-axis) region, it can be seen that the actual and predicted values in

that region are not much close to each other i.e. there is error involve for those values.

4.3 GRNNThe data set used is taken from Morrisville State College [13]. The data set was first

applied to the irrelevancy filter in the form of feature set given in equation (3.2), the

irrelevancy filter discarded those features that were irrelevant to the target variable i.e.

wind power. After that the relevant features were passed to redundancy filter which

discarded the redundant features among relevant features. Then the filtered set was

given to the GRNN for training. 25% of the data set was used for testing and 75% for

training. Results obtained are given in Table 4-6

Table 4-6 Results on GRNN

MSE Accuracy %

Testing 0.0426 77.81

Figure 4-8 Testing plot of short term wind power prediction

The testing plot is shown in the Figure 4-9. From the plot it can be seen that the actual

and the predicted plots are comparable in the region from 150 to 328, but in the

remaining region the two plots are at distant from each other. Therefore we can say

that the values contributing to the error are lying in the region 1-150.



26

4.4 SVR

The data set used is taken from Morrisville State College [13]. The data set was first

applied to the irrelevancy filter in the form of feature set given in equation (3.2), the

irrelevancy filter discarded those features that were irrelevant to the target variable i.e.

wind power. After that the relevant features were passed to redundancy filter which

discarded the redundant features among relevant features. Then the filtered set was

given to the SVR for training. 25% of the data set was used for training and 75% for

training. Results obtained are given in Table 4-7

Table 4-7 Results on SVR

MSE Accuracy %

Testing 0.0038 97.62

The testing plot is shown in the Figure 4-10. From the plot it can be seen that the

actual and the predicted plots are almost same on the whole region. But if we see the

plot carefully the values contributing to error are lying in the region 45-75. Hence

SVR is giving much better prediction accuracy, which is essential in wind power

prediction.

Figure 4-9 Testing plot of GRNN



27

Figure 4-10 Testing plot of SVR



28

Chapter 5 : Conclusion and Future Work

5.1 Conclusion

Different models were applied to predict wind power accurately; it includes Grassi

model of ANN, GRNN, SVR and hybrid model of ANN with EPSO along with two

stage feature selection module which comprised of irrelevancy filter and redundancy

filter. Models only involving ANN i.e. Grassi model of ANN and GRNN are not

providing good results of prediction. The main reason behind this was trapping of the

ANN in the local minima i.e. the weight solution provided by the learning algorithm

was sub optimal, which is one of the weaknesses that ANN have. Compare to Grassi

ANN and GRNN models hybrid model of ANN with EPSO along with feature

selection module performed well, because in this model the local minima trapping

problem was almost solved by the EPSO. EPSO has a property that if sub optimal

solution is acquired then in the next step random step is provided to the particles in

the swarm which will help the particles in the swarm to get out of the local minima. In

comparison to all models SVR performed very well, because SVR is free of many

problems that ANN has e.g. trapping in local minima because it is based on structural

risk minimization. From all these it can conclude that SVR model is providing good

results for our problem.

5.2 Future Work

The data set used [13] in short term wind power prediction is taken from the internet.

Since there are also wind farms on short scales in Pakistan that are operating in some

regions, the work presented in this thesis can be checked on those stations and their

performance can be evaluated in real time situations. Since the operating stations of

wind farm in Pakistan are not of commercial scale, the models that are mentioned

here require some historical data for training so that they can predict wind power for

future. Therefore, by working on wind power prediction system from earlier along

with the other works on wind farms will help a lot in the time when the stations will

opened for commercial purposes. Additionally, the work presented here mostly

comprises of predicting wind power on short term basis because most of the timeshort term wind power predictions are needed, but there are also situations where we



29

need long term wind power predictions. The models that are presented here can also

work for long term wind power prediction we only have to change the data set on

which the model is trained. Therefore in future these models can be tested for long

term wind power predictions and changes can be made in order to achieve highaccuracy.



30

References [1] M. C. Mabel and E. Fernandez, “Analysis of wind power generation and

prediction using ANN: A case study”, Renewable Energy, Vol. 33, pp. 986 – 992,

2008.

[2] G. Grassi and P. Vecchio, “Wind energy prediction using a two-hidden layer

neural network”, Communications in Nonlinear Science and Numerical

Simulation, Vol. 15, pp. 2262 – 2266, 2010.

[3] N. Amjady, F. Keynia, H. Zareipour, “Wind Power Prediction by a New

Forecast Engine Composed of Modified Hybrid Neural Network and Enhanced

Particle Swarm Optimization”, IEEE Transactions on Sustainable Energy, Vol.

2, no. 3, pp. 265-276, July 2011.

[4] Andries P. Engelbrecht, Computational Intelligence an Introduction, 2nd Edition,

John Wiley & Sons Inc., 2007.

[5] R. Jursa and K. Rohrig, “Short-term wind power forecasting using evolutionary

algorithms for the automated specification of artificial intelligence models”,

International Journal of Forecasting , Vol. 24, pp. 694 – 709, 2008.[6] N. Amjady and F. Keynia, “Day-Ahead Price Forecasting of Electricity Markets

by Mutual Information Technique and Cascaded Neuro-Evolutionary

Algorithm”, IEEE Transactions on Power Systems, Vol. 24, pp. 306-318,

February 2009.

[7] Wikipedia, “Radial Basis Function Neural Network” [Online]. Available

http://en.wikipedia.org/wiki/Radial_basis_function_network

[8] Banyan Research Express, “Support vector regression” [Online]. Available:

http://research.ncku.edu.tw/re/articles/e/20080620/3.html

[9] Kyushu University, “Efficient Learning Algorithms for Support Vector

Machines” [Online]. Available:

http://www-kairo.csce.kyushu-u.ac.jp/~norikazu/research.en.html

[10] DTREG, “Probabilistic and General Regression Neural Networks” [Online].

Available: http://www.dtreg.com/pnn.htm

[11] Artificial Intelligence Webring, “General Regression Neural Networks”

[Online]. Available:



31

http://www.lwebzem.com/computer_science_algorithms/2011/10/09/general-

regression-neural-networks/

[12] N. Amjady, A. Daraeepour, F. Keynia, “Day-ahead electricity price forecasting

by modified relief algorithm and hybrid neural network”, IET General on

Transmission Distribution, Vol. 4, pp. 432 – 444, 2010.

[13] Morrisville State College, “Data Set”

[Online].Available:http://www.morrisville.edu/alternativeenergy/windturbine.as

px

[14] Broomhead, D. S.; Lowe, David (1988). Radial basis functions, multi-variable

functional interpolation and adaptive networks (Technical report). RSRE. 4148.

[15]

Broomhead, D. S.; Lowe, David (1988). "Multivariable functional interpolationand adaptive networks". Complex Systems 2: 321 – 355.

[16] Schwenker, Friedhelm; Kestler, Hans A.; Palm, Günther (2001). "Three

learning phases for radial-basis-function networks". Neural Networks 14: 439 –

458.

[17] Park, J.; I. W. Sandberg (Summer 1991). "Universal Approximation Using

Radial-Basis-Function Networks". Neural Computation 3 (2): 246 – 257.

doi:10.1162/neco.1991.3.2.246. Retrieved 26 March 2013.

[18] J. Moody and C. J. Darken, "Fast learning in networks of locally tuned

processing units," Neural Computation, 1, 281-294 (1989). Also see Radial

basis function networks according to Moody and Darken

[19] T. Poggio and F. Girosi, "Networks for approximation and learning," Proc.

IEEE 78(9), 1484-1487 (1990).

[20] Roger D. Jones, Y. C. Lee, C. W. Barnes, G. W. Flake, K. Lee, P. S. Lewis,

and S. Qian, ?Function approximation and time series prediction with neural

networks,? Proceedings of the International Joint Conference on Neural

Networks, June 17-21, p. I-649 (1990).

[21] Martin D. Buhmann (2003). Radial Basis Functions: Theory and

Implementations. Cambridge University. ISBN 0-521-63338-9.

[22] Yee, Paul V. and Haykin, Simon (2001). Regularized Radial Basis Function

Networks: Theory and Applications. John Wiley. ISBN 0-471-35349-3.

[23] John R. Davies, Stephen V. Coggeshall, Roger D. Jones, and Daniel Schutzer,

"Intelligent Security Systems," in Freedman, Roy S., Flein, Robert A., and



32

Lederman, Jess, Editors (1995). Artificial Intelligence in the Capital Markets.

Chicago: Irwin. ISBN 1-55738-811-3.

[24] Simon Haykin (1999). Neural Networks: A Comprehensive Foundation (2nd

edition ed.). Upper Saddle River, NJ: Prentice Hall. ISBN 0-13-908385-5.[25] S. Chen, C. F. N. Cowan, and P. M. Grant, "Orthogonal Least Squares

Learning Algorithm for Radial Basis Function Networks", IEEE Transactions

on Neural Networks, Vol 2, No 2 (Mar) 1991.

wind power prediction using artificial neural networks and evolu8th semesterfinal thesis

Documents