probability distribution and deviation information fusion...
TRANSCRIPT
Research ArticleProbability Distribution and Deviation Information FusionDriven Support Vector Regression Model and Its Application
Changhao Fan and Xuefeng Yan
Key Laboratory of Advanced Control and Optimization for Chemical Processes of Ministry of EducationEast China University of Science and Technology MeiLong Road No 130 Shanghai 200237 China
Correspondence should be addressed to Xuefeng Yan xfyanecusteducn
Received 29 June 2017 Revised 25 August 2017 Accepted 30 August 2017 Published 12 October 2017
Academic Editor Xinkai Chen
Copyright copy 2017 Changhao Fan and Xuefeng Yan This is an open access article distributed under the Creative CommonsAttribution License which permits unrestricted use distribution and reproduction in any medium provided the original work isproperly cited
Inmodeling only information from the deviation between the output of the support vector regression (SVR)model and the trainingsample is considered whereas the other prior information of the training sample such as probability distribution information isignored Probabilistic distribution information describes the overall distribution of sample data in a training sample that containsdifferent degrees of noise and potential outliers as well as helping develop a high-accuracy model To mine and use the probabilitydistribution information of a training sample a new support vector regression model that incorporates probability distributioninformation weight SVR (PDISVR) is proposed In the PDISVR model the probability distribution of each sample is consideredas the weight and is then introduced into the error coefficient and slack variables of SVR Thus the deviation and probabilitydistribution information of the training sample are both used in the PDISVRmodel to eliminate the influence of noise and outliersin the training sample and to improve predictive performance Furthermore exampleswith different degrees of noisewere employedto demonstrate the performance of PDISVR which was then compared with those of three SVR-basedmethodsThe results showedthat PDISVR performs better than the three other methods
1 Introduction
Since its proposal by Vapnik the support vector machine(SVM) has been used in many areas including both patternrecognition and regression estimation [1 2] The originalSVM is utilized to provide a pair of parameters as a solutionto a quadratic program problem SVM has some advantagessuch as low standard deviation and easy generation as well assome disadvantages such as the redundancy of the regressionfunction and the low efficiency of support vector selectionTo address these disadvantages various improvements to thesupport vector algorithm and its kernel function have beenproposed Suykens proposed least-square support vectorregression (LS-SVR) for a regression modeling problem [34] By transferring inequality constraints to equality con-straints LS-SVR simplifies the solution to quadratic programproblems [5] In the field of regression Smola proposedthe linear programming support vector regression (LP-SVR)model [6 7] LP-SVR has numerous strengths such as the
using of more general kernel functions and fast learningability LP-SVR can control the accuracy and sparseness ofthe original SVR by using the linear kernel combination as asolution approach In addition a new kernel function multi-kernel function (MK) has been introduced into the standardSVM model MK provides lower fault and requires a shortertraining period than the original kernel function Multiple-kernel SVR (MKSVR) is very popular in some systems Yeh etal [8] developed MKSVR for stock market forecasts Lin andJhuo [9] discovered a method to generate MKSVR param-eters for integration into a system that converts the pixelsof a checkpoint into the brightness value Zhong and Carr[10] used theMKSVRmodel to estimate pure and impure car-bondioxide-oilmatrixmetalloproteinases in aCO2 enhancedoil recovery process
The SVR model also has been improved by prior knowl-edge [11 12] There are numerous types of prior knowledgeincluding the average value and monotonicity of the sampledata To appropriately use prior knowledge three types of
HindawiMathematical Problems in EngineeringVolume 2017 Article ID 9650769 11 pageshttpsdoiorg10115520179650769
2 Mathematical Problems in Engineering
methods are utilized in SVR [13] Our team previouslyworked on the monotonous a priori knowledge of sampledata Our monotonous a priori knowledge of the sample datais described by first-order difference inequality constraints ofkernel expansion and additive kernels [14] The constraintsare directly added to kernel formulation to acquire a convexoptimization problem For additive kernels SVMs are con-ducted through the addition of dissociate kernels for everyinput dimensionThese operations confer higher accuracy tothe SVR model in support vector (SV) selection
Inevitably even small noise can debase the accuracy ofthe model Furthermore in some situations part of the noisyinformation may be ten to even dozens of times larger thanthe normal data These outliers introduce bias and inac-curacies to SVR Nevertheless the probability distributionof the sample data is a good indicator of noise From theperspective of the probability distribution of sample datanormal data and data that contain the least amount of noisehave the highest probability in the sample data By contrastdata that contain large amount of noise have relatively smallprobability Thus outliers in the sample data will have thesmallest probability Therefore the probability distribution isthe prior knowledge that helps weaken the influence fromnoise and outliers in the sample data We consider thisinformation to modify our SVR model
This article is structured as follows Section 2 introducesstandard SVR algorithms Section 3 describes the proposedalgorithm that integrates probability distribution informa-tion into the SVR framework Section 4 provides someexperimental results that were obtained from comparing theproposed algorithm with other algorithms Finally Section 5presents some conclusions about the proposed algorithm
2 Review of SVR
To better describe the proposed algorism the mathematicalclarification of the basic concepts of SVR and the usage ofdeviation information should be provided
21 Support Vector Regression (SVR) SVR is originally usedto solve linear regression problems For given training sam-ples X = (x119894 119910119894) | x119894 isin 119877119889 119910119894 isin 119877 119894 = 1 2 119899 fittingaims to find the dependency between the independent vari-able x119894 and the dependent variable 119910119894 Specifically it aims toidentify an optimal function and minimize prospective risk119877(120596) = int 119871(119910 119891(x120596))119889119875(x 119910) where 119891(x120596) is predictivefunction set 120596 isin Ω is the generalized parameters of thefunction 119871(119910 119891(x120596)) is the loss function and
119891 (x120596) = 120596 sdot x + 119887 (1)
is the fitting function [15] Thus the solution of the optimallinear function for SVR is expressed as the following con-straint optimization problem
min 119877 (120596 120585119894 120585lowast119894 ) = 12 ⟨120596 sdot 120596⟩ + 119862 119899sum119894=1
(120585119894 + 120585lowast119894)
120596 sdot x minus 119910119894 + 119887 le 120576 + 120585lowast119894
st 119910119894 minus 120596 sdot x minus 119887 le 120576 + 120585119894120585119894 120585lowast119894 ge 0 119894 = 1 2 119899
(2)
where the penalty coefficient C that determines the accuracyof the function fitting and the degree of the error greaterthan 120576 is given in advance Parameter 120576 is used to control thesize of the fitting error the size of the support vector andthe size of the generalization capability Taking into accountthe accuracy of the fitting error the introduction of slackvariables 120585119894 120585lowast119894 becomes necessary Figure 1 in reference [10]illustrates this linear fitting problem
However the previous solution is only for a linear regres-sion problem Nonlinear regression necessitates the kernelfunction in the SVR model [16] The kernel function can beexpressed as follows
K (x z) = ⟨120601 (x) sdot 120601 (z)⟩ (3)
where 120601 is the mapping from a low-dimensional spaceto a high-dimensional space The independent variable x119894becomes a vector that should be mapped to a feature space sothat a nonlinear problem could be changed into a linear prob-lem After introducing the kernel function the new fittingfunction becomes
119891 (x) = 119899sum119894=1
120596119894119896 (x xi) + 119887 = K (xX119879)120596 + 119887 (4)
where the symbol X119879 indicates the transpose of the matrixX
The changing of the fitting function leads to the followingconstraint optimization problem
min 119877 (120596 120585119894 120585lowast119894 ) = 12 ⟨120596 sdot 120596⟩ + 119862 119899sum119894=1
(120585119894 + 120585lowast119894)
K (xX119879)120596 minus 119910119894 + 119887 le 120576 + 120585lowast119894
st 119910119894 minus K (xX119879)120596 minus 119887 le 120576 + 120585119894120585119894 120585lowast119894 ge 0 119894 = 1 2 119899
(5)
In this constraint optimization problem the length of 120596 andx is n The notion K(sdot sdot) is the kernel function that fulfillsMercerrsquos requirements
The standard SVR is a compromise between structuralrisk minimization and empirical risk minimization In par-ticular for the support vector regression learning algorithmthe structural risk term is (12)⟨120596 sdot 120596⟩ and the empirical riskitem is sum119899
119894=1(120585119894 + 120585lowast
119894) However calculating the structural risk
term (12)⟨120596 sdot120596⟩ requires enormous time and resources [17]Researchers found counting the minimization of the 1-normof the parameter 120596 will reduce the time and resources spent
Mathematical Problems in Engineering 3
on calculationThen the optimization formula turns into thefollowing form
min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1
(120585119894 + 120585lowast119894)
K (xX119879)120596 minus 119910119894 + 119887 le 120576 + 120585lowast119894
st 119910119894 minus K (xX119879)120596 minus 119887 le 120576 + 120585119894120585119894 120585lowast119894 ge 0 119894 = 1 2 119899
(6)
Although the time and resource spent on modeling arereduced there is no considerable difference in the finalaccuracy
22 Support Vector Regression with Deviation Information asa Consideration Traditional SVR does not possess a specialmethod for addressing noise in sample data An efficient wayto weaken noise is to adjust parameters in the SVR modelThese parameters are called hyperparameters in SVRsHyperparameters exert a considerable impact on algorithmperformance The general way to test the performance ofhyperparameters is via the deviation between the model out-put and the sample data [18] The obtained deviation is thencompared with other deviations to select the minimum devi-ation as the final resultThe parameters that correspond to theminimum deviation are the best parameters in the optimiza-tion process Usually this process is conducted using an intel-ligent optimization algorithm such as particle swarm opti-mization (PSO) [19] and genetic algorithm (GA) [20] Thedeviation is set as the fitness function in an intelligent opti-mization algorithm In this section we refer to thismethod asdeviation-minimized SVR (DM-SVR)
In most of the circumstances the deviation betweenthe model output and sample data is represented by thecorrelation coefficient 119903 or the mean square error (MSE)Given vector y as the model output and vector y as thesample output the correlation coefficient r can be expressedas
119903= sum119873
119894=1119894119910119894 minus sum119873
119894=1119894sum119873119894=1
119910119894119873radic(sum119873119894=1
119894
2 minus (sum119873119894=1
119894)2 119873) (sum119873
119894=11199101198942 minus (sum119873
119894=1119910119894)2 119873)
(7)
The formula for mean square error (MSE) is as follows
MSE = sum119873119894=1
(1003816100381610038161003816119894 minus 1199101198941003816100381610038161003816)2119873 (8)
In short if the value of MSE is close to zero and the value of ris close to one that group of parameters will produce the bestperformance
3 Probability Distribution InformationWeighted Support Vector Regression
Although DM-SVR can reduce influence from the noise italso has some weaknesses The main disadvantage of thismethod is the time it spends on training There are manyparameters that need to be optimized in SVR If thereare extra parameters to optimize these works would makethe train process inefficient To solve the uncertainty oferror parameter 120576 we introduce the probability distributioninformation (PDI) into SVR and designate it as PDISVR
31 Probability Distribution of the Output The probabilitydistribution information is the same as the probability distri-bution function and describes the likelihood that the outputvalue of a continuous random variable is near a certain pointIntegrating the probability density function is the proper wayto calculate the probability value of the random variable inthat certain region From the sample data we could set thefrequency of output to appear as different values Then weset frequency as119873(y) where y is the output value vector Let119875(y) be the probability of the samplersquos output Therefore therelationship between 119875(y) and119873(y) can be expressed as
119875 (y) = 119873 (y)sumyisinY 119873(y) (9)
where Y is the range of y Then we can easily obtainthe probability distribution function The next step is theidentification of the probability of every point
32 Optimization Formula with Probability Distribution Infor-mation Weight Once we have obtained the probability dis-tribution of output it should be integrated into the basicSVR model In the basic SVR model the error parameter 120576indicates the accuracy of model fitting by providing an areathat does not have any loss for the objective function How-ever due to the influence of noise some sample data containexcessive noise information If the same parameters 120576 areadapted the performance of themodel is reduced To preventthis situation SVR should be adjusted in accordance withnoise informationWe propose illustrating noise informationthrough the probability distribution of the output Samples inthe regionswith low probability distributions have a relativelylarge proportion of noise For this reason in modeling theregion with higher probability should have a smaller errorparameter than the lower probability regionThus the proba-bility distribution function increases the accuracy of the SVRmodel in the area with the high probability of output
Define the 120576-insensitive loss function as
1003816100381610038161003816y minus 119891 (x)1003816100381610038161003816120576 = 0 if 1003816100381610038161003816y minus 119891 (x)1003816100381610038161003816 le 1205761003816100381610038161003816y minus 119891 (x)1003816100381610038161003816 minus 120576 otherwise (10)
where119891(x) is a regression estimation function constructed bylearning the sample and y is the target output value that cor-responds to x By defining the 120576-insensitive loss function theSVR model specifies the error requirement of the estimated
4 Mathematical Problems in Engineering
Feature space
0+
minus
Figure 1 Linear problem illustration of a PDISVR model
function on the sample dataThis requirement is the same forevery sample point To avoid this situation the artificially seterror parameter 120576 is divided by the probability distributionvector 119875(y) Figure 1 illustrates the change from a constant120576 to a vector 120576 The distance between two hyperplanes hasbeen modified in some area where the density of the pointsbecomes different Furthermore in the high-density areathe model has a smaller error parameter By contrast inthe low-density area the model has a large error parameterThe density of the output points is directly related to theprobability of the samplersquos output y Therefore the divisionof PDI would make the SVR model emphasize the area witha high density of points This technique can improve overallaccuracy despite sacrificing the accuracy of low-density areasAccording to (9) and (10) the PDISVR can be expressedas
min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1
(120585119894 + 120585lowast119894)
K (xX119879)120596 minus 119910119894 + 119887 le 120576119875 (119910119894) + 120585lowast119894
st 119910119894 minus K (xX119879)120596 minus 119887 le 120576119875 (119910119894) + 120585119894120585119894 120585lowast119894 ge 0 119894 = 1 2 119899
(11)
By comparing (11) with the standard form of SVR we cansee that the error parameter 120576 changes in accordance with 119910119894Then the PDISVRmodel will have low error tolerance for thehigh density of points
To further improve the performance of the SVR modelwe consider adding an extra fragment to the PDISVR frame-work The PDISVR model only has a unique error parameter120576 However 120576 is too small to have an obvious impact onthe accuracy of the model Hence we propose an additionalmethod to introduce PDI that is we apply the same operationas that on error parameter 120576 on the slack variable 120585 Given thetreated error parameter 120576 in the PDISVR model we divided
Table 1 Typical parameters for PSO algorithms
Parameter ValueMaximum generation 200Population size 20Cognitive efficient (1199061) 15maximum velocity (119881max) 1Social efficient (1199062) 17Initial inertia weights 09Final inertia weights 04minimum velocity (119881min) 10
the slack variable 120585 by probability distribution information119875(y) Subsequently the final PDISVR model is
min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1
(120585119894 + 120585lowast119894)
K (xX119879)120596 minus 119910119894 + 119887 le 1119875 (119910119894) (120576 + 120585lowast119894)
st 119910119894 minus K (xX119879)120596 minus 119887 le 1119875 (119910119894) (120576 + 120585119894)120585119894 120585lowast119894 ge 0 119894 = 1 2 119899
(12)
33 Parameters Optimization Based on PSO Normally theperformance of the different SVRs is intensively dependenton the parameters selection A PSO based hyperparametersselecting method in [10] is used in this paper After datasetwas normalized the control parameters of PSO includingmaximum velocity (119881max) minimum velocity (119881min) initialinertia weight (120596min) final inertia weight (120596max) cognitivecoefficient (1206011) social coefficient (1206012) maximum generationand population size should be initialized according to theexperience of operators In our experiment we set thecontrol parameters of PSO based on Table 1 In the followingexperiments there are at most five hyperparameters ofdifferent SVRs that need to be optimized in PSO Theseparameters include penalizing coefficient (C) radial basisfunction kernel parameter (120574) and polynomial degree (d)(120576) in 120576-insensitivity function and mixing coefficient (m) inmultikernel function In our experiments different compar-ative methods adapt different groups of parameters and inorder to search the global optimum reasonably the parameterm is limited in [0 1] 120574 in [10minus2 10] d in [1 10] C in[10 1000] and 120576 in [10minus4 10minus1] During the process ofsearching best parameters by PSO particles update theirpositions by changing velocity and converge finally at a globaloptimum within the searching space In this study the parti-clesrsquo positions are the combination ofm 120574 d C and 120576 whichare denominated as P Then V-fold (V = 5) cross-validationresampling method is applied to validate the performanceof searched best parameters until criterions are met Toevaluate the performance of training process mean squareerror (MSE) is chosen as the fitness function which isformulated as (8) Figure 2 shows the workflow to find theoptimum values of each parameter in models
Mathematical Problems in Engineering 5
Start
Prepare andnormalize dataset
Initialize control parameters of PSO including
generation population size
Set parameter P for SVR
V-fold cross-validationaccuracy
NoOr
Testing SVR based on theoptimized parameters
Yes
Predict result using well-trained SVR model
End
[VGCH VGR] [GCH GR] 1 2 maximum
Search best value m dC by PSO algorithm
Figure 2 Workflow on hyperparametersrsquo selection
4 Experimental Results
To verify the effect of the probability distribution informationon the standard SVR model we employed three kinds ofnumerical experiments with real datasets In these experi-ments we considered three kernels including linear kernelpolynomial kernel andGaussian kernel as SVRsrsquo kernel func-tions All of the experiments were operated onMATLABwithIntel i5 CPU and 6 GB internal storage
Experimental studies have mainly compared differentSVR models including basic SVR MKSVR and heuristicweighted SVR [21] The correlation coefficient r and meansquare error (MSE) are used to evaluate generalization per-formance The formulas of these two criteria are listed inSection 22
41 Example 1 Example 1 tested the previous four methodswith one-high-dimensional functions as shown in [22]
119891 (x) = 119899sum119894=1
[ln (119909119894 minus 2)]2 + [ln (10 minus 119909119894)]2
minus ( 119899prod119894=1
119909119894)02
+ 119890119894 (119896) 21 le 119909119894 le 99 (119899 = 3)
(13)
In the above function the symbol 119890119894(119896) indicates the randomfluctuation variable between minus119896 and k From the range of [2199] for 119909119894 above we generated 100 data at random Through(13) we obtained the output of these 100 data We evenlydivided the 100 data into five parts which comprised fourtraining parts and one testing part After the cross-validationmethod introduced in Section 33 optimal hyperparametersfor different SVR algorithms are selected
After obtaining the optimal hyperparameters we thendetermined the influence of noise on different SVR algo-rithmsThe range of the magnitude of the noise k which wasset as 01 05 1 3 6 and 10 was set in accordance with theoutput range To obtain objective comparisons 10 groups ofnoise were added to each training sample of the algorithmusing the MATLAB toolbox which completely generated10 training datasets Moreover testing data were directlygenerated from the objective function equation (13) Theresults for the criterion of these ten experiments are recordedby their average and standard deviation values as shown inFigures 3ndash5
In these three figures the average criterion values indi-cated that the general performance of the algorithm and thestandard deviation are representative of the algorithmrsquos stabil-ity From Figures 3ndash5 we can see that the performance of theproposed PDISVR is less affected by noisy information thanthose of the other three SVR algorithms In Figure 3 the result
6 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 8 9 10045
05055
06065
07075
08085
09095
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(a)
0 1 2 3 4 5 6 7 8 9 100
5
10
15
20
25
30
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(b)
0 1 2 3 4 5 6 7 8 9 100
001
002
003
004
005
006
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(c)
0 1 2 3 4 5 6 7 8 9 100
20406080
100120140160
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(d)
Figure 3 Four modelsrsquo prediction results in linear kernel with different noise (a) is the average value of correlation coefficient 119903 (b) is theaverage value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
line of PDISVR is more stable than other three methods Andit achieves best prediction performance when adding largernoise in samples among all models Compared to Figure 3the PDISVRrsquos ability to predict is not always the best inFigures 4 and 5 That means the PDISVR with linear kernelis suitable for this dataset Besides in the area with largeintensity of noise the basic SVR andMKSVR poorly handledthe effects of noise Although HW-LSSVR resisted some ofthe effects of noise its performance slightly worsened with ahigh intensity of noisy samplesThe average of the predictionaccuracy and standard deviation of PDISVR were relativelybetter in fitting models with noise of 1 3 6 and 10 Withnoise of 01 and 05 although the differences among theaverage values were small the PDISVR was more stable thanother algorithms in some certain circumstances
42 Example 2 The effects from rough error cannot beignored in real production processes To better simulate realproduction conditions and reveal the robustness of the pro-posed PDISVR when the training samples involved outliersthe rough error term should be added to the function in theprevious model A total of 80 data with a noise intensity of 1were haphazardly generated by (13) as a fundamental trainingsample Test samples containing 20 data were also generated
by (13) Then the dependent variables of the 17th and 48thdata in the fundamental training sample were attached 45times119890and 3 times 119890 respectively as two trivial outliers The dependentvariable of the 50th datum in the fundamental trainingsample was attached 10 times 119890 as one strong outlier Thus thenew training sample that contained one strong outlier thatis the 50th datum and two trivial outliers that is the 17thand 48th data was constructed
To better compare the predictive performance of the dif-ferent SVR algorithms the same four algorithmswere trainedten times in samples with three outliers The average valuesand standard deviation values of 119903 and MSE represented theperformance of these algorithms
As indicated in Tables 2ndash4 the PDISVR algorithm per-formed better in the testing experiments than the other algo-rithms The unweighted SVR and MKSVR were influencedby noise and produced biased estimates in predicting resultswhereas PDISVRdramatically reduced this secondary actionGiven the misjudgments on the outliers in this complicatedsystem the HWSVR algorithm could not obtain satisfactoryresults even when it adapted weighted error parameters
In order to illustrate the quality of PID-SVRrsquos weightelement we compared the weighting results in Table 2 Theweight values of the PDISVR algorithm in training samples
Mathematical Problems in Engineering 7
0 1 2 3 4 5 6 7 8 9 10
04
05
06
07
08
09
1
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(a)
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
6
7
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(b)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
0005001
0015002
0025003
0035004
0045
Noise intensity
Aver
age v
alue
(c)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
051
152
253
354
45
Noise intensity
Stan
dard
dev
iatio
n
(d)
Figure 4 Four modelsrsquo prediction results in radial basis function kernel with different noise (a) is the average value of correlation coefficient119903 (b) is the average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
Table 2 Testing results of SVR algorithms with rough error (linear kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 089278 086844 084402 085828Standard deviation 0000258 0002867 0002707 0001998
MSEAverage value 030614 052311 058221 045612Standard deviation 0003953 0111161 0045897 0028321
Table 3 Testing results of SVR algorithms with rough error (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 078424 078126 075976 064346Standard deviation 0000297 0000169 0000774 0000244
MSEAverage value 070686 073748 062362 084296Standard deviation 000316 000150 000506 000136
are listed in Figure 6 The weight values of the HWSVRalgorithm are listed in Figure 7 As shown in Figure 6 theweights of two trivial outliers were 0 and 010531 for the 17thand 48th data respectively and the weight of one strong out-lier was 000036 which indicated that the PDISVR precisely
detected the outliers As shown in Figure 7 the HWSVR didnot perform as well as PDISVR One strong outlier had aweight of 00751 and two trivial outliers had weights of 03143and 02729 which were unsuitable for modeling given thatsmaller weights such as that of the 23rd datum (00126)
8 Mathematical Problems in Engineering
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 10
04
05
06
07
08
09
1
Noise intensity
Aver
age v
alue
(a)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 1002468
1012141618
Noise intensity
Aver
age v
alue
(b)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
0005
001
0015
002
0025
003
0035
Noise intensity
Stan
dard
dev
iatio
n
(c)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
50
100
150
200
250
Noise intensity
Stan
dard
dev
iatio
n
(d)
Figure 5 Four modelsrsquo prediction results in polynomial kernel with different noise (a) is the average value of correlation coefficient 119903 (b) isthe average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
Table 4 Testing results of SVR algorithms with rough error (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 083278 079399 075874 08137Standard deviation 0000377 0004657 0009121 0003568
MSEAverage value 04473 0727 148208 062433Standard deviation 0003168 0076291 1462867 0028654
could affect outlier detection Thus the influence from twotrivial outliers on the predictability of the PDISVR algorithmwas reduced and the influence from one strong outlier waseliminated By contrast the effect of outliers remained in theHWSVR algorithm
43 Example 3 To test our regressionmodel in amore realis-tic way we imported six more realistic datasets from the UCIMachine Learning Repository [23ndash25] Department of FoodScience University of Copenhagen database [26] and somereal chemical industrial process [14] See Table 5 for moredetailed information In these datasets four out of five datawere used for training and one-fifth of the data was used for
testing The hyperparameters used in this example are alsoobtained by the process introduced in Section 33
As shown in Tables 6ndash8 the proposed PDISVR obtainedthe best predictive ability in the majority of the criterion Forexample in the case of Auto-MPG the proposed PDISVRwas best achieved with both standards Thus the proposedPDISVR is appropriate for the Auto-MPG dataset In thedatasets for crude oil distillation and computer hardwarethe proposed PDISVR only obtained the best correlationcoefficient r and could not establish a suitablemodel at a pointwhere the probability distribution was low thus increasingthe MSE Therefore the use of PDISVR requires validation
Mathematical Problems in Engineering 9
Table 5 Details of the experimental datasets
Data Number of instances Number of attributes Dataset information
Combustion side reaction 144 6The combustion side reaction is the process by which
the solvent and reactants are oxidized to carbondioxide carbon monoxide and water
Auto-MPG dataset 398 8 Data concerning city-cycle fuel consumption
Computer hardware dataset 209 9 Relative CPU performance in terms of cycling timememory size and other parameters
Soil samples measured byNear-Infrared Spectroscopy(NIR)
108 1050 This data is about the soil organic matter (SOM) fromnorthern Sweden (681210N 181490E)
Energy efficiency 768 9This study looked into assessing the heating load andcooling load requirements of buildings (that is energy
efficiency) as a function of building parameters
Crude oil distillation process 30 1 Data related to different components with differentboiling points
Table 6 Comparative results of previous SVR models in real datasets (linear kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 07141 06790 06797 06939MSE 00051 00050 00046 00044
Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802
Computer hardware 119903 09533 08750 08748 09012MSE 79872 83737 83815 65772
NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858
Energy efficiency 119903 1 1 1 09999MSE 00047 11092 00221 00818
Crude oil distillation process r 09372 07684 09372 09927MSE 10162 9724 14510 81628
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 010531
X = 17Y = 0
X = 50
Y = 000036228
Figure 6 Weights of the training sample for PDISVR
through additional research and dataset message Moreoverin the sample of crude oil boiling point there are far lessquantity of data and the PDISVR cannot differentiate the
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 027292
X = 17Y = 031429
X = 50
Y = 0075054
Figure 7 Weights of the training sample for HWSVR
noise according to its probability distribution Thus theproposed PDISVR is applied to improve the SVR in the caseof large datasets
10 Mathematical Problems in Engineering
Table 7 Comparative results of previous SVR models in real datasets (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 06664 0425 06031 05827MSE 00038 00095 06031 00578
Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482
Computer hardware 119903 07815 08597 08647 08851MSE 27961 19986 15355 13160
NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235
Energy efficiency 119903 1 09796 09901 09997MSE 0000269 41875 3019 00474
Crude oil distillation process 119903 09899 09739 09913 09573MSE 477425 787604 11198 3176248
Table 8 Comparative results of previous SVR models in real datasets (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 05796 06853 08078 05518MSE 00081 00106 03546 01237
Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559
Computer hardware 119903 08337 08571 08519 08423MSE 125509 195298 198641 139781
NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073
Energy efficiency 119903 1 09964 09988 09867MSE 00221 04598 01676 41448
Crude oil distillation process 119903 09571 08071 09807 09913MSE 3218016 1129353 543637 11054000
5 Conclusion
In traditional SVR modeling the deviation between themodel outputs and the real data is the only way to representthe influence of noise Other information such as possibilitydistribution information is not emphasized Therefore weproposed a special method that uses the possibility dis-tribution information to modify the basic error parameterand slack variables of SVR Given that these parametersare weighted by the probability distribution of the modeloutput points they can be adjusted by SVR itself and nolonger require optimization by any intelligent optimizationalgorithmThe proposed algorithm is superior to other SVR-based algorithms in dealing with noisy data and outliers insimulation and actual datasets
Conflicts of Interest
The authors declare that they have no conflicts of interest
References
[1] VN VapnikTheNature of Statistical LearningTheory Springer2000
[2] S Yin and J Yin ldquoTuning kernel parameters for SVM based onexpected square distance ratiordquo Information Sciences vol 370-371 pp 92ndash102 2016
[3] J A K Suykens J Vandewalle and B de Moor ldquoOptimalcontrol by least squares support vector machinesrdquo NeuralNetworks vol 14 no 1 pp 23ndash35 2001
[4] J A K Suykens J De Brabanter L Lukas and J VandewalleldquoWeighted least squares support vector machines robustnessand sparce approximationrdquo Neurocomputing vol 48 pp 85ndash105 2002
[5] H-S Tang S-T Xue R Chen and T Sato ldquoOnline weightedLS-SVM for hysteretic structural system identificationrdquo Engi-neering Structures vol 28 no 12 pp 1728ndash1735 2006
[6] A Smola B Schoelkopf and G Raetsch ldquoLinear programsfor automatic accuracy control in regressionrdquo in Proceedings of9th International Conference on Artificial Neural Networks(ICANN99) pp 575ndash580 IEE UK September 1999
[7] A Smola and B Scholkopf Learning with Kernels MIT PressCambridge UK 2002
[8] C Y Yeh CWHuang and S J Lee ldquoAmultiple-kernel supportvector regression approach for stock market price forecastingrdquoExpert Systems with Applications vol 38 no 3 pp 2177ndash21862011
[9] W-J Lin and S-S Jhuo ldquoA fast luminance inspector forbacklight modules based on multiple kernel support vector
Mathematical Problems in Engineering 11
regressionrdquo IEEE Transactions on Components Packaging andManufacturing Technology vol 4 no 8 pp 1391ndash1401 2014
[10] Z Zhong and T R Carr ldquoApplication of mixed kernels function(MKF) based support vector regressionmodel (SVR) for CO2 ndashReservoir oil minimum miscibility pressure predictionrdquo Fuelvol 184 pp 590ndash603 2016
[11] G Bloch F Lauer G Colin and Y Chamaillard ldquoSupportvector regression from simulation data and few experimentalsamplesrdquo Information Sciences vol 178 no 20 pp 3813ndash38272008
[12] J Zhou B Duan J Huang and N Li ldquoIncorporating priorknowledge and multi-kernel into linear programming supportvector regressionrdquo Soft Computing vol 19 no 7 pp 2047ndash20612015
[13] F Lauer and G Bloch ldquoIncorporating prior knowledge insupport vector machines for classification A reviewrdquo Neuro-computing vol 71 no 7-9 pp 1578ndash1594 2008
[14] C Pan Y Dong X Yan and W Zhao ldquoHybrid model for mainand side reactions of p-xylene oxidation with factor influencebased monotone additive SVRrdquo Chemometrics and IntelligentLaboratory Systems vol 136 pp 36ndash46 2014
[15] O Naghash-Almasi and M H Khooban ldquoPI adaptive LS-SVRcontrol scheme with disturbance rejection for a class of uncer-tain nonlinear systemsrdquo Engineering Applications of ArtificialIntelligence vol 52 pp 135ndash144 2016
[16] Y Wang and L Zhang ldquoA combined fault diagnosis method forpower transformer in big data environmentrdquo MathematicalProblems in Engineering vol 2017 Article ID 9670290 6 pages2017
[17] H Dai B Zhang andWWang ldquoAmultiwavelet support vectorregression method for efficient reliability assessmentrdquo Reliabil-ity Engineering and System Safety vol 136 pp 132ndash139 2015
[18] P-Y Hao ldquoPairing support vector algorithm for data regres-sionrdquo Neurocomputing vol 225 pp 174ndash187 2017
[19] W-CHong ldquoChaotic particle swarmoptimization algorithm ina support vector regression electric load forecasting modelrdquoEnergy Conversion and Management vol 50 no 1 pp 105ndash1172009
[20] G Wang T R Carr Y Ju and C Li ldquoIdentifying organic-richMarcellus Shale lithofacies by support vector machine classifierin the Appalachian basinrdquo Computers and Geosciences vol 64pp 52ndash60 2014
[21] W Wen Z Hao and X Yang ldquoA heuristic weight-setting strat-egy and iteratively updating algorithm for weighted least-squares support vector regressionrdquoNeurocomputing vol 71 no16-18 pp 3096ndash3103 2008
[22] H Xiong Z Chen H Qiu H Hao and H Xu ldquoAdaptiveSVR-HDMR metamodeling technique for high dimensionalproblemsrdquo International Conference onModelling Identificationand Control AASRI Procediapp vol 3 pp 95ndash100 2012
[23] M Lichman Auto MPG Data Set UCI Machine LearningRepository httparchiveicsuciedumldatasetsAuto+MPG2013
[24] M Lichman Computer Hardware Data Set UCI MachineLearningRepository httparchiveicsuciedumldatasetsCom-puter+Hardware 2013
[25] A Tsanas and A Xifara ldquoAccurate quantitative estimation ofenergy performance of residential buildings using statisticalmachine learning toolsrdquo Energy and Buildings vol 49 pp 560ndash567 2012
[26] R Rinnan and A Rinnan ldquoApplication of near infrared reflec-tance (NIR) and fluorescence spectroscopy to analysis ofmicro-biological and chemical properties of arctic soilrdquo Soil Biologyand Biochemistry vol 39 no 7 pp 1664ndash1673 2007
Submit your manuscripts athttpswwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
2 Mathematical Problems in Engineering
methods are utilized in SVR [13] Our team previouslyworked on the monotonous a priori knowledge of sampledata Our monotonous a priori knowledge of the sample datais described by first-order difference inequality constraints ofkernel expansion and additive kernels [14] The constraintsare directly added to kernel formulation to acquire a convexoptimization problem For additive kernels SVMs are con-ducted through the addition of dissociate kernels for everyinput dimensionThese operations confer higher accuracy tothe SVR model in support vector (SV) selection
Inevitably even small noise can debase the accuracy ofthe model Furthermore in some situations part of the noisyinformation may be ten to even dozens of times larger thanthe normal data These outliers introduce bias and inac-curacies to SVR Nevertheless the probability distributionof the sample data is a good indicator of noise From theperspective of the probability distribution of sample datanormal data and data that contain the least amount of noisehave the highest probability in the sample data By contrastdata that contain large amount of noise have relatively smallprobability Thus outliers in the sample data will have thesmallest probability Therefore the probability distribution isthe prior knowledge that helps weaken the influence fromnoise and outliers in the sample data We consider thisinformation to modify our SVR model
This article is structured as follows Section 2 introducesstandard SVR algorithms Section 3 describes the proposedalgorithm that integrates probability distribution informa-tion into the SVR framework Section 4 provides someexperimental results that were obtained from comparing theproposed algorithm with other algorithms Finally Section 5presents some conclusions about the proposed algorithm
2 Review of SVR
To better describe the proposed algorism the mathematicalclarification of the basic concepts of SVR and the usage ofdeviation information should be provided
21 Support Vector Regression (SVR) SVR is originally usedto solve linear regression problems For given training sam-ples X = (x119894 119910119894) | x119894 isin 119877119889 119910119894 isin 119877 119894 = 1 2 119899 fittingaims to find the dependency between the independent vari-able x119894 and the dependent variable 119910119894 Specifically it aims toidentify an optimal function and minimize prospective risk119877(120596) = int 119871(119910 119891(x120596))119889119875(x 119910) where 119891(x120596) is predictivefunction set 120596 isin Ω is the generalized parameters of thefunction 119871(119910 119891(x120596)) is the loss function and
119891 (x120596) = 120596 sdot x + 119887 (1)
is the fitting function [15] Thus the solution of the optimallinear function for SVR is expressed as the following con-straint optimization problem
min 119877 (120596 120585119894 120585lowast119894 ) = 12 ⟨120596 sdot 120596⟩ + 119862 119899sum119894=1
(120585119894 + 120585lowast119894)
120596 sdot x minus 119910119894 + 119887 le 120576 + 120585lowast119894
st 119910119894 minus 120596 sdot x minus 119887 le 120576 + 120585119894120585119894 120585lowast119894 ge 0 119894 = 1 2 119899
(2)
where the penalty coefficient C that determines the accuracyof the function fitting and the degree of the error greaterthan 120576 is given in advance Parameter 120576 is used to control thesize of the fitting error the size of the support vector andthe size of the generalization capability Taking into accountthe accuracy of the fitting error the introduction of slackvariables 120585119894 120585lowast119894 becomes necessary Figure 1 in reference [10]illustrates this linear fitting problem
However the previous solution is only for a linear regres-sion problem Nonlinear regression necessitates the kernelfunction in the SVR model [16] The kernel function can beexpressed as follows
K (x z) = ⟨120601 (x) sdot 120601 (z)⟩ (3)
where 120601 is the mapping from a low-dimensional spaceto a high-dimensional space The independent variable x119894becomes a vector that should be mapped to a feature space sothat a nonlinear problem could be changed into a linear prob-lem After introducing the kernel function the new fittingfunction becomes
119891 (x) = 119899sum119894=1
120596119894119896 (x xi) + 119887 = K (xX119879)120596 + 119887 (4)
where the symbol X119879 indicates the transpose of the matrixX
The changing of the fitting function leads to the followingconstraint optimization problem
min 119877 (120596 120585119894 120585lowast119894 ) = 12 ⟨120596 sdot 120596⟩ + 119862 119899sum119894=1
(120585119894 + 120585lowast119894)
K (xX119879)120596 minus 119910119894 + 119887 le 120576 + 120585lowast119894
st 119910119894 minus K (xX119879)120596 minus 119887 le 120576 + 120585119894120585119894 120585lowast119894 ge 0 119894 = 1 2 119899
(5)
In this constraint optimization problem the length of 120596 andx is n The notion K(sdot sdot) is the kernel function that fulfillsMercerrsquos requirements
The standard SVR is a compromise between structuralrisk minimization and empirical risk minimization In par-ticular for the support vector regression learning algorithmthe structural risk term is (12)⟨120596 sdot 120596⟩ and the empirical riskitem is sum119899
119894=1(120585119894 + 120585lowast
119894) However calculating the structural risk
term (12)⟨120596 sdot120596⟩ requires enormous time and resources [17]Researchers found counting the minimization of the 1-normof the parameter 120596 will reduce the time and resources spent
Mathematical Problems in Engineering 3
on calculationThen the optimization formula turns into thefollowing form
min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1
(120585119894 + 120585lowast119894)
K (xX119879)120596 minus 119910119894 + 119887 le 120576 + 120585lowast119894
st 119910119894 minus K (xX119879)120596 minus 119887 le 120576 + 120585119894120585119894 120585lowast119894 ge 0 119894 = 1 2 119899
(6)
Although the time and resource spent on modeling arereduced there is no considerable difference in the finalaccuracy
22 Support Vector Regression with Deviation Information asa Consideration Traditional SVR does not possess a specialmethod for addressing noise in sample data An efficient wayto weaken noise is to adjust parameters in the SVR modelThese parameters are called hyperparameters in SVRsHyperparameters exert a considerable impact on algorithmperformance The general way to test the performance ofhyperparameters is via the deviation between the model out-put and the sample data [18] The obtained deviation is thencompared with other deviations to select the minimum devi-ation as the final resultThe parameters that correspond to theminimum deviation are the best parameters in the optimiza-tion process Usually this process is conducted using an intel-ligent optimization algorithm such as particle swarm opti-mization (PSO) [19] and genetic algorithm (GA) [20] Thedeviation is set as the fitness function in an intelligent opti-mization algorithm In this section we refer to thismethod asdeviation-minimized SVR (DM-SVR)
In most of the circumstances the deviation betweenthe model output and sample data is represented by thecorrelation coefficient 119903 or the mean square error (MSE)Given vector y as the model output and vector y as thesample output the correlation coefficient r can be expressedas
119903= sum119873
119894=1119894119910119894 minus sum119873
119894=1119894sum119873119894=1
119910119894119873radic(sum119873119894=1
119894
2 minus (sum119873119894=1
119894)2 119873) (sum119873
119894=11199101198942 minus (sum119873
119894=1119910119894)2 119873)
(7)
The formula for mean square error (MSE) is as follows
MSE = sum119873119894=1
(1003816100381610038161003816119894 minus 1199101198941003816100381610038161003816)2119873 (8)
In short if the value of MSE is close to zero and the value of ris close to one that group of parameters will produce the bestperformance
3 Probability Distribution InformationWeighted Support Vector Regression
Although DM-SVR can reduce influence from the noise italso has some weaknesses The main disadvantage of thismethod is the time it spends on training There are manyparameters that need to be optimized in SVR If thereare extra parameters to optimize these works would makethe train process inefficient To solve the uncertainty oferror parameter 120576 we introduce the probability distributioninformation (PDI) into SVR and designate it as PDISVR
31 Probability Distribution of the Output The probabilitydistribution information is the same as the probability distri-bution function and describes the likelihood that the outputvalue of a continuous random variable is near a certain pointIntegrating the probability density function is the proper wayto calculate the probability value of the random variable inthat certain region From the sample data we could set thefrequency of output to appear as different values Then weset frequency as119873(y) where y is the output value vector Let119875(y) be the probability of the samplersquos output Therefore therelationship between 119875(y) and119873(y) can be expressed as
119875 (y) = 119873 (y)sumyisinY 119873(y) (9)
where Y is the range of y Then we can easily obtainthe probability distribution function The next step is theidentification of the probability of every point
32 Optimization Formula with Probability Distribution Infor-mation Weight Once we have obtained the probability dis-tribution of output it should be integrated into the basicSVR model In the basic SVR model the error parameter 120576indicates the accuracy of model fitting by providing an areathat does not have any loss for the objective function How-ever due to the influence of noise some sample data containexcessive noise information If the same parameters 120576 areadapted the performance of themodel is reduced To preventthis situation SVR should be adjusted in accordance withnoise informationWe propose illustrating noise informationthrough the probability distribution of the output Samples inthe regionswith low probability distributions have a relativelylarge proportion of noise For this reason in modeling theregion with higher probability should have a smaller errorparameter than the lower probability regionThus the proba-bility distribution function increases the accuracy of the SVRmodel in the area with the high probability of output
Define the 120576-insensitive loss function as
1003816100381610038161003816y minus 119891 (x)1003816100381610038161003816120576 = 0 if 1003816100381610038161003816y minus 119891 (x)1003816100381610038161003816 le 1205761003816100381610038161003816y minus 119891 (x)1003816100381610038161003816 minus 120576 otherwise (10)
where119891(x) is a regression estimation function constructed bylearning the sample and y is the target output value that cor-responds to x By defining the 120576-insensitive loss function theSVR model specifies the error requirement of the estimated
4 Mathematical Problems in Engineering
Feature space
0+
minus
Figure 1 Linear problem illustration of a PDISVR model
function on the sample dataThis requirement is the same forevery sample point To avoid this situation the artificially seterror parameter 120576 is divided by the probability distributionvector 119875(y) Figure 1 illustrates the change from a constant120576 to a vector 120576 The distance between two hyperplanes hasbeen modified in some area where the density of the pointsbecomes different Furthermore in the high-density areathe model has a smaller error parameter By contrast inthe low-density area the model has a large error parameterThe density of the output points is directly related to theprobability of the samplersquos output y Therefore the divisionof PDI would make the SVR model emphasize the area witha high density of points This technique can improve overallaccuracy despite sacrificing the accuracy of low-density areasAccording to (9) and (10) the PDISVR can be expressedas
min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1
(120585119894 + 120585lowast119894)
K (xX119879)120596 minus 119910119894 + 119887 le 120576119875 (119910119894) + 120585lowast119894
st 119910119894 minus K (xX119879)120596 minus 119887 le 120576119875 (119910119894) + 120585119894120585119894 120585lowast119894 ge 0 119894 = 1 2 119899
(11)
By comparing (11) with the standard form of SVR we cansee that the error parameter 120576 changes in accordance with 119910119894Then the PDISVRmodel will have low error tolerance for thehigh density of points
To further improve the performance of the SVR modelwe consider adding an extra fragment to the PDISVR frame-work The PDISVR model only has a unique error parameter120576 However 120576 is too small to have an obvious impact onthe accuracy of the model Hence we propose an additionalmethod to introduce PDI that is we apply the same operationas that on error parameter 120576 on the slack variable 120585 Given thetreated error parameter 120576 in the PDISVR model we divided
Table 1 Typical parameters for PSO algorithms
Parameter ValueMaximum generation 200Population size 20Cognitive efficient (1199061) 15maximum velocity (119881max) 1Social efficient (1199062) 17Initial inertia weights 09Final inertia weights 04minimum velocity (119881min) 10
the slack variable 120585 by probability distribution information119875(y) Subsequently the final PDISVR model is
min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1
(120585119894 + 120585lowast119894)
K (xX119879)120596 minus 119910119894 + 119887 le 1119875 (119910119894) (120576 + 120585lowast119894)
st 119910119894 minus K (xX119879)120596 minus 119887 le 1119875 (119910119894) (120576 + 120585119894)120585119894 120585lowast119894 ge 0 119894 = 1 2 119899
(12)
33 Parameters Optimization Based on PSO Normally theperformance of the different SVRs is intensively dependenton the parameters selection A PSO based hyperparametersselecting method in [10] is used in this paper After datasetwas normalized the control parameters of PSO includingmaximum velocity (119881max) minimum velocity (119881min) initialinertia weight (120596min) final inertia weight (120596max) cognitivecoefficient (1206011) social coefficient (1206012) maximum generationand population size should be initialized according to theexperience of operators In our experiment we set thecontrol parameters of PSO based on Table 1 In the followingexperiments there are at most five hyperparameters ofdifferent SVRs that need to be optimized in PSO Theseparameters include penalizing coefficient (C) radial basisfunction kernel parameter (120574) and polynomial degree (d)(120576) in 120576-insensitivity function and mixing coefficient (m) inmultikernel function In our experiments different compar-ative methods adapt different groups of parameters and inorder to search the global optimum reasonably the parameterm is limited in [0 1] 120574 in [10minus2 10] d in [1 10] C in[10 1000] and 120576 in [10minus4 10minus1] During the process ofsearching best parameters by PSO particles update theirpositions by changing velocity and converge finally at a globaloptimum within the searching space In this study the parti-clesrsquo positions are the combination ofm 120574 d C and 120576 whichare denominated as P Then V-fold (V = 5) cross-validationresampling method is applied to validate the performanceof searched best parameters until criterions are met Toevaluate the performance of training process mean squareerror (MSE) is chosen as the fitness function which isformulated as (8) Figure 2 shows the workflow to find theoptimum values of each parameter in models
Mathematical Problems in Engineering 5
Start
Prepare andnormalize dataset
Initialize control parameters of PSO including
generation population size
Set parameter P for SVR
V-fold cross-validationaccuracy
NoOr
Testing SVR based on theoptimized parameters
Yes
Predict result using well-trained SVR model
End
[VGCH VGR] [GCH GR] 1 2 maximum
Search best value m dC by PSO algorithm
Figure 2 Workflow on hyperparametersrsquo selection
4 Experimental Results
To verify the effect of the probability distribution informationon the standard SVR model we employed three kinds ofnumerical experiments with real datasets In these experi-ments we considered three kernels including linear kernelpolynomial kernel andGaussian kernel as SVRsrsquo kernel func-tions All of the experiments were operated onMATLABwithIntel i5 CPU and 6 GB internal storage
Experimental studies have mainly compared differentSVR models including basic SVR MKSVR and heuristicweighted SVR [21] The correlation coefficient r and meansquare error (MSE) are used to evaluate generalization per-formance The formulas of these two criteria are listed inSection 22
41 Example 1 Example 1 tested the previous four methodswith one-high-dimensional functions as shown in [22]
119891 (x) = 119899sum119894=1
[ln (119909119894 minus 2)]2 + [ln (10 minus 119909119894)]2
minus ( 119899prod119894=1
119909119894)02
+ 119890119894 (119896) 21 le 119909119894 le 99 (119899 = 3)
(13)
In the above function the symbol 119890119894(119896) indicates the randomfluctuation variable between minus119896 and k From the range of [2199] for 119909119894 above we generated 100 data at random Through(13) we obtained the output of these 100 data We evenlydivided the 100 data into five parts which comprised fourtraining parts and one testing part After the cross-validationmethod introduced in Section 33 optimal hyperparametersfor different SVR algorithms are selected
After obtaining the optimal hyperparameters we thendetermined the influence of noise on different SVR algo-rithmsThe range of the magnitude of the noise k which wasset as 01 05 1 3 6 and 10 was set in accordance with theoutput range To obtain objective comparisons 10 groups ofnoise were added to each training sample of the algorithmusing the MATLAB toolbox which completely generated10 training datasets Moreover testing data were directlygenerated from the objective function equation (13) Theresults for the criterion of these ten experiments are recordedby their average and standard deviation values as shown inFigures 3ndash5
In these three figures the average criterion values indi-cated that the general performance of the algorithm and thestandard deviation are representative of the algorithmrsquos stabil-ity From Figures 3ndash5 we can see that the performance of theproposed PDISVR is less affected by noisy information thanthose of the other three SVR algorithms In Figure 3 the result
6 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 8 9 10045
05055
06065
07075
08085
09095
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(a)
0 1 2 3 4 5 6 7 8 9 100
5
10
15
20
25
30
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(b)
0 1 2 3 4 5 6 7 8 9 100
001
002
003
004
005
006
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(c)
0 1 2 3 4 5 6 7 8 9 100
20406080
100120140160
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(d)
Figure 3 Four modelsrsquo prediction results in linear kernel with different noise (a) is the average value of correlation coefficient 119903 (b) is theaverage value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
line of PDISVR is more stable than other three methods Andit achieves best prediction performance when adding largernoise in samples among all models Compared to Figure 3the PDISVRrsquos ability to predict is not always the best inFigures 4 and 5 That means the PDISVR with linear kernelis suitable for this dataset Besides in the area with largeintensity of noise the basic SVR andMKSVR poorly handledthe effects of noise Although HW-LSSVR resisted some ofthe effects of noise its performance slightly worsened with ahigh intensity of noisy samplesThe average of the predictionaccuracy and standard deviation of PDISVR were relativelybetter in fitting models with noise of 1 3 6 and 10 Withnoise of 01 and 05 although the differences among theaverage values were small the PDISVR was more stable thanother algorithms in some certain circumstances
42 Example 2 The effects from rough error cannot beignored in real production processes To better simulate realproduction conditions and reveal the robustness of the pro-posed PDISVR when the training samples involved outliersthe rough error term should be added to the function in theprevious model A total of 80 data with a noise intensity of 1were haphazardly generated by (13) as a fundamental trainingsample Test samples containing 20 data were also generated
by (13) Then the dependent variables of the 17th and 48thdata in the fundamental training sample were attached 45times119890and 3 times 119890 respectively as two trivial outliers The dependentvariable of the 50th datum in the fundamental trainingsample was attached 10 times 119890 as one strong outlier Thus thenew training sample that contained one strong outlier thatis the 50th datum and two trivial outliers that is the 17thand 48th data was constructed
To better compare the predictive performance of the dif-ferent SVR algorithms the same four algorithmswere trainedten times in samples with three outliers The average valuesand standard deviation values of 119903 and MSE represented theperformance of these algorithms
As indicated in Tables 2ndash4 the PDISVR algorithm per-formed better in the testing experiments than the other algo-rithms The unweighted SVR and MKSVR were influencedby noise and produced biased estimates in predicting resultswhereas PDISVRdramatically reduced this secondary actionGiven the misjudgments on the outliers in this complicatedsystem the HWSVR algorithm could not obtain satisfactoryresults even when it adapted weighted error parameters
In order to illustrate the quality of PID-SVRrsquos weightelement we compared the weighting results in Table 2 Theweight values of the PDISVR algorithm in training samples
Mathematical Problems in Engineering 7
0 1 2 3 4 5 6 7 8 9 10
04
05
06
07
08
09
1
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(a)
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
6
7
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(b)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
0005001
0015002
0025003
0035004
0045
Noise intensity
Aver
age v
alue
(c)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
051
152
253
354
45
Noise intensity
Stan
dard
dev
iatio
n
(d)
Figure 4 Four modelsrsquo prediction results in radial basis function kernel with different noise (a) is the average value of correlation coefficient119903 (b) is the average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
Table 2 Testing results of SVR algorithms with rough error (linear kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 089278 086844 084402 085828Standard deviation 0000258 0002867 0002707 0001998
MSEAverage value 030614 052311 058221 045612Standard deviation 0003953 0111161 0045897 0028321
Table 3 Testing results of SVR algorithms with rough error (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 078424 078126 075976 064346Standard deviation 0000297 0000169 0000774 0000244
MSEAverage value 070686 073748 062362 084296Standard deviation 000316 000150 000506 000136
are listed in Figure 6 The weight values of the HWSVRalgorithm are listed in Figure 7 As shown in Figure 6 theweights of two trivial outliers were 0 and 010531 for the 17thand 48th data respectively and the weight of one strong out-lier was 000036 which indicated that the PDISVR precisely
detected the outliers As shown in Figure 7 the HWSVR didnot perform as well as PDISVR One strong outlier had aweight of 00751 and two trivial outliers had weights of 03143and 02729 which were unsuitable for modeling given thatsmaller weights such as that of the 23rd datum (00126)
8 Mathematical Problems in Engineering
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 10
04
05
06
07
08
09
1
Noise intensity
Aver
age v
alue
(a)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 1002468
1012141618
Noise intensity
Aver
age v
alue
(b)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
0005
001
0015
002
0025
003
0035
Noise intensity
Stan
dard
dev
iatio
n
(c)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
50
100
150
200
250
Noise intensity
Stan
dard
dev
iatio
n
(d)
Figure 5 Four modelsrsquo prediction results in polynomial kernel with different noise (a) is the average value of correlation coefficient 119903 (b) isthe average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
Table 4 Testing results of SVR algorithms with rough error (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 083278 079399 075874 08137Standard deviation 0000377 0004657 0009121 0003568
MSEAverage value 04473 0727 148208 062433Standard deviation 0003168 0076291 1462867 0028654
could affect outlier detection Thus the influence from twotrivial outliers on the predictability of the PDISVR algorithmwas reduced and the influence from one strong outlier waseliminated By contrast the effect of outliers remained in theHWSVR algorithm
43 Example 3 To test our regressionmodel in amore realis-tic way we imported six more realistic datasets from the UCIMachine Learning Repository [23ndash25] Department of FoodScience University of Copenhagen database [26] and somereal chemical industrial process [14] See Table 5 for moredetailed information In these datasets four out of five datawere used for training and one-fifth of the data was used for
testing The hyperparameters used in this example are alsoobtained by the process introduced in Section 33
As shown in Tables 6ndash8 the proposed PDISVR obtainedthe best predictive ability in the majority of the criterion Forexample in the case of Auto-MPG the proposed PDISVRwas best achieved with both standards Thus the proposedPDISVR is appropriate for the Auto-MPG dataset In thedatasets for crude oil distillation and computer hardwarethe proposed PDISVR only obtained the best correlationcoefficient r and could not establish a suitablemodel at a pointwhere the probability distribution was low thus increasingthe MSE Therefore the use of PDISVR requires validation
Mathematical Problems in Engineering 9
Table 5 Details of the experimental datasets
Data Number of instances Number of attributes Dataset information
Combustion side reaction 144 6The combustion side reaction is the process by which
the solvent and reactants are oxidized to carbondioxide carbon monoxide and water
Auto-MPG dataset 398 8 Data concerning city-cycle fuel consumption
Computer hardware dataset 209 9 Relative CPU performance in terms of cycling timememory size and other parameters
Soil samples measured byNear-Infrared Spectroscopy(NIR)
108 1050 This data is about the soil organic matter (SOM) fromnorthern Sweden (681210N 181490E)
Energy efficiency 768 9This study looked into assessing the heating load andcooling load requirements of buildings (that is energy
efficiency) as a function of building parameters
Crude oil distillation process 30 1 Data related to different components with differentboiling points
Table 6 Comparative results of previous SVR models in real datasets (linear kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 07141 06790 06797 06939MSE 00051 00050 00046 00044
Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802
Computer hardware 119903 09533 08750 08748 09012MSE 79872 83737 83815 65772
NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858
Energy efficiency 119903 1 1 1 09999MSE 00047 11092 00221 00818
Crude oil distillation process r 09372 07684 09372 09927MSE 10162 9724 14510 81628
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 010531
X = 17Y = 0
X = 50
Y = 000036228
Figure 6 Weights of the training sample for PDISVR
through additional research and dataset message Moreoverin the sample of crude oil boiling point there are far lessquantity of data and the PDISVR cannot differentiate the
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 027292
X = 17Y = 031429
X = 50
Y = 0075054
Figure 7 Weights of the training sample for HWSVR
noise according to its probability distribution Thus theproposed PDISVR is applied to improve the SVR in the caseof large datasets
10 Mathematical Problems in Engineering
Table 7 Comparative results of previous SVR models in real datasets (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 06664 0425 06031 05827MSE 00038 00095 06031 00578
Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482
Computer hardware 119903 07815 08597 08647 08851MSE 27961 19986 15355 13160
NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235
Energy efficiency 119903 1 09796 09901 09997MSE 0000269 41875 3019 00474
Crude oil distillation process 119903 09899 09739 09913 09573MSE 477425 787604 11198 3176248
Table 8 Comparative results of previous SVR models in real datasets (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 05796 06853 08078 05518MSE 00081 00106 03546 01237
Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559
Computer hardware 119903 08337 08571 08519 08423MSE 125509 195298 198641 139781
NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073
Energy efficiency 119903 1 09964 09988 09867MSE 00221 04598 01676 41448
Crude oil distillation process 119903 09571 08071 09807 09913MSE 3218016 1129353 543637 11054000
5 Conclusion
In traditional SVR modeling the deviation between themodel outputs and the real data is the only way to representthe influence of noise Other information such as possibilitydistribution information is not emphasized Therefore weproposed a special method that uses the possibility dis-tribution information to modify the basic error parameterand slack variables of SVR Given that these parametersare weighted by the probability distribution of the modeloutput points they can be adjusted by SVR itself and nolonger require optimization by any intelligent optimizationalgorithmThe proposed algorithm is superior to other SVR-based algorithms in dealing with noisy data and outliers insimulation and actual datasets
Conflicts of Interest
The authors declare that they have no conflicts of interest
References
[1] VN VapnikTheNature of Statistical LearningTheory Springer2000
[2] S Yin and J Yin ldquoTuning kernel parameters for SVM based onexpected square distance ratiordquo Information Sciences vol 370-371 pp 92ndash102 2016
[3] J A K Suykens J Vandewalle and B de Moor ldquoOptimalcontrol by least squares support vector machinesrdquo NeuralNetworks vol 14 no 1 pp 23ndash35 2001
[4] J A K Suykens J De Brabanter L Lukas and J VandewalleldquoWeighted least squares support vector machines robustnessand sparce approximationrdquo Neurocomputing vol 48 pp 85ndash105 2002
[5] H-S Tang S-T Xue R Chen and T Sato ldquoOnline weightedLS-SVM for hysteretic structural system identificationrdquo Engi-neering Structures vol 28 no 12 pp 1728ndash1735 2006
[6] A Smola B Schoelkopf and G Raetsch ldquoLinear programsfor automatic accuracy control in regressionrdquo in Proceedings of9th International Conference on Artificial Neural Networks(ICANN99) pp 575ndash580 IEE UK September 1999
[7] A Smola and B Scholkopf Learning with Kernels MIT PressCambridge UK 2002
[8] C Y Yeh CWHuang and S J Lee ldquoAmultiple-kernel supportvector regression approach for stock market price forecastingrdquoExpert Systems with Applications vol 38 no 3 pp 2177ndash21862011
[9] W-J Lin and S-S Jhuo ldquoA fast luminance inspector forbacklight modules based on multiple kernel support vector
Mathematical Problems in Engineering 11
regressionrdquo IEEE Transactions on Components Packaging andManufacturing Technology vol 4 no 8 pp 1391ndash1401 2014
[10] Z Zhong and T R Carr ldquoApplication of mixed kernels function(MKF) based support vector regressionmodel (SVR) for CO2 ndashReservoir oil minimum miscibility pressure predictionrdquo Fuelvol 184 pp 590ndash603 2016
[11] G Bloch F Lauer G Colin and Y Chamaillard ldquoSupportvector regression from simulation data and few experimentalsamplesrdquo Information Sciences vol 178 no 20 pp 3813ndash38272008
[12] J Zhou B Duan J Huang and N Li ldquoIncorporating priorknowledge and multi-kernel into linear programming supportvector regressionrdquo Soft Computing vol 19 no 7 pp 2047ndash20612015
[13] F Lauer and G Bloch ldquoIncorporating prior knowledge insupport vector machines for classification A reviewrdquo Neuro-computing vol 71 no 7-9 pp 1578ndash1594 2008
[14] C Pan Y Dong X Yan and W Zhao ldquoHybrid model for mainand side reactions of p-xylene oxidation with factor influencebased monotone additive SVRrdquo Chemometrics and IntelligentLaboratory Systems vol 136 pp 36ndash46 2014
[15] O Naghash-Almasi and M H Khooban ldquoPI adaptive LS-SVRcontrol scheme with disturbance rejection for a class of uncer-tain nonlinear systemsrdquo Engineering Applications of ArtificialIntelligence vol 52 pp 135ndash144 2016
[16] Y Wang and L Zhang ldquoA combined fault diagnosis method forpower transformer in big data environmentrdquo MathematicalProblems in Engineering vol 2017 Article ID 9670290 6 pages2017
[17] H Dai B Zhang andWWang ldquoAmultiwavelet support vectorregression method for efficient reliability assessmentrdquo Reliabil-ity Engineering and System Safety vol 136 pp 132ndash139 2015
[18] P-Y Hao ldquoPairing support vector algorithm for data regres-sionrdquo Neurocomputing vol 225 pp 174ndash187 2017
[19] W-CHong ldquoChaotic particle swarmoptimization algorithm ina support vector regression electric load forecasting modelrdquoEnergy Conversion and Management vol 50 no 1 pp 105ndash1172009
[20] G Wang T R Carr Y Ju and C Li ldquoIdentifying organic-richMarcellus Shale lithofacies by support vector machine classifierin the Appalachian basinrdquo Computers and Geosciences vol 64pp 52ndash60 2014
[21] W Wen Z Hao and X Yang ldquoA heuristic weight-setting strat-egy and iteratively updating algorithm for weighted least-squares support vector regressionrdquoNeurocomputing vol 71 no16-18 pp 3096ndash3103 2008
[22] H Xiong Z Chen H Qiu H Hao and H Xu ldquoAdaptiveSVR-HDMR metamodeling technique for high dimensionalproblemsrdquo International Conference onModelling Identificationand Control AASRI Procediapp vol 3 pp 95ndash100 2012
[23] M Lichman Auto MPG Data Set UCI Machine LearningRepository httparchiveicsuciedumldatasetsAuto+MPG2013
[24] M Lichman Computer Hardware Data Set UCI MachineLearningRepository httparchiveicsuciedumldatasetsCom-puter+Hardware 2013
[25] A Tsanas and A Xifara ldquoAccurate quantitative estimation ofenergy performance of residential buildings using statisticalmachine learning toolsrdquo Energy and Buildings vol 49 pp 560ndash567 2012
[26] R Rinnan and A Rinnan ldquoApplication of near infrared reflec-tance (NIR) and fluorescence spectroscopy to analysis ofmicro-biological and chemical properties of arctic soilrdquo Soil Biologyand Biochemistry vol 39 no 7 pp 1664ndash1673 2007
Submit your manuscripts athttpswwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 3
on calculationThen the optimization formula turns into thefollowing form
min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1
(120585119894 + 120585lowast119894)
K (xX119879)120596 minus 119910119894 + 119887 le 120576 + 120585lowast119894
st 119910119894 minus K (xX119879)120596 minus 119887 le 120576 + 120585119894120585119894 120585lowast119894 ge 0 119894 = 1 2 119899
(6)
Although the time and resource spent on modeling arereduced there is no considerable difference in the finalaccuracy
22 Support Vector Regression with Deviation Information asa Consideration Traditional SVR does not possess a specialmethod for addressing noise in sample data An efficient wayto weaken noise is to adjust parameters in the SVR modelThese parameters are called hyperparameters in SVRsHyperparameters exert a considerable impact on algorithmperformance The general way to test the performance ofhyperparameters is via the deviation between the model out-put and the sample data [18] The obtained deviation is thencompared with other deviations to select the minimum devi-ation as the final resultThe parameters that correspond to theminimum deviation are the best parameters in the optimiza-tion process Usually this process is conducted using an intel-ligent optimization algorithm such as particle swarm opti-mization (PSO) [19] and genetic algorithm (GA) [20] Thedeviation is set as the fitness function in an intelligent opti-mization algorithm In this section we refer to thismethod asdeviation-minimized SVR (DM-SVR)
In most of the circumstances the deviation betweenthe model output and sample data is represented by thecorrelation coefficient 119903 or the mean square error (MSE)Given vector y as the model output and vector y as thesample output the correlation coefficient r can be expressedas
119903= sum119873
119894=1119894119910119894 minus sum119873
119894=1119894sum119873119894=1
119910119894119873radic(sum119873119894=1
119894
2 minus (sum119873119894=1
119894)2 119873) (sum119873
119894=11199101198942 minus (sum119873
119894=1119910119894)2 119873)
(7)
The formula for mean square error (MSE) is as follows
MSE = sum119873119894=1
(1003816100381610038161003816119894 minus 1199101198941003816100381610038161003816)2119873 (8)
In short if the value of MSE is close to zero and the value of ris close to one that group of parameters will produce the bestperformance
3 Probability Distribution InformationWeighted Support Vector Regression
Although DM-SVR can reduce influence from the noise italso has some weaknesses The main disadvantage of thismethod is the time it spends on training There are manyparameters that need to be optimized in SVR If thereare extra parameters to optimize these works would makethe train process inefficient To solve the uncertainty oferror parameter 120576 we introduce the probability distributioninformation (PDI) into SVR and designate it as PDISVR
31 Probability Distribution of the Output The probabilitydistribution information is the same as the probability distri-bution function and describes the likelihood that the outputvalue of a continuous random variable is near a certain pointIntegrating the probability density function is the proper wayto calculate the probability value of the random variable inthat certain region From the sample data we could set thefrequency of output to appear as different values Then weset frequency as119873(y) where y is the output value vector Let119875(y) be the probability of the samplersquos output Therefore therelationship between 119875(y) and119873(y) can be expressed as
119875 (y) = 119873 (y)sumyisinY 119873(y) (9)
where Y is the range of y Then we can easily obtainthe probability distribution function The next step is theidentification of the probability of every point
32 Optimization Formula with Probability Distribution Infor-mation Weight Once we have obtained the probability dis-tribution of output it should be integrated into the basicSVR model In the basic SVR model the error parameter 120576indicates the accuracy of model fitting by providing an areathat does not have any loss for the objective function How-ever due to the influence of noise some sample data containexcessive noise information If the same parameters 120576 areadapted the performance of themodel is reduced To preventthis situation SVR should be adjusted in accordance withnoise informationWe propose illustrating noise informationthrough the probability distribution of the output Samples inthe regionswith low probability distributions have a relativelylarge proportion of noise For this reason in modeling theregion with higher probability should have a smaller errorparameter than the lower probability regionThus the proba-bility distribution function increases the accuracy of the SVRmodel in the area with the high probability of output
Define the 120576-insensitive loss function as
1003816100381610038161003816y minus 119891 (x)1003816100381610038161003816120576 = 0 if 1003816100381610038161003816y minus 119891 (x)1003816100381610038161003816 le 1205761003816100381610038161003816y minus 119891 (x)1003816100381610038161003816 minus 120576 otherwise (10)
where119891(x) is a regression estimation function constructed bylearning the sample and y is the target output value that cor-responds to x By defining the 120576-insensitive loss function theSVR model specifies the error requirement of the estimated
4 Mathematical Problems in Engineering
Feature space
0+
minus
Figure 1 Linear problem illustration of a PDISVR model
function on the sample dataThis requirement is the same forevery sample point To avoid this situation the artificially seterror parameter 120576 is divided by the probability distributionvector 119875(y) Figure 1 illustrates the change from a constant120576 to a vector 120576 The distance between two hyperplanes hasbeen modified in some area where the density of the pointsbecomes different Furthermore in the high-density areathe model has a smaller error parameter By contrast inthe low-density area the model has a large error parameterThe density of the output points is directly related to theprobability of the samplersquos output y Therefore the divisionof PDI would make the SVR model emphasize the area witha high density of points This technique can improve overallaccuracy despite sacrificing the accuracy of low-density areasAccording to (9) and (10) the PDISVR can be expressedas
min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1
(120585119894 + 120585lowast119894)
K (xX119879)120596 minus 119910119894 + 119887 le 120576119875 (119910119894) + 120585lowast119894
st 119910119894 minus K (xX119879)120596 minus 119887 le 120576119875 (119910119894) + 120585119894120585119894 120585lowast119894 ge 0 119894 = 1 2 119899
(11)
By comparing (11) with the standard form of SVR we cansee that the error parameter 120576 changes in accordance with 119910119894Then the PDISVRmodel will have low error tolerance for thehigh density of points
To further improve the performance of the SVR modelwe consider adding an extra fragment to the PDISVR frame-work The PDISVR model only has a unique error parameter120576 However 120576 is too small to have an obvious impact onthe accuracy of the model Hence we propose an additionalmethod to introduce PDI that is we apply the same operationas that on error parameter 120576 on the slack variable 120585 Given thetreated error parameter 120576 in the PDISVR model we divided
Table 1 Typical parameters for PSO algorithms
Parameter ValueMaximum generation 200Population size 20Cognitive efficient (1199061) 15maximum velocity (119881max) 1Social efficient (1199062) 17Initial inertia weights 09Final inertia weights 04minimum velocity (119881min) 10
the slack variable 120585 by probability distribution information119875(y) Subsequently the final PDISVR model is
min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1
(120585119894 + 120585lowast119894)
K (xX119879)120596 minus 119910119894 + 119887 le 1119875 (119910119894) (120576 + 120585lowast119894)
st 119910119894 minus K (xX119879)120596 minus 119887 le 1119875 (119910119894) (120576 + 120585119894)120585119894 120585lowast119894 ge 0 119894 = 1 2 119899
(12)
33 Parameters Optimization Based on PSO Normally theperformance of the different SVRs is intensively dependenton the parameters selection A PSO based hyperparametersselecting method in [10] is used in this paper After datasetwas normalized the control parameters of PSO includingmaximum velocity (119881max) minimum velocity (119881min) initialinertia weight (120596min) final inertia weight (120596max) cognitivecoefficient (1206011) social coefficient (1206012) maximum generationand population size should be initialized according to theexperience of operators In our experiment we set thecontrol parameters of PSO based on Table 1 In the followingexperiments there are at most five hyperparameters ofdifferent SVRs that need to be optimized in PSO Theseparameters include penalizing coefficient (C) radial basisfunction kernel parameter (120574) and polynomial degree (d)(120576) in 120576-insensitivity function and mixing coefficient (m) inmultikernel function In our experiments different compar-ative methods adapt different groups of parameters and inorder to search the global optimum reasonably the parameterm is limited in [0 1] 120574 in [10minus2 10] d in [1 10] C in[10 1000] and 120576 in [10minus4 10minus1] During the process ofsearching best parameters by PSO particles update theirpositions by changing velocity and converge finally at a globaloptimum within the searching space In this study the parti-clesrsquo positions are the combination ofm 120574 d C and 120576 whichare denominated as P Then V-fold (V = 5) cross-validationresampling method is applied to validate the performanceof searched best parameters until criterions are met Toevaluate the performance of training process mean squareerror (MSE) is chosen as the fitness function which isformulated as (8) Figure 2 shows the workflow to find theoptimum values of each parameter in models
Mathematical Problems in Engineering 5
Start
Prepare andnormalize dataset
Initialize control parameters of PSO including
generation population size
Set parameter P for SVR
V-fold cross-validationaccuracy
NoOr
Testing SVR based on theoptimized parameters
Yes
Predict result using well-trained SVR model
End
[VGCH VGR] [GCH GR] 1 2 maximum
Search best value m dC by PSO algorithm
Figure 2 Workflow on hyperparametersrsquo selection
4 Experimental Results
To verify the effect of the probability distribution informationon the standard SVR model we employed three kinds ofnumerical experiments with real datasets In these experi-ments we considered three kernels including linear kernelpolynomial kernel andGaussian kernel as SVRsrsquo kernel func-tions All of the experiments were operated onMATLABwithIntel i5 CPU and 6 GB internal storage
Experimental studies have mainly compared differentSVR models including basic SVR MKSVR and heuristicweighted SVR [21] The correlation coefficient r and meansquare error (MSE) are used to evaluate generalization per-formance The formulas of these two criteria are listed inSection 22
41 Example 1 Example 1 tested the previous four methodswith one-high-dimensional functions as shown in [22]
119891 (x) = 119899sum119894=1
[ln (119909119894 minus 2)]2 + [ln (10 minus 119909119894)]2
minus ( 119899prod119894=1
119909119894)02
+ 119890119894 (119896) 21 le 119909119894 le 99 (119899 = 3)
(13)
In the above function the symbol 119890119894(119896) indicates the randomfluctuation variable between minus119896 and k From the range of [2199] for 119909119894 above we generated 100 data at random Through(13) we obtained the output of these 100 data We evenlydivided the 100 data into five parts which comprised fourtraining parts and one testing part After the cross-validationmethod introduced in Section 33 optimal hyperparametersfor different SVR algorithms are selected
After obtaining the optimal hyperparameters we thendetermined the influence of noise on different SVR algo-rithmsThe range of the magnitude of the noise k which wasset as 01 05 1 3 6 and 10 was set in accordance with theoutput range To obtain objective comparisons 10 groups ofnoise were added to each training sample of the algorithmusing the MATLAB toolbox which completely generated10 training datasets Moreover testing data were directlygenerated from the objective function equation (13) Theresults for the criterion of these ten experiments are recordedby their average and standard deviation values as shown inFigures 3ndash5
In these three figures the average criterion values indi-cated that the general performance of the algorithm and thestandard deviation are representative of the algorithmrsquos stabil-ity From Figures 3ndash5 we can see that the performance of theproposed PDISVR is less affected by noisy information thanthose of the other three SVR algorithms In Figure 3 the result
6 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 8 9 10045
05055
06065
07075
08085
09095
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(a)
0 1 2 3 4 5 6 7 8 9 100
5
10
15
20
25
30
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(b)
0 1 2 3 4 5 6 7 8 9 100
001
002
003
004
005
006
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(c)
0 1 2 3 4 5 6 7 8 9 100
20406080
100120140160
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(d)
Figure 3 Four modelsrsquo prediction results in linear kernel with different noise (a) is the average value of correlation coefficient 119903 (b) is theaverage value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
line of PDISVR is more stable than other three methods Andit achieves best prediction performance when adding largernoise in samples among all models Compared to Figure 3the PDISVRrsquos ability to predict is not always the best inFigures 4 and 5 That means the PDISVR with linear kernelis suitable for this dataset Besides in the area with largeintensity of noise the basic SVR andMKSVR poorly handledthe effects of noise Although HW-LSSVR resisted some ofthe effects of noise its performance slightly worsened with ahigh intensity of noisy samplesThe average of the predictionaccuracy and standard deviation of PDISVR were relativelybetter in fitting models with noise of 1 3 6 and 10 Withnoise of 01 and 05 although the differences among theaverage values were small the PDISVR was more stable thanother algorithms in some certain circumstances
42 Example 2 The effects from rough error cannot beignored in real production processes To better simulate realproduction conditions and reveal the robustness of the pro-posed PDISVR when the training samples involved outliersthe rough error term should be added to the function in theprevious model A total of 80 data with a noise intensity of 1were haphazardly generated by (13) as a fundamental trainingsample Test samples containing 20 data were also generated
by (13) Then the dependent variables of the 17th and 48thdata in the fundamental training sample were attached 45times119890and 3 times 119890 respectively as two trivial outliers The dependentvariable of the 50th datum in the fundamental trainingsample was attached 10 times 119890 as one strong outlier Thus thenew training sample that contained one strong outlier thatis the 50th datum and two trivial outliers that is the 17thand 48th data was constructed
To better compare the predictive performance of the dif-ferent SVR algorithms the same four algorithmswere trainedten times in samples with three outliers The average valuesand standard deviation values of 119903 and MSE represented theperformance of these algorithms
As indicated in Tables 2ndash4 the PDISVR algorithm per-formed better in the testing experiments than the other algo-rithms The unweighted SVR and MKSVR were influencedby noise and produced biased estimates in predicting resultswhereas PDISVRdramatically reduced this secondary actionGiven the misjudgments on the outliers in this complicatedsystem the HWSVR algorithm could not obtain satisfactoryresults even when it adapted weighted error parameters
In order to illustrate the quality of PID-SVRrsquos weightelement we compared the weighting results in Table 2 Theweight values of the PDISVR algorithm in training samples
Mathematical Problems in Engineering 7
0 1 2 3 4 5 6 7 8 9 10
04
05
06
07
08
09
1
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(a)
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
6
7
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(b)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
0005001
0015002
0025003
0035004
0045
Noise intensity
Aver
age v
alue
(c)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
051
152
253
354
45
Noise intensity
Stan
dard
dev
iatio
n
(d)
Figure 4 Four modelsrsquo prediction results in radial basis function kernel with different noise (a) is the average value of correlation coefficient119903 (b) is the average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
Table 2 Testing results of SVR algorithms with rough error (linear kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 089278 086844 084402 085828Standard deviation 0000258 0002867 0002707 0001998
MSEAverage value 030614 052311 058221 045612Standard deviation 0003953 0111161 0045897 0028321
Table 3 Testing results of SVR algorithms with rough error (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 078424 078126 075976 064346Standard deviation 0000297 0000169 0000774 0000244
MSEAverage value 070686 073748 062362 084296Standard deviation 000316 000150 000506 000136
are listed in Figure 6 The weight values of the HWSVRalgorithm are listed in Figure 7 As shown in Figure 6 theweights of two trivial outliers were 0 and 010531 for the 17thand 48th data respectively and the weight of one strong out-lier was 000036 which indicated that the PDISVR precisely
detected the outliers As shown in Figure 7 the HWSVR didnot perform as well as PDISVR One strong outlier had aweight of 00751 and two trivial outliers had weights of 03143and 02729 which were unsuitable for modeling given thatsmaller weights such as that of the 23rd datum (00126)
8 Mathematical Problems in Engineering
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 10
04
05
06
07
08
09
1
Noise intensity
Aver
age v
alue
(a)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 1002468
1012141618
Noise intensity
Aver
age v
alue
(b)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
0005
001
0015
002
0025
003
0035
Noise intensity
Stan
dard
dev
iatio
n
(c)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
50
100
150
200
250
Noise intensity
Stan
dard
dev
iatio
n
(d)
Figure 5 Four modelsrsquo prediction results in polynomial kernel with different noise (a) is the average value of correlation coefficient 119903 (b) isthe average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
Table 4 Testing results of SVR algorithms with rough error (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 083278 079399 075874 08137Standard deviation 0000377 0004657 0009121 0003568
MSEAverage value 04473 0727 148208 062433Standard deviation 0003168 0076291 1462867 0028654
could affect outlier detection Thus the influence from twotrivial outliers on the predictability of the PDISVR algorithmwas reduced and the influence from one strong outlier waseliminated By contrast the effect of outliers remained in theHWSVR algorithm
43 Example 3 To test our regressionmodel in amore realis-tic way we imported six more realistic datasets from the UCIMachine Learning Repository [23ndash25] Department of FoodScience University of Copenhagen database [26] and somereal chemical industrial process [14] See Table 5 for moredetailed information In these datasets four out of five datawere used for training and one-fifth of the data was used for
testing The hyperparameters used in this example are alsoobtained by the process introduced in Section 33
As shown in Tables 6ndash8 the proposed PDISVR obtainedthe best predictive ability in the majority of the criterion Forexample in the case of Auto-MPG the proposed PDISVRwas best achieved with both standards Thus the proposedPDISVR is appropriate for the Auto-MPG dataset In thedatasets for crude oil distillation and computer hardwarethe proposed PDISVR only obtained the best correlationcoefficient r and could not establish a suitablemodel at a pointwhere the probability distribution was low thus increasingthe MSE Therefore the use of PDISVR requires validation
Mathematical Problems in Engineering 9
Table 5 Details of the experimental datasets
Data Number of instances Number of attributes Dataset information
Combustion side reaction 144 6The combustion side reaction is the process by which
the solvent and reactants are oxidized to carbondioxide carbon monoxide and water
Auto-MPG dataset 398 8 Data concerning city-cycle fuel consumption
Computer hardware dataset 209 9 Relative CPU performance in terms of cycling timememory size and other parameters
Soil samples measured byNear-Infrared Spectroscopy(NIR)
108 1050 This data is about the soil organic matter (SOM) fromnorthern Sweden (681210N 181490E)
Energy efficiency 768 9This study looked into assessing the heating load andcooling load requirements of buildings (that is energy
efficiency) as a function of building parameters
Crude oil distillation process 30 1 Data related to different components with differentboiling points
Table 6 Comparative results of previous SVR models in real datasets (linear kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 07141 06790 06797 06939MSE 00051 00050 00046 00044
Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802
Computer hardware 119903 09533 08750 08748 09012MSE 79872 83737 83815 65772
NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858
Energy efficiency 119903 1 1 1 09999MSE 00047 11092 00221 00818
Crude oil distillation process r 09372 07684 09372 09927MSE 10162 9724 14510 81628
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 010531
X = 17Y = 0
X = 50
Y = 000036228
Figure 6 Weights of the training sample for PDISVR
through additional research and dataset message Moreoverin the sample of crude oil boiling point there are far lessquantity of data and the PDISVR cannot differentiate the
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 027292
X = 17Y = 031429
X = 50
Y = 0075054
Figure 7 Weights of the training sample for HWSVR
noise according to its probability distribution Thus theproposed PDISVR is applied to improve the SVR in the caseof large datasets
10 Mathematical Problems in Engineering
Table 7 Comparative results of previous SVR models in real datasets (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 06664 0425 06031 05827MSE 00038 00095 06031 00578
Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482
Computer hardware 119903 07815 08597 08647 08851MSE 27961 19986 15355 13160
NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235
Energy efficiency 119903 1 09796 09901 09997MSE 0000269 41875 3019 00474
Crude oil distillation process 119903 09899 09739 09913 09573MSE 477425 787604 11198 3176248
Table 8 Comparative results of previous SVR models in real datasets (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 05796 06853 08078 05518MSE 00081 00106 03546 01237
Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559
Computer hardware 119903 08337 08571 08519 08423MSE 125509 195298 198641 139781
NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073
Energy efficiency 119903 1 09964 09988 09867MSE 00221 04598 01676 41448
Crude oil distillation process 119903 09571 08071 09807 09913MSE 3218016 1129353 543637 11054000
5 Conclusion
In traditional SVR modeling the deviation between themodel outputs and the real data is the only way to representthe influence of noise Other information such as possibilitydistribution information is not emphasized Therefore weproposed a special method that uses the possibility dis-tribution information to modify the basic error parameterand slack variables of SVR Given that these parametersare weighted by the probability distribution of the modeloutput points they can be adjusted by SVR itself and nolonger require optimization by any intelligent optimizationalgorithmThe proposed algorithm is superior to other SVR-based algorithms in dealing with noisy data and outliers insimulation and actual datasets
Conflicts of Interest
The authors declare that they have no conflicts of interest
References
[1] VN VapnikTheNature of Statistical LearningTheory Springer2000
[2] S Yin and J Yin ldquoTuning kernel parameters for SVM based onexpected square distance ratiordquo Information Sciences vol 370-371 pp 92ndash102 2016
[3] J A K Suykens J Vandewalle and B de Moor ldquoOptimalcontrol by least squares support vector machinesrdquo NeuralNetworks vol 14 no 1 pp 23ndash35 2001
[4] J A K Suykens J De Brabanter L Lukas and J VandewalleldquoWeighted least squares support vector machines robustnessand sparce approximationrdquo Neurocomputing vol 48 pp 85ndash105 2002
[5] H-S Tang S-T Xue R Chen and T Sato ldquoOnline weightedLS-SVM for hysteretic structural system identificationrdquo Engi-neering Structures vol 28 no 12 pp 1728ndash1735 2006
[6] A Smola B Schoelkopf and G Raetsch ldquoLinear programsfor automatic accuracy control in regressionrdquo in Proceedings of9th International Conference on Artificial Neural Networks(ICANN99) pp 575ndash580 IEE UK September 1999
[7] A Smola and B Scholkopf Learning with Kernels MIT PressCambridge UK 2002
[8] C Y Yeh CWHuang and S J Lee ldquoAmultiple-kernel supportvector regression approach for stock market price forecastingrdquoExpert Systems with Applications vol 38 no 3 pp 2177ndash21862011
[9] W-J Lin and S-S Jhuo ldquoA fast luminance inspector forbacklight modules based on multiple kernel support vector
Mathematical Problems in Engineering 11
regressionrdquo IEEE Transactions on Components Packaging andManufacturing Technology vol 4 no 8 pp 1391ndash1401 2014
[10] Z Zhong and T R Carr ldquoApplication of mixed kernels function(MKF) based support vector regressionmodel (SVR) for CO2 ndashReservoir oil minimum miscibility pressure predictionrdquo Fuelvol 184 pp 590ndash603 2016
[11] G Bloch F Lauer G Colin and Y Chamaillard ldquoSupportvector regression from simulation data and few experimentalsamplesrdquo Information Sciences vol 178 no 20 pp 3813ndash38272008
[12] J Zhou B Duan J Huang and N Li ldquoIncorporating priorknowledge and multi-kernel into linear programming supportvector regressionrdquo Soft Computing vol 19 no 7 pp 2047ndash20612015
[13] F Lauer and G Bloch ldquoIncorporating prior knowledge insupport vector machines for classification A reviewrdquo Neuro-computing vol 71 no 7-9 pp 1578ndash1594 2008
[14] C Pan Y Dong X Yan and W Zhao ldquoHybrid model for mainand side reactions of p-xylene oxidation with factor influencebased monotone additive SVRrdquo Chemometrics and IntelligentLaboratory Systems vol 136 pp 36ndash46 2014
[15] O Naghash-Almasi and M H Khooban ldquoPI adaptive LS-SVRcontrol scheme with disturbance rejection for a class of uncer-tain nonlinear systemsrdquo Engineering Applications of ArtificialIntelligence vol 52 pp 135ndash144 2016
[16] Y Wang and L Zhang ldquoA combined fault diagnosis method forpower transformer in big data environmentrdquo MathematicalProblems in Engineering vol 2017 Article ID 9670290 6 pages2017
[17] H Dai B Zhang andWWang ldquoAmultiwavelet support vectorregression method for efficient reliability assessmentrdquo Reliabil-ity Engineering and System Safety vol 136 pp 132ndash139 2015
[18] P-Y Hao ldquoPairing support vector algorithm for data regres-sionrdquo Neurocomputing vol 225 pp 174ndash187 2017
[19] W-CHong ldquoChaotic particle swarmoptimization algorithm ina support vector regression electric load forecasting modelrdquoEnergy Conversion and Management vol 50 no 1 pp 105ndash1172009
[20] G Wang T R Carr Y Ju and C Li ldquoIdentifying organic-richMarcellus Shale lithofacies by support vector machine classifierin the Appalachian basinrdquo Computers and Geosciences vol 64pp 52ndash60 2014
[21] W Wen Z Hao and X Yang ldquoA heuristic weight-setting strat-egy and iteratively updating algorithm for weighted least-squares support vector regressionrdquoNeurocomputing vol 71 no16-18 pp 3096ndash3103 2008
[22] H Xiong Z Chen H Qiu H Hao and H Xu ldquoAdaptiveSVR-HDMR metamodeling technique for high dimensionalproblemsrdquo International Conference onModelling Identificationand Control AASRI Procediapp vol 3 pp 95ndash100 2012
[23] M Lichman Auto MPG Data Set UCI Machine LearningRepository httparchiveicsuciedumldatasetsAuto+MPG2013
[24] M Lichman Computer Hardware Data Set UCI MachineLearningRepository httparchiveicsuciedumldatasetsCom-puter+Hardware 2013
[25] A Tsanas and A Xifara ldquoAccurate quantitative estimation ofenergy performance of residential buildings using statisticalmachine learning toolsrdquo Energy and Buildings vol 49 pp 560ndash567 2012
[26] R Rinnan and A Rinnan ldquoApplication of near infrared reflec-tance (NIR) and fluorescence spectroscopy to analysis ofmicro-biological and chemical properties of arctic soilrdquo Soil Biologyand Biochemistry vol 39 no 7 pp 1664ndash1673 2007
Submit your manuscripts athttpswwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
4 Mathematical Problems in Engineering
Feature space
0+
minus
Figure 1 Linear problem illustration of a PDISVR model
function on the sample dataThis requirement is the same forevery sample point To avoid this situation the artificially seterror parameter 120576 is divided by the probability distributionvector 119875(y) Figure 1 illustrates the change from a constant120576 to a vector 120576 The distance between two hyperplanes hasbeen modified in some area where the density of the pointsbecomes different Furthermore in the high-density areathe model has a smaller error parameter By contrast inthe low-density area the model has a large error parameterThe density of the output points is directly related to theprobability of the samplersquos output y Therefore the divisionof PDI would make the SVR model emphasize the area witha high density of points This technique can improve overallaccuracy despite sacrificing the accuracy of low-density areasAccording to (9) and (10) the PDISVR can be expressedas
min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1
(120585119894 + 120585lowast119894)
K (xX119879)120596 minus 119910119894 + 119887 le 120576119875 (119910119894) + 120585lowast119894
st 119910119894 minus K (xX119879)120596 minus 119887 le 120576119875 (119910119894) + 120585119894120585119894 120585lowast119894 ge 0 119894 = 1 2 119899
(11)
By comparing (11) with the standard form of SVR we cansee that the error parameter 120576 changes in accordance with 119910119894Then the PDISVRmodel will have low error tolerance for thehigh density of points
To further improve the performance of the SVR modelwe consider adding an extra fragment to the PDISVR frame-work The PDISVR model only has a unique error parameter120576 However 120576 is too small to have an obvious impact onthe accuracy of the model Hence we propose an additionalmethod to introduce PDI that is we apply the same operationas that on error parameter 120576 on the slack variable 120585 Given thetreated error parameter 120576 in the PDISVR model we divided
Table 1 Typical parameters for PSO algorithms
Parameter ValueMaximum generation 200Population size 20Cognitive efficient (1199061) 15maximum velocity (119881max) 1Social efficient (1199062) 17Initial inertia weights 09Final inertia weights 04minimum velocity (119881min) 10
the slack variable 120585 by probability distribution information119875(y) Subsequently the final PDISVR model is
min 119877 (120596 120585119894 120585lowast119894 ) = 1205961 + 119862 119899sum119894=1
(120585119894 + 120585lowast119894)
K (xX119879)120596 minus 119910119894 + 119887 le 1119875 (119910119894) (120576 + 120585lowast119894)
st 119910119894 minus K (xX119879)120596 minus 119887 le 1119875 (119910119894) (120576 + 120585119894)120585119894 120585lowast119894 ge 0 119894 = 1 2 119899
(12)
33 Parameters Optimization Based on PSO Normally theperformance of the different SVRs is intensively dependenton the parameters selection A PSO based hyperparametersselecting method in [10] is used in this paper After datasetwas normalized the control parameters of PSO includingmaximum velocity (119881max) minimum velocity (119881min) initialinertia weight (120596min) final inertia weight (120596max) cognitivecoefficient (1206011) social coefficient (1206012) maximum generationand population size should be initialized according to theexperience of operators In our experiment we set thecontrol parameters of PSO based on Table 1 In the followingexperiments there are at most five hyperparameters ofdifferent SVRs that need to be optimized in PSO Theseparameters include penalizing coefficient (C) radial basisfunction kernel parameter (120574) and polynomial degree (d)(120576) in 120576-insensitivity function and mixing coefficient (m) inmultikernel function In our experiments different compar-ative methods adapt different groups of parameters and inorder to search the global optimum reasonably the parameterm is limited in [0 1] 120574 in [10minus2 10] d in [1 10] C in[10 1000] and 120576 in [10minus4 10minus1] During the process ofsearching best parameters by PSO particles update theirpositions by changing velocity and converge finally at a globaloptimum within the searching space In this study the parti-clesrsquo positions are the combination ofm 120574 d C and 120576 whichare denominated as P Then V-fold (V = 5) cross-validationresampling method is applied to validate the performanceof searched best parameters until criterions are met Toevaluate the performance of training process mean squareerror (MSE) is chosen as the fitness function which isformulated as (8) Figure 2 shows the workflow to find theoptimum values of each parameter in models
Mathematical Problems in Engineering 5
Start
Prepare andnormalize dataset
Initialize control parameters of PSO including
generation population size
Set parameter P for SVR
V-fold cross-validationaccuracy
NoOr
Testing SVR based on theoptimized parameters
Yes
Predict result using well-trained SVR model
End
[VGCH VGR] [GCH GR] 1 2 maximum
Search best value m dC by PSO algorithm
Figure 2 Workflow on hyperparametersrsquo selection
4 Experimental Results
To verify the effect of the probability distribution informationon the standard SVR model we employed three kinds ofnumerical experiments with real datasets In these experi-ments we considered three kernels including linear kernelpolynomial kernel andGaussian kernel as SVRsrsquo kernel func-tions All of the experiments were operated onMATLABwithIntel i5 CPU and 6 GB internal storage
Experimental studies have mainly compared differentSVR models including basic SVR MKSVR and heuristicweighted SVR [21] The correlation coefficient r and meansquare error (MSE) are used to evaluate generalization per-formance The formulas of these two criteria are listed inSection 22
41 Example 1 Example 1 tested the previous four methodswith one-high-dimensional functions as shown in [22]
119891 (x) = 119899sum119894=1
[ln (119909119894 minus 2)]2 + [ln (10 minus 119909119894)]2
minus ( 119899prod119894=1
119909119894)02
+ 119890119894 (119896) 21 le 119909119894 le 99 (119899 = 3)
(13)
In the above function the symbol 119890119894(119896) indicates the randomfluctuation variable between minus119896 and k From the range of [2199] for 119909119894 above we generated 100 data at random Through(13) we obtained the output of these 100 data We evenlydivided the 100 data into five parts which comprised fourtraining parts and one testing part After the cross-validationmethod introduced in Section 33 optimal hyperparametersfor different SVR algorithms are selected
After obtaining the optimal hyperparameters we thendetermined the influence of noise on different SVR algo-rithmsThe range of the magnitude of the noise k which wasset as 01 05 1 3 6 and 10 was set in accordance with theoutput range To obtain objective comparisons 10 groups ofnoise were added to each training sample of the algorithmusing the MATLAB toolbox which completely generated10 training datasets Moreover testing data were directlygenerated from the objective function equation (13) Theresults for the criterion of these ten experiments are recordedby their average and standard deviation values as shown inFigures 3ndash5
In these three figures the average criterion values indi-cated that the general performance of the algorithm and thestandard deviation are representative of the algorithmrsquos stabil-ity From Figures 3ndash5 we can see that the performance of theproposed PDISVR is less affected by noisy information thanthose of the other three SVR algorithms In Figure 3 the result
6 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 8 9 10045
05055
06065
07075
08085
09095
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(a)
0 1 2 3 4 5 6 7 8 9 100
5
10
15
20
25
30
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(b)
0 1 2 3 4 5 6 7 8 9 100
001
002
003
004
005
006
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(c)
0 1 2 3 4 5 6 7 8 9 100
20406080
100120140160
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(d)
Figure 3 Four modelsrsquo prediction results in linear kernel with different noise (a) is the average value of correlation coefficient 119903 (b) is theaverage value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
line of PDISVR is more stable than other three methods Andit achieves best prediction performance when adding largernoise in samples among all models Compared to Figure 3the PDISVRrsquos ability to predict is not always the best inFigures 4 and 5 That means the PDISVR with linear kernelis suitable for this dataset Besides in the area with largeintensity of noise the basic SVR andMKSVR poorly handledthe effects of noise Although HW-LSSVR resisted some ofthe effects of noise its performance slightly worsened with ahigh intensity of noisy samplesThe average of the predictionaccuracy and standard deviation of PDISVR were relativelybetter in fitting models with noise of 1 3 6 and 10 Withnoise of 01 and 05 although the differences among theaverage values were small the PDISVR was more stable thanother algorithms in some certain circumstances
42 Example 2 The effects from rough error cannot beignored in real production processes To better simulate realproduction conditions and reveal the robustness of the pro-posed PDISVR when the training samples involved outliersthe rough error term should be added to the function in theprevious model A total of 80 data with a noise intensity of 1were haphazardly generated by (13) as a fundamental trainingsample Test samples containing 20 data were also generated
by (13) Then the dependent variables of the 17th and 48thdata in the fundamental training sample were attached 45times119890and 3 times 119890 respectively as two trivial outliers The dependentvariable of the 50th datum in the fundamental trainingsample was attached 10 times 119890 as one strong outlier Thus thenew training sample that contained one strong outlier thatis the 50th datum and two trivial outliers that is the 17thand 48th data was constructed
To better compare the predictive performance of the dif-ferent SVR algorithms the same four algorithmswere trainedten times in samples with three outliers The average valuesand standard deviation values of 119903 and MSE represented theperformance of these algorithms
As indicated in Tables 2ndash4 the PDISVR algorithm per-formed better in the testing experiments than the other algo-rithms The unweighted SVR and MKSVR were influencedby noise and produced biased estimates in predicting resultswhereas PDISVRdramatically reduced this secondary actionGiven the misjudgments on the outliers in this complicatedsystem the HWSVR algorithm could not obtain satisfactoryresults even when it adapted weighted error parameters
In order to illustrate the quality of PID-SVRrsquos weightelement we compared the weighting results in Table 2 Theweight values of the PDISVR algorithm in training samples
Mathematical Problems in Engineering 7
0 1 2 3 4 5 6 7 8 9 10
04
05
06
07
08
09
1
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(a)
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
6
7
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(b)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
0005001
0015002
0025003
0035004
0045
Noise intensity
Aver
age v
alue
(c)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
051
152
253
354
45
Noise intensity
Stan
dard
dev
iatio
n
(d)
Figure 4 Four modelsrsquo prediction results in radial basis function kernel with different noise (a) is the average value of correlation coefficient119903 (b) is the average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
Table 2 Testing results of SVR algorithms with rough error (linear kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 089278 086844 084402 085828Standard deviation 0000258 0002867 0002707 0001998
MSEAverage value 030614 052311 058221 045612Standard deviation 0003953 0111161 0045897 0028321
Table 3 Testing results of SVR algorithms with rough error (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 078424 078126 075976 064346Standard deviation 0000297 0000169 0000774 0000244
MSEAverage value 070686 073748 062362 084296Standard deviation 000316 000150 000506 000136
are listed in Figure 6 The weight values of the HWSVRalgorithm are listed in Figure 7 As shown in Figure 6 theweights of two trivial outliers were 0 and 010531 for the 17thand 48th data respectively and the weight of one strong out-lier was 000036 which indicated that the PDISVR precisely
detected the outliers As shown in Figure 7 the HWSVR didnot perform as well as PDISVR One strong outlier had aweight of 00751 and two trivial outliers had weights of 03143and 02729 which were unsuitable for modeling given thatsmaller weights such as that of the 23rd datum (00126)
8 Mathematical Problems in Engineering
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 10
04
05
06
07
08
09
1
Noise intensity
Aver
age v
alue
(a)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 1002468
1012141618
Noise intensity
Aver
age v
alue
(b)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
0005
001
0015
002
0025
003
0035
Noise intensity
Stan
dard
dev
iatio
n
(c)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
50
100
150
200
250
Noise intensity
Stan
dard
dev
iatio
n
(d)
Figure 5 Four modelsrsquo prediction results in polynomial kernel with different noise (a) is the average value of correlation coefficient 119903 (b) isthe average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
Table 4 Testing results of SVR algorithms with rough error (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 083278 079399 075874 08137Standard deviation 0000377 0004657 0009121 0003568
MSEAverage value 04473 0727 148208 062433Standard deviation 0003168 0076291 1462867 0028654
could affect outlier detection Thus the influence from twotrivial outliers on the predictability of the PDISVR algorithmwas reduced and the influence from one strong outlier waseliminated By contrast the effect of outliers remained in theHWSVR algorithm
43 Example 3 To test our regressionmodel in amore realis-tic way we imported six more realistic datasets from the UCIMachine Learning Repository [23ndash25] Department of FoodScience University of Copenhagen database [26] and somereal chemical industrial process [14] See Table 5 for moredetailed information In these datasets four out of five datawere used for training and one-fifth of the data was used for
testing The hyperparameters used in this example are alsoobtained by the process introduced in Section 33
As shown in Tables 6ndash8 the proposed PDISVR obtainedthe best predictive ability in the majority of the criterion Forexample in the case of Auto-MPG the proposed PDISVRwas best achieved with both standards Thus the proposedPDISVR is appropriate for the Auto-MPG dataset In thedatasets for crude oil distillation and computer hardwarethe proposed PDISVR only obtained the best correlationcoefficient r and could not establish a suitablemodel at a pointwhere the probability distribution was low thus increasingthe MSE Therefore the use of PDISVR requires validation
Mathematical Problems in Engineering 9
Table 5 Details of the experimental datasets
Data Number of instances Number of attributes Dataset information
Combustion side reaction 144 6The combustion side reaction is the process by which
the solvent and reactants are oxidized to carbondioxide carbon monoxide and water
Auto-MPG dataset 398 8 Data concerning city-cycle fuel consumption
Computer hardware dataset 209 9 Relative CPU performance in terms of cycling timememory size and other parameters
Soil samples measured byNear-Infrared Spectroscopy(NIR)
108 1050 This data is about the soil organic matter (SOM) fromnorthern Sweden (681210N 181490E)
Energy efficiency 768 9This study looked into assessing the heating load andcooling load requirements of buildings (that is energy
efficiency) as a function of building parameters
Crude oil distillation process 30 1 Data related to different components with differentboiling points
Table 6 Comparative results of previous SVR models in real datasets (linear kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 07141 06790 06797 06939MSE 00051 00050 00046 00044
Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802
Computer hardware 119903 09533 08750 08748 09012MSE 79872 83737 83815 65772
NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858
Energy efficiency 119903 1 1 1 09999MSE 00047 11092 00221 00818
Crude oil distillation process r 09372 07684 09372 09927MSE 10162 9724 14510 81628
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 010531
X = 17Y = 0
X = 50
Y = 000036228
Figure 6 Weights of the training sample for PDISVR
through additional research and dataset message Moreoverin the sample of crude oil boiling point there are far lessquantity of data and the PDISVR cannot differentiate the
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 027292
X = 17Y = 031429
X = 50
Y = 0075054
Figure 7 Weights of the training sample for HWSVR
noise according to its probability distribution Thus theproposed PDISVR is applied to improve the SVR in the caseof large datasets
10 Mathematical Problems in Engineering
Table 7 Comparative results of previous SVR models in real datasets (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 06664 0425 06031 05827MSE 00038 00095 06031 00578
Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482
Computer hardware 119903 07815 08597 08647 08851MSE 27961 19986 15355 13160
NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235
Energy efficiency 119903 1 09796 09901 09997MSE 0000269 41875 3019 00474
Crude oil distillation process 119903 09899 09739 09913 09573MSE 477425 787604 11198 3176248
Table 8 Comparative results of previous SVR models in real datasets (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 05796 06853 08078 05518MSE 00081 00106 03546 01237
Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559
Computer hardware 119903 08337 08571 08519 08423MSE 125509 195298 198641 139781
NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073
Energy efficiency 119903 1 09964 09988 09867MSE 00221 04598 01676 41448
Crude oil distillation process 119903 09571 08071 09807 09913MSE 3218016 1129353 543637 11054000
5 Conclusion
In traditional SVR modeling the deviation between themodel outputs and the real data is the only way to representthe influence of noise Other information such as possibilitydistribution information is not emphasized Therefore weproposed a special method that uses the possibility dis-tribution information to modify the basic error parameterand slack variables of SVR Given that these parametersare weighted by the probability distribution of the modeloutput points they can be adjusted by SVR itself and nolonger require optimization by any intelligent optimizationalgorithmThe proposed algorithm is superior to other SVR-based algorithms in dealing with noisy data and outliers insimulation and actual datasets
Conflicts of Interest
The authors declare that they have no conflicts of interest
References
[1] VN VapnikTheNature of Statistical LearningTheory Springer2000
[2] S Yin and J Yin ldquoTuning kernel parameters for SVM based onexpected square distance ratiordquo Information Sciences vol 370-371 pp 92ndash102 2016
[3] J A K Suykens J Vandewalle and B de Moor ldquoOptimalcontrol by least squares support vector machinesrdquo NeuralNetworks vol 14 no 1 pp 23ndash35 2001
[4] J A K Suykens J De Brabanter L Lukas and J VandewalleldquoWeighted least squares support vector machines robustnessand sparce approximationrdquo Neurocomputing vol 48 pp 85ndash105 2002
[5] H-S Tang S-T Xue R Chen and T Sato ldquoOnline weightedLS-SVM for hysteretic structural system identificationrdquo Engi-neering Structures vol 28 no 12 pp 1728ndash1735 2006
[6] A Smola B Schoelkopf and G Raetsch ldquoLinear programsfor automatic accuracy control in regressionrdquo in Proceedings of9th International Conference on Artificial Neural Networks(ICANN99) pp 575ndash580 IEE UK September 1999
[7] A Smola and B Scholkopf Learning with Kernels MIT PressCambridge UK 2002
[8] C Y Yeh CWHuang and S J Lee ldquoAmultiple-kernel supportvector regression approach for stock market price forecastingrdquoExpert Systems with Applications vol 38 no 3 pp 2177ndash21862011
[9] W-J Lin and S-S Jhuo ldquoA fast luminance inspector forbacklight modules based on multiple kernel support vector
Mathematical Problems in Engineering 11
regressionrdquo IEEE Transactions on Components Packaging andManufacturing Technology vol 4 no 8 pp 1391ndash1401 2014
[10] Z Zhong and T R Carr ldquoApplication of mixed kernels function(MKF) based support vector regressionmodel (SVR) for CO2 ndashReservoir oil minimum miscibility pressure predictionrdquo Fuelvol 184 pp 590ndash603 2016
[11] G Bloch F Lauer G Colin and Y Chamaillard ldquoSupportvector regression from simulation data and few experimentalsamplesrdquo Information Sciences vol 178 no 20 pp 3813ndash38272008
[12] J Zhou B Duan J Huang and N Li ldquoIncorporating priorknowledge and multi-kernel into linear programming supportvector regressionrdquo Soft Computing vol 19 no 7 pp 2047ndash20612015
[13] F Lauer and G Bloch ldquoIncorporating prior knowledge insupport vector machines for classification A reviewrdquo Neuro-computing vol 71 no 7-9 pp 1578ndash1594 2008
[14] C Pan Y Dong X Yan and W Zhao ldquoHybrid model for mainand side reactions of p-xylene oxidation with factor influencebased monotone additive SVRrdquo Chemometrics and IntelligentLaboratory Systems vol 136 pp 36ndash46 2014
[15] O Naghash-Almasi and M H Khooban ldquoPI adaptive LS-SVRcontrol scheme with disturbance rejection for a class of uncer-tain nonlinear systemsrdquo Engineering Applications of ArtificialIntelligence vol 52 pp 135ndash144 2016
[16] Y Wang and L Zhang ldquoA combined fault diagnosis method forpower transformer in big data environmentrdquo MathematicalProblems in Engineering vol 2017 Article ID 9670290 6 pages2017
[17] H Dai B Zhang andWWang ldquoAmultiwavelet support vectorregression method for efficient reliability assessmentrdquo Reliabil-ity Engineering and System Safety vol 136 pp 132ndash139 2015
[18] P-Y Hao ldquoPairing support vector algorithm for data regres-sionrdquo Neurocomputing vol 225 pp 174ndash187 2017
[19] W-CHong ldquoChaotic particle swarmoptimization algorithm ina support vector regression electric load forecasting modelrdquoEnergy Conversion and Management vol 50 no 1 pp 105ndash1172009
[20] G Wang T R Carr Y Ju and C Li ldquoIdentifying organic-richMarcellus Shale lithofacies by support vector machine classifierin the Appalachian basinrdquo Computers and Geosciences vol 64pp 52ndash60 2014
[21] W Wen Z Hao and X Yang ldquoA heuristic weight-setting strat-egy and iteratively updating algorithm for weighted least-squares support vector regressionrdquoNeurocomputing vol 71 no16-18 pp 3096ndash3103 2008
[22] H Xiong Z Chen H Qiu H Hao and H Xu ldquoAdaptiveSVR-HDMR metamodeling technique for high dimensionalproblemsrdquo International Conference onModelling Identificationand Control AASRI Procediapp vol 3 pp 95ndash100 2012
[23] M Lichman Auto MPG Data Set UCI Machine LearningRepository httparchiveicsuciedumldatasetsAuto+MPG2013
[24] M Lichman Computer Hardware Data Set UCI MachineLearningRepository httparchiveicsuciedumldatasetsCom-puter+Hardware 2013
[25] A Tsanas and A Xifara ldquoAccurate quantitative estimation ofenergy performance of residential buildings using statisticalmachine learning toolsrdquo Energy and Buildings vol 49 pp 560ndash567 2012
[26] R Rinnan and A Rinnan ldquoApplication of near infrared reflec-tance (NIR) and fluorescence spectroscopy to analysis ofmicro-biological and chemical properties of arctic soilrdquo Soil Biologyand Biochemistry vol 39 no 7 pp 1664ndash1673 2007
Submit your manuscripts athttpswwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 5
Start
Prepare andnormalize dataset
Initialize control parameters of PSO including
generation population size
Set parameter P for SVR
V-fold cross-validationaccuracy
NoOr
Testing SVR based on theoptimized parameters
Yes
Predict result using well-trained SVR model
End
[VGCH VGR] [GCH GR] 1 2 maximum
Search best value m dC by PSO algorithm
Figure 2 Workflow on hyperparametersrsquo selection
4 Experimental Results
To verify the effect of the probability distribution informationon the standard SVR model we employed three kinds ofnumerical experiments with real datasets In these experi-ments we considered three kernels including linear kernelpolynomial kernel andGaussian kernel as SVRsrsquo kernel func-tions All of the experiments were operated onMATLABwithIntel i5 CPU and 6 GB internal storage
Experimental studies have mainly compared differentSVR models including basic SVR MKSVR and heuristicweighted SVR [21] The correlation coefficient r and meansquare error (MSE) are used to evaluate generalization per-formance The formulas of these two criteria are listed inSection 22
41 Example 1 Example 1 tested the previous four methodswith one-high-dimensional functions as shown in [22]
119891 (x) = 119899sum119894=1
[ln (119909119894 minus 2)]2 + [ln (10 minus 119909119894)]2
minus ( 119899prod119894=1
119909119894)02
+ 119890119894 (119896) 21 le 119909119894 le 99 (119899 = 3)
(13)
In the above function the symbol 119890119894(119896) indicates the randomfluctuation variable between minus119896 and k From the range of [2199] for 119909119894 above we generated 100 data at random Through(13) we obtained the output of these 100 data We evenlydivided the 100 data into five parts which comprised fourtraining parts and one testing part After the cross-validationmethod introduced in Section 33 optimal hyperparametersfor different SVR algorithms are selected
After obtaining the optimal hyperparameters we thendetermined the influence of noise on different SVR algo-rithmsThe range of the magnitude of the noise k which wasset as 01 05 1 3 6 and 10 was set in accordance with theoutput range To obtain objective comparisons 10 groups ofnoise were added to each training sample of the algorithmusing the MATLAB toolbox which completely generated10 training datasets Moreover testing data were directlygenerated from the objective function equation (13) Theresults for the criterion of these ten experiments are recordedby their average and standard deviation values as shown inFigures 3ndash5
In these three figures the average criterion values indi-cated that the general performance of the algorithm and thestandard deviation are representative of the algorithmrsquos stabil-ity From Figures 3ndash5 we can see that the performance of theproposed PDISVR is less affected by noisy information thanthose of the other three SVR algorithms In Figure 3 the result
6 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 8 9 10045
05055
06065
07075
08085
09095
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(a)
0 1 2 3 4 5 6 7 8 9 100
5
10
15
20
25
30
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(b)
0 1 2 3 4 5 6 7 8 9 100
001
002
003
004
005
006
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(c)
0 1 2 3 4 5 6 7 8 9 100
20406080
100120140160
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(d)
Figure 3 Four modelsrsquo prediction results in linear kernel with different noise (a) is the average value of correlation coefficient 119903 (b) is theaverage value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
line of PDISVR is more stable than other three methods Andit achieves best prediction performance when adding largernoise in samples among all models Compared to Figure 3the PDISVRrsquos ability to predict is not always the best inFigures 4 and 5 That means the PDISVR with linear kernelis suitable for this dataset Besides in the area with largeintensity of noise the basic SVR andMKSVR poorly handledthe effects of noise Although HW-LSSVR resisted some ofthe effects of noise its performance slightly worsened with ahigh intensity of noisy samplesThe average of the predictionaccuracy and standard deviation of PDISVR were relativelybetter in fitting models with noise of 1 3 6 and 10 Withnoise of 01 and 05 although the differences among theaverage values were small the PDISVR was more stable thanother algorithms in some certain circumstances
42 Example 2 The effects from rough error cannot beignored in real production processes To better simulate realproduction conditions and reveal the robustness of the pro-posed PDISVR when the training samples involved outliersthe rough error term should be added to the function in theprevious model A total of 80 data with a noise intensity of 1were haphazardly generated by (13) as a fundamental trainingsample Test samples containing 20 data were also generated
by (13) Then the dependent variables of the 17th and 48thdata in the fundamental training sample were attached 45times119890and 3 times 119890 respectively as two trivial outliers The dependentvariable of the 50th datum in the fundamental trainingsample was attached 10 times 119890 as one strong outlier Thus thenew training sample that contained one strong outlier thatis the 50th datum and two trivial outliers that is the 17thand 48th data was constructed
To better compare the predictive performance of the dif-ferent SVR algorithms the same four algorithmswere trainedten times in samples with three outliers The average valuesand standard deviation values of 119903 and MSE represented theperformance of these algorithms
As indicated in Tables 2ndash4 the PDISVR algorithm per-formed better in the testing experiments than the other algo-rithms The unweighted SVR and MKSVR were influencedby noise and produced biased estimates in predicting resultswhereas PDISVRdramatically reduced this secondary actionGiven the misjudgments on the outliers in this complicatedsystem the HWSVR algorithm could not obtain satisfactoryresults even when it adapted weighted error parameters
In order to illustrate the quality of PID-SVRrsquos weightelement we compared the weighting results in Table 2 Theweight values of the PDISVR algorithm in training samples
Mathematical Problems in Engineering 7
0 1 2 3 4 5 6 7 8 9 10
04
05
06
07
08
09
1
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(a)
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
6
7
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(b)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
0005001
0015002
0025003
0035004
0045
Noise intensity
Aver
age v
alue
(c)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
051
152
253
354
45
Noise intensity
Stan
dard
dev
iatio
n
(d)
Figure 4 Four modelsrsquo prediction results in radial basis function kernel with different noise (a) is the average value of correlation coefficient119903 (b) is the average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
Table 2 Testing results of SVR algorithms with rough error (linear kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 089278 086844 084402 085828Standard deviation 0000258 0002867 0002707 0001998
MSEAverage value 030614 052311 058221 045612Standard deviation 0003953 0111161 0045897 0028321
Table 3 Testing results of SVR algorithms with rough error (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 078424 078126 075976 064346Standard deviation 0000297 0000169 0000774 0000244
MSEAverage value 070686 073748 062362 084296Standard deviation 000316 000150 000506 000136
are listed in Figure 6 The weight values of the HWSVRalgorithm are listed in Figure 7 As shown in Figure 6 theweights of two trivial outliers were 0 and 010531 for the 17thand 48th data respectively and the weight of one strong out-lier was 000036 which indicated that the PDISVR precisely
detected the outliers As shown in Figure 7 the HWSVR didnot perform as well as PDISVR One strong outlier had aweight of 00751 and two trivial outliers had weights of 03143and 02729 which were unsuitable for modeling given thatsmaller weights such as that of the 23rd datum (00126)
8 Mathematical Problems in Engineering
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 10
04
05
06
07
08
09
1
Noise intensity
Aver
age v
alue
(a)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 1002468
1012141618
Noise intensity
Aver
age v
alue
(b)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
0005
001
0015
002
0025
003
0035
Noise intensity
Stan
dard
dev
iatio
n
(c)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
50
100
150
200
250
Noise intensity
Stan
dard
dev
iatio
n
(d)
Figure 5 Four modelsrsquo prediction results in polynomial kernel with different noise (a) is the average value of correlation coefficient 119903 (b) isthe average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
Table 4 Testing results of SVR algorithms with rough error (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 083278 079399 075874 08137Standard deviation 0000377 0004657 0009121 0003568
MSEAverage value 04473 0727 148208 062433Standard deviation 0003168 0076291 1462867 0028654
could affect outlier detection Thus the influence from twotrivial outliers on the predictability of the PDISVR algorithmwas reduced and the influence from one strong outlier waseliminated By contrast the effect of outliers remained in theHWSVR algorithm
43 Example 3 To test our regressionmodel in amore realis-tic way we imported six more realistic datasets from the UCIMachine Learning Repository [23ndash25] Department of FoodScience University of Copenhagen database [26] and somereal chemical industrial process [14] See Table 5 for moredetailed information In these datasets four out of five datawere used for training and one-fifth of the data was used for
testing The hyperparameters used in this example are alsoobtained by the process introduced in Section 33
As shown in Tables 6ndash8 the proposed PDISVR obtainedthe best predictive ability in the majority of the criterion Forexample in the case of Auto-MPG the proposed PDISVRwas best achieved with both standards Thus the proposedPDISVR is appropriate for the Auto-MPG dataset In thedatasets for crude oil distillation and computer hardwarethe proposed PDISVR only obtained the best correlationcoefficient r and could not establish a suitablemodel at a pointwhere the probability distribution was low thus increasingthe MSE Therefore the use of PDISVR requires validation
Mathematical Problems in Engineering 9
Table 5 Details of the experimental datasets
Data Number of instances Number of attributes Dataset information
Combustion side reaction 144 6The combustion side reaction is the process by which
the solvent and reactants are oxidized to carbondioxide carbon monoxide and water
Auto-MPG dataset 398 8 Data concerning city-cycle fuel consumption
Computer hardware dataset 209 9 Relative CPU performance in terms of cycling timememory size and other parameters
Soil samples measured byNear-Infrared Spectroscopy(NIR)
108 1050 This data is about the soil organic matter (SOM) fromnorthern Sweden (681210N 181490E)
Energy efficiency 768 9This study looked into assessing the heating load andcooling load requirements of buildings (that is energy
efficiency) as a function of building parameters
Crude oil distillation process 30 1 Data related to different components with differentboiling points
Table 6 Comparative results of previous SVR models in real datasets (linear kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 07141 06790 06797 06939MSE 00051 00050 00046 00044
Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802
Computer hardware 119903 09533 08750 08748 09012MSE 79872 83737 83815 65772
NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858
Energy efficiency 119903 1 1 1 09999MSE 00047 11092 00221 00818
Crude oil distillation process r 09372 07684 09372 09927MSE 10162 9724 14510 81628
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 010531
X = 17Y = 0
X = 50
Y = 000036228
Figure 6 Weights of the training sample for PDISVR
through additional research and dataset message Moreoverin the sample of crude oil boiling point there are far lessquantity of data and the PDISVR cannot differentiate the
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 027292
X = 17Y = 031429
X = 50
Y = 0075054
Figure 7 Weights of the training sample for HWSVR
noise according to its probability distribution Thus theproposed PDISVR is applied to improve the SVR in the caseof large datasets
10 Mathematical Problems in Engineering
Table 7 Comparative results of previous SVR models in real datasets (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 06664 0425 06031 05827MSE 00038 00095 06031 00578
Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482
Computer hardware 119903 07815 08597 08647 08851MSE 27961 19986 15355 13160
NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235
Energy efficiency 119903 1 09796 09901 09997MSE 0000269 41875 3019 00474
Crude oil distillation process 119903 09899 09739 09913 09573MSE 477425 787604 11198 3176248
Table 8 Comparative results of previous SVR models in real datasets (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 05796 06853 08078 05518MSE 00081 00106 03546 01237
Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559
Computer hardware 119903 08337 08571 08519 08423MSE 125509 195298 198641 139781
NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073
Energy efficiency 119903 1 09964 09988 09867MSE 00221 04598 01676 41448
Crude oil distillation process 119903 09571 08071 09807 09913MSE 3218016 1129353 543637 11054000
5 Conclusion
In traditional SVR modeling the deviation between themodel outputs and the real data is the only way to representthe influence of noise Other information such as possibilitydistribution information is not emphasized Therefore weproposed a special method that uses the possibility dis-tribution information to modify the basic error parameterand slack variables of SVR Given that these parametersare weighted by the probability distribution of the modeloutput points they can be adjusted by SVR itself and nolonger require optimization by any intelligent optimizationalgorithmThe proposed algorithm is superior to other SVR-based algorithms in dealing with noisy data and outliers insimulation and actual datasets
Conflicts of Interest
The authors declare that they have no conflicts of interest
References
[1] VN VapnikTheNature of Statistical LearningTheory Springer2000
[2] S Yin and J Yin ldquoTuning kernel parameters for SVM based onexpected square distance ratiordquo Information Sciences vol 370-371 pp 92ndash102 2016
[3] J A K Suykens J Vandewalle and B de Moor ldquoOptimalcontrol by least squares support vector machinesrdquo NeuralNetworks vol 14 no 1 pp 23ndash35 2001
[4] J A K Suykens J De Brabanter L Lukas and J VandewalleldquoWeighted least squares support vector machines robustnessand sparce approximationrdquo Neurocomputing vol 48 pp 85ndash105 2002
[5] H-S Tang S-T Xue R Chen and T Sato ldquoOnline weightedLS-SVM for hysteretic structural system identificationrdquo Engi-neering Structures vol 28 no 12 pp 1728ndash1735 2006
[6] A Smola B Schoelkopf and G Raetsch ldquoLinear programsfor automatic accuracy control in regressionrdquo in Proceedings of9th International Conference on Artificial Neural Networks(ICANN99) pp 575ndash580 IEE UK September 1999
[7] A Smola and B Scholkopf Learning with Kernels MIT PressCambridge UK 2002
[8] C Y Yeh CWHuang and S J Lee ldquoAmultiple-kernel supportvector regression approach for stock market price forecastingrdquoExpert Systems with Applications vol 38 no 3 pp 2177ndash21862011
[9] W-J Lin and S-S Jhuo ldquoA fast luminance inspector forbacklight modules based on multiple kernel support vector
Mathematical Problems in Engineering 11
regressionrdquo IEEE Transactions on Components Packaging andManufacturing Technology vol 4 no 8 pp 1391ndash1401 2014
[10] Z Zhong and T R Carr ldquoApplication of mixed kernels function(MKF) based support vector regressionmodel (SVR) for CO2 ndashReservoir oil minimum miscibility pressure predictionrdquo Fuelvol 184 pp 590ndash603 2016
[11] G Bloch F Lauer G Colin and Y Chamaillard ldquoSupportvector regression from simulation data and few experimentalsamplesrdquo Information Sciences vol 178 no 20 pp 3813ndash38272008
[12] J Zhou B Duan J Huang and N Li ldquoIncorporating priorknowledge and multi-kernel into linear programming supportvector regressionrdquo Soft Computing vol 19 no 7 pp 2047ndash20612015
[13] F Lauer and G Bloch ldquoIncorporating prior knowledge insupport vector machines for classification A reviewrdquo Neuro-computing vol 71 no 7-9 pp 1578ndash1594 2008
[14] C Pan Y Dong X Yan and W Zhao ldquoHybrid model for mainand side reactions of p-xylene oxidation with factor influencebased monotone additive SVRrdquo Chemometrics and IntelligentLaboratory Systems vol 136 pp 36ndash46 2014
[15] O Naghash-Almasi and M H Khooban ldquoPI adaptive LS-SVRcontrol scheme with disturbance rejection for a class of uncer-tain nonlinear systemsrdquo Engineering Applications of ArtificialIntelligence vol 52 pp 135ndash144 2016
[16] Y Wang and L Zhang ldquoA combined fault diagnosis method forpower transformer in big data environmentrdquo MathematicalProblems in Engineering vol 2017 Article ID 9670290 6 pages2017
[17] H Dai B Zhang andWWang ldquoAmultiwavelet support vectorregression method for efficient reliability assessmentrdquo Reliabil-ity Engineering and System Safety vol 136 pp 132ndash139 2015
[18] P-Y Hao ldquoPairing support vector algorithm for data regres-sionrdquo Neurocomputing vol 225 pp 174ndash187 2017
[19] W-CHong ldquoChaotic particle swarmoptimization algorithm ina support vector regression electric load forecasting modelrdquoEnergy Conversion and Management vol 50 no 1 pp 105ndash1172009
[20] G Wang T R Carr Y Ju and C Li ldquoIdentifying organic-richMarcellus Shale lithofacies by support vector machine classifierin the Appalachian basinrdquo Computers and Geosciences vol 64pp 52ndash60 2014
[21] W Wen Z Hao and X Yang ldquoA heuristic weight-setting strat-egy and iteratively updating algorithm for weighted least-squares support vector regressionrdquoNeurocomputing vol 71 no16-18 pp 3096ndash3103 2008
[22] H Xiong Z Chen H Qiu H Hao and H Xu ldquoAdaptiveSVR-HDMR metamodeling technique for high dimensionalproblemsrdquo International Conference onModelling Identificationand Control AASRI Procediapp vol 3 pp 95ndash100 2012
[23] M Lichman Auto MPG Data Set UCI Machine LearningRepository httparchiveicsuciedumldatasetsAuto+MPG2013
[24] M Lichman Computer Hardware Data Set UCI MachineLearningRepository httparchiveicsuciedumldatasetsCom-puter+Hardware 2013
[25] A Tsanas and A Xifara ldquoAccurate quantitative estimation ofenergy performance of residential buildings using statisticalmachine learning toolsrdquo Energy and Buildings vol 49 pp 560ndash567 2012
[26] R Rinnan and A Rinnan ldquoApplication of near infrared reflec-tance (NIR) and fluorescence spectroscopy to analysis ofmicro-biological and chemical properties of arctic soilrdquo Soil Biologyand Biochemistry vol 39 no 7 pp 1664ndash1673 2007
Submit your manuscripts athttpswwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
6 Mathematical Problems in Engineering
0 1 2 3 4 5 6 7 8 9 10045
05055
06065
07075
08085
09095
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(a)
0 1 2 3 4 5 6 7 8 9 100
5
10
15
20
25
30
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(b)
0 1 2 3 4 5 6 7 8 9 100
001
002
003
004
005
006
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(c)
0 1 2 3 4 5 6 7 8 9 100
20406080
100120140160
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(d)
Figure 3 Four modelsrsquo prediction results in linear kernel with different noise (a) is the average value of correlation coefficient 119903 (b) is theaverage value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
line of PDISVR is more stable than other three methods Andit achieves best prediction performance when adding largernoise in samples among all models Compared to Figure 3the PDISVRrsquos ability to predict is not always the best inFigures 4 and 5 That means the PDISVR with linear kernelis suitable for this dataset Besides in the area with largeintensity of noise the basic SVR andMKSVR poorly handledthe effects of noise Although HW-LSSVR resisted some ofthe effects of noise its performance slightly worsened with ahigh intensity of noisy samplesThe average of the predictionaccuracy and standard deviation of PDISVR were relativelybetter in fitting models with noise of 1 3 6 and 10 Withnoise of 01 and 05 although the differences among theaverage values were small the PDISVR was more stable thanother algorithms in some certain circumstances
42 Example 2 The effects from rough error cannot beignored in real production processes To better simulate realproduction conditions and reveal the robustness of the pro-posed PDISVR when the training samples involved outliersthe rough error term should be added to the function in theprevious model A total of 80 data with a noise intensity of 1were haphazardly generated by (13) as a fundamental trainingsample Test samples containing 20 data were also generated
by (13) Then the dependent variables of the 17th and 48thdata in the fundamental training sample were attached 45times119890and 3 times 119890 respectively as two trivial outliers The dependentvariable of the 50th datum in the fundamental trainingsample was attached 10 times 119890 as one strong outlier Thus thenew training sample that contained one strong outlier thatis the 50th datum and two trivial outliers that is the 17thand 48th data was constructed
To better compare the predictive performance of the dif-ferent SVR algorithms the same four algorithmswere trainedten times in samples with three outliers The average valuesand standard deviation values of 119903 and MSE represented theperformance of these algorithms
As indicated in Tables 2ndash4 the PDISVR algorithm per-formed better in the testing experiments than the other algo-rithms The unweighted SVR and MKSVR were influencedby noise and produced biased estimates in predicting resultswhereas PDISVRdramatically reduced this secondary actionGiven the misjudgments on the outliers in this complicatedsystem the HWSVR algorithm could not obtain satisfactoryresults even when it adapted weighted error parameters
In order to illustrate the quality of PID-SVRrsquos weightelement we compared the weighting results in Table 2 Theweight values of the PDISVR algorithm in training samples
Mathematical Problems in Engineering 7
0 1 2 3 4 5 6 7 8 9 10
04
05
06
07
08
09
1
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(a)
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
6
7
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(b)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
0005001
0015002
0025003
0035004
0045
Noise intensity
Aver
age v
alue
(c)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
051
152
253
354
45
Noise intensity
Stan
dard
dev
iatio
n
(d)
Figure 4 Four modelsrsquo prediction results in radial basis function kernel with different noise (a) is the average value of correlation coefficient119903 (b) is the average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
Table 2 Testing results of SVR algorithms with rough error (linear kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 089278 086844 084402 085828Standard deviation 0000258 0002867 0002707 0001998
MSEAverage value 030614 052311 058221 045612Standard deviation 0003953 0111161 0045897 0028321
Table 3 Testing results of SVR algorithms with rough error (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 078424 078126 075976 064346Standard deviation 0000297 0000169 0000774 0000244
MSEAverage value 070686 073748 062362 084296Standard deviation 000316 000150 000506 000136
are listed in Figure 6 The weight values of the HWSVRalgorithm are listed in Figure 7 As shown in Figure 6 theweights of two trivial outliers were 0 and 010531 for the 17thand 48th data respectively and the weight of one strong out-lier was 000036 which indicated that the PDISVR precisely
detected the outliers As shown in Figure 7 the HWSVR didnot perform as well as PDISVR One strong outlier had aweight of 00751 and two trivial outliers had weights of 03143and 02729 which were unsuitable for modeling given thatsmaller weights such as that of the 23rd datum (00126)
8 Mathematical Problems in Engineering
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 10
04
05
06
07
08
09
1
Noise intensity
Aver
age v
alue
(a)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 1002468
1012141618
Noise intensity
Aver
age v
alue
(b)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
0005
001
0015
002
0025
003
0035
Noise intensity
Stan
dard
dev
iatio
n
(c)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
50
100
150
200
250
Noise intensity
Stan
dard
dev
iatio
n
(d)
Figure 5 Four modelsrsquo prediction results in polynomial kernel with different noise (a) is the average value of correlation coefficient 119903 (b) isthe average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
Table 4 Testing results of SVR algorithms with rough error (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 083278 079399 075874 08137Standard deviation 0000377 0004657 0009121 0003568
MSEAverage value 04473 0727 148208 062433Standard deviation 0003168 0076291 1462867 0028654
could affect outlier detection Thus the influence from twotrivial outliers on the predictability of the PDISVR algorithmwas reduced and the influence from one strong outlier waseliminated By contrast the effect of outliers remained in theHWSVR algorithm
43 Example 3 To test our regressionmodel in amore realis-tic way we imported six more realistic datasets from the UCIMachine Learning Repository [23ndash25] Department of FoodScience University of Copenhagen database [26] and somereal chemical industrial process [14] See Table 5 for moredetailed information In these datasets four out of five datawere used for training and one-fifth of the data was used for
testing The hyperparameters used in this example are alsoobtained by the process introduced in Section 33
As shown in Tables 6ndash8 the proposed PDISVR obtainedthe best predictive ability in the majority of the criterion Forexample in the case of Auto-MPG the proposed PDISVRwas best achieved with both standards Thus the proposedPDISVR is appropriate for the Auto-MPG dataset In thedatasets for crude oil distillation and computer hardwarethe proposed PDISVR only obtained the best correlationcoefficient r and could not establish a suitablemodel at a pointwhere the probability distribution was low thus increasingthe MSE Therefore the use of PDISVR requires validation
Mathematical Problems in Engineering 9
Table 5 Details of the experimental datasets
Data Number of instances Number of attributes Dataset information
Combustion side reaction 144 6The combustion side reaction is the process by which
the solvent and reactants are oxidized to carbondioxide carbon monoxide and water
Auto-MPG dataset 398 8 Data concerning city-cycle fuel consumption
Computer hardware dataset 209 9 Relative CPU performance in terms of cycling timememory size and other parameters
Soil samples measured byNear-Infrared Spectroscopy(NIR)
108 1050 This data is about the soil organic matter (SOM) fromnorthern Sweden (681210N 181490E)
Energy efficiency 768 9This study looked into assessing the heating load andcooling load requirements of buildings (that is energy
efficiency) as a function of building parameters
Crude oil distillation process 30 1 Data related to different components with differentboiling points
Table 6 Comparative results of previous SVR models in real datasets (linear kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 07141 06790 06797 06939MSE 00051 00050 00046 00044
Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802
Computer hardware 119903 09533 08750 08748 09012MSE 79872 83737 83815 65772
NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858
Energy efficiency 119903 1 1 1 09999MSE 00047 11092 00221 00818
Crude oil distillation process r 09372 07684 09372 09927MSE 10162 9724 14510 81628
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 010531
X = 17Y = 0
X = 50
Y = 000036228
Figure 6 Weights of the training sample for PDISVR
through additional research and dataset message Moreoverin the sample of crude oil boiling point there are far lessquantity of data and the PDISVR cannot differentiate the
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 027292
X = 17Y = 031429
X = 50
Y = 0075054
Figure 7 Weights of the training sample for HWSVR
noise according to its probability distribution Thus theproposed PDISVR is applied to improve the SVR in the caseof large datasets
10 Mathematical Problems in Engineering
Table 7 Comparative results of previous SVR models in real datasets (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 06664 0425 06031 05827MSE 00038 00095 06031 00578
Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482
Computer hardware 119903 07815 08597 08647 08851MSE 27961 19986 15355 13160
NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235
Energy efficiency 119903 1 09796 09901 09997MSE 0000269 41875 3019 00474
Crude oil distillation process 119903 09899 09739 09913 09573MSE 477425 787604 11198 3176248
Table 8 Comparative results of previous SVR models in real datasets (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 05796 06853 08078 05518MSE 00081 00106 03546 01237
Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559
Computer hardware 119903 08337 08571 08519 08423MSE 125509 195298 198641 139781
NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073
Energy efficiency 119903 1 09964 09988 09867MSE 00221 04598 01676 41448
Crude oil distillation process 119903 09571 08071 09807 09913MSE 3218016 1129353 543637 11054000
5 Conclusion
In traditional SVR modeling the deviation between themodel outputs and the real data is the only way to representthe influence of noise Other information such as possibilitydistribution information is not emphasized Therefore weproposed a special method that uses the possibility dis-tribution information to modify the basic error parameterand slack variables of SVR Given that these parametersare weighted by the probability distribution of the modeloutput points they can be adjusted by SVR itself and nolonger require optimization by any intelligent optimizationalgorithmThe proposed algorithm is superior to other SVR-based algorithms in dealing with noisy data and outliers insimulation and actual datasets
Conflicts of Interest
The authors declare that they have no conflicts of interest
References
[1] VN VapnikTheNature of Statistical LearningTheory Springer2000
[2] S Yin and J Yin ldquoTuning kernel parameters for SVM based onexpected square distance ratiordquo Information Sciences vol 370-371 pp 92ndash102 2016
[3] J A K Suykens J Vandewalle and B de Moor ldquoOptimalcontrol by least squares support vector machinesrdquo NeuralNetworks vol 14 no 1 pp 23ndash35 2001
[4] J A K Suykens J De Brabanter L Lukas and J VandewalleldquoWeighted least squares support vector machines robustnessand sparce approximationrdquo Neurocomputing vol 48 pp 85ndash105 2002
[5] H-S Tang S-T Xue R Chen and T Sato ldquoOnline weightedLS-SVM for hysteretic structural system identificationrdquo Engi-neering Structures vol 28 no 12 pp 1728ndash1735 2006
[6] A Smola B Schoelkopf and G Raetsch ldquoLinear programsfor automatic accuracy control in regressionrdquo in Proceedings of9th International Conference on Artificial Neural Networks(ICANN99) pp 575ndash580 IEE UK September 1999
[7] A Smola and B Scholkopf Learning with Kernels MIT PressCambridge UK 2002
[8] C Y Yeh CWHuang and S J Lee ldquoAmultiple-kernel supportvector regression approach for stock market price forecastingrdquoExpert Systems with Applications vol 38 no 3 pp 2177ndash21862011
[9] W-J Lin and S-S Jhuo ldquoA fast luminance inspector forbacklight modules based on multiple kernel support vector
Mathematical Problems in Engineering 11
regressionrdquo IEEE Transactions on Components Packaging andManufacturing Technology vol 4 no 8 pp 1391ndash1401 2014
[10] Z Zhong and T R Carr ldquoApplication of mixed kernels function(MKF) based support vector regressionmodel (SVR) for CO2 ndashReservoir oil minimum miscibility pressure predictionrdquo Fuelvol 184 pp 590ndash603 2016
[11] G Bloch F Lauer G Colin and Y Chamaillard ldquoSupportvector regression from simulation data and few experimentalsamplesrdquo Information Sciences vol 178 no 20 pp 3813ndash38272008
[12] J Zhou B Duan J Huang and N Li ldquoIncorporating priorknowledge and multi-kernel into linear programming supportvector regressionrdquo Soft Computing vol 19 no 7 pp 2047ndash20612015
[13] F Lauer and G Bloch ldquoIncorporating prior knowledge insupport vector machines for classification A reviewrdquo Neuro-computing vol 71 no 7-9 pp 1578ndash1594 2008
[14] C Pan Y Dong X Yan and W Zhao ldquoHybrid model for mainand side reactions of p-xylene oxidation with factor influencebased monotone additive SVRrdquo Chemometrics and IntelligentLaboratory Systems vol 136 pp 36ndash46 2014
[15] O Naghash-Almasi and M H Khooban ldquoPI adaptive LS-SVRcontrol scheme with disturbance rejection for a class of uncer-tain nonlinear systemsrdquo Engineering Applications of ArtificialIntelligence vol 52 pp 135ndash144 2016
[16] Y Wang and L Zhang ldquoA combined fault diagnosis method forpower transformer in big data environmentrdquo MathematicalProblems in Engineering vol 2017 Article ID 9670290 6 pages2017
[17] H Dai B Zhang andWWang ldquoAmultiwavelet support vectorregression method for efficient reliability assessmentrdquo Reliabil-ity Engineering and System Safety vol 136 pp 132ndash139 2015
[18] P-Y Hao ldquoPairing support vector algorithm for data regres-sionrdquo Neurocomputing vol 225 pp 174ndash187 2017
[19] W-CHong ldquoChaotic particle swarmoptimization algorithm ina support vector regression electric load forecasting modelrdquoEnergy Conversion and Management vol 50 no 1 pp 105ndash1172009
[20] G Wang T R Carr Y Ju and C Li ldquoIdentifying organic-richMarcellus Shale lithofacies by support vector machine classifierin the Appalachian basinrdquo Computers and Geosciences vol 64pp 52ndash60 2014
[21] W Wen Z Hao and X Yang ldquoA heuristic weight-setting strat-egy and iteratively updating algorithm for weighted least-squares support vector regressionrdquoNeurocomputing vol 71 no16-18 pp 3096ndash3103 2008
[22] H Xiong Z Chen H Qiu H Hao and H Xu ldquoAdaptiveSVR-HDMR metamodeling technique for high dimensionalproblemsrdquo International Conference onModelling Identificationand Control AASRI Procediapp vol 3 pp 95ndash100 2012
[23] M Lichman Auto MPG Data Set UCI Machine LearningRepository httparchiveicsuciedumldatasetsAuto+MPG2013
[24] M Lichman Computer Hardware Data Set UCI MachineLearningRepository httparchiveicsuciedumldatasetsCom-puter+Hardware 2013
[25] A Tsanas and A Xifara ldquoAccurate quantitative estimation ofenergy performance of residential buildings using statisticalmachine learning toolsrdquo Energy and Buildings vol 49 pp 560ndash567 2012
[26] R Rinnan and A Rinnan ldquoApplication of near infrared reflec-tance (NIR) and fluorescence spectroscopy to analysis ofmicro-biological and chemical properties of arctic soilrdquo Soil Biologyand Biochemistry vol 39 no 7 pp 1664ndash1673 2007
Submit your manuscripts athttpswwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 7
0 1 2 3 4 5 6 7 8 9 10
04
05
06
07
08
09
1
Noise intensity
Aver
age v
alue
PDISVRSVR
HWSVRMKSVR
(a)
0 1 2 3 4 5 6 7 8 9 100
1
2
3
4
5
6
7
Noise intensity
Stan
dard
dev
iatio
n
PDISVRSVR
HWSVRMKSVR
(b)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
0005001
0015002
0025003
0035004
0045
Noise intensity
Aver
age v
alue
(c)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
051
152
253
354
45
Noise intensity
Stan
dard
dev
iatio
n
(d)
Figure 4 Four modelsrsquo prediction results in radial basis function kernel with different noise (a) is the average value of correlation coefficient119903 (b) is the average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
Table 2 Testing results of SVR algorithms with rough error (linear kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 089278 086844 084402 085828Standard deviation 0000258 0002867 0002707 0001998
MSEAverage value 030614 052311 058221 045612Standard deviation 0003953 0111161 0045897 0028321
Table 3 Testing results of SVR algorithms with rough error (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 078424 078126 075976 064346Standard deviation 0000297 0000169 0000774 0000244
MSEAverage value 070686 073748 062362 084296Standard deviation 000316 000150 000506 000136
are listed in Figure 6 The weight values of the HWSVRalgorithm are listed in Figure 7 As shown in Figure 6 theweights of two trivial outliers were 0 and 010531 for the 17thand 48th data respectively and the weight of one strong out-lier was 000036 which indicated that the PDISVR precisely
detected the outliers As shown in Figure 7 the HWSVR didnot perform as well as PDISVR One strong outlier had aweight of 00751 and two trivial outliers had weights of 03143and 02729 which were unsuitable for modeling given thatsmaller weights such as that of the 23rd datum (00126)
8 Mathematical Problems in Engineering
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 10
04
05
06
07
08
09
1
Noise intensity
Aver
age v
alue
(a)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 1002468
1012141618
Noise intensity
Aver
age v
alue
(b)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
0005
001
0015
002
0025
003
0035
Noise intensity
Stan
dard
dev
iatio
n
(c)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
50
100
150
200
250
Noise intensity
Stan
dard
dev
iatio
n
(d)
Figure 5 Four modelsrsquo prediction results in polynomial kernel with different noise (a) is the average value of correlation coefficient 119903 (b) isthe average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
Table 4 Testing results of SVR algorithms with rough error (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 083278 079399 075874 08137Standard deviation 0000377 0004657 0009121 0003568
MSEAverage value 04473 0727 148208 062433Standard deviation 0003168 0076291 1462867 0028654
could affect outlier detection Thus the influence from twotrivial outliers on the predictability of the PDISVR algorithmwas reduced and the influence from one strong outlier waseliminated By contrast the effect of outliers remained in theHWSVR algorithm
43 Example 3 To test our regressionmodel in amore realis-tic way we imported six more realistic datasets from the UCIMachine Learning Repository [23ndash25] Department of FoodScience University of Copenhagen database [26] and somereal chemical industrial process [14] See Table 5 for moredetailed information In these datasets four out of five datawere used for training and one-fifth of the data was used for
testing The hyperparameters used in this example are alsoobtained by the process introduced in Section 33
As shown in Tables 6ndash8 the proposed PDISVR obtainedthe best predictive ability in the majority of the criterion Forexample in the case of Auto-MPG the proposed PDISVRwas best achieved with both standards Thus the proposedPDISVR is appropriate for the Auto-MPG dataset In thedatasets for crude oil distillation and computer hardwarethe proposed PDISVR only obtained the best correlationcoefficient r and could not establish a suitablemodel at a pointwhere the probability distribution was low thus increasingthe MSE Therefore the use of PDISVR requires validation
Mathematical Problems in Engineering 9
Table 5 Details of the experimental datasets
Data Number of instances Number of attributes Dataset information
Combustion side reaction 144 6The combustion side reaction is the process by which
the solvent and reactants are oxidized to carbondioxide carbon monoxide and water
Auto-MPG dataset 398 8 Data concerning city-cycle fuel consumption
Computer hardware dataset 209 9 Relative CPU performance in terms of cycling timememory size and other parameters
Soil samples measured byNear-Infrared Spectroscopy(NIR)
108 1050 This data is about the soil organic matter (SOM) fromnorthern Sweden (681210N 181490E)
Energy efficiency 768 9This study looked into assessing the heating load andcooling load requirements of buildings (that is energy
efficiency) as a function of building parameters
Crude oil distillation process 30 1 Data related to different components with differentboiling points
Table 6 Comparative results of previous SVR models in real datasets (linear kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 07141 06790 06797 06939MSE 00051 00050 00046 00044
Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802
Computer hardware 119903 09533 08750 08748 09012MSE 79872 83737 83815 65772
NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858
Energy efficiency 119903 1 1 1 09999MSE 00047 11092 00221 00818
Crude oil distillation process r 09372 07684 09372 09927MSE 10162 9724 14510 81628
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 010531
X = 17Y = 0
X = 50
Y = 000036228
Figure 6 Weights of the training sample for PDISVR
through additional research and dataset message Moreoverin the sample of crude oil boiling point there are far lessquantity of data and the PDISVR cannot differentiate the
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 027292
X = 17Y = 031429
X = 50
Y = 0075054
Figure 7 Weights of the training sample for HWSVR
noise according to its probability distribution Thus theproposed PDISVR is applied to improve the SVR in the caseof large datasets
10 Mathematical Problems in Engineering
Table 7 Comparative results of previous SVR models in real datasets (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 06664 0425 06031 05827MSE 00038 00095 06031 00578
Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482
Computer hardware 119903 07815 08597 08647 08851MSE 27961 19986 15355 13160
NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235
Energy efficiency 119903 1 09796 09901 09997MSE 0000269 41875 3019 00474
Crude oil distillation process 119903 09899 09739 09913 09573MSE 477425 787604 11198 3176248
Table 8 Comparative results of previous SVR models in real datasets (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 05796 06853 08078 05518MSE 00081 00106 03546 01237
Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559
Computer hardware 119903 08337 08571 08519 08423MSE 125509 195298 198641 139781
NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073
Energy efficiency 119903 1 09964 09988 09867MSE 00221 04598 01676 41448
Crude oil distillation process 119903 09571 08071 09807 09913MSE 3218016 1129353 543637 11054000
5 Conclusion
In traditional SVR modeling the deviation between themodel outputs and the real data is the only way to representthe influence of noise Other information such as possibilitydistribution information is not emphasized Therefore weproposed a special method that uses the possibility dis-tribution information to modify the basic error parameterand slack variables of SVR Given that these parametersare weighted by the probability distribution of the modeloutput points they can be adjusted by SVR itself and nolonger require optimization by any intelligent optimizationalgorithmThe proposed algorithm is superior to other SVR-based algorithms in dealing with noisy data and outliers insimulation and actual datasets
Conflicts of Interest
The authors declare that they have no conflicts of interest
References
[1] VN VapnikTheNature of Statistical LearningTheory Springer2000
[2] S Yin and J Yin ldquoTuning kernel parameters for SVM based onexpected square distance ratiordquo Information Sciences vol 370-371 pp 92ndash102 2016
[3] J A K Suykens J Vandewalle and B de Moor ldquoOptimalcontrol by least squares support vector machinesrdquo NeuralNetworks vol 14 no 1 pp 23ndash35 2001
[4] J A K Suykens J De Brabanter L Lukas and J VandewalleldquoWeighted least squares support vector machines robustnessand sparce approximationrdquo Neurocomputing vol 48 pp 85ndash105 2002
[5] H-S Tang S-T Xue R Chen and T Sato ldquoOnline weightedLS-SVM for hysteretic structural system identificationrdquo Engi-neering Structures vol 28 no 12 pp 1728ndash1735 2006
[6] A Smola B Schoelkopf and G Raetsch ldquoLinear programsfor automatic accuracy control in regressionrdquo in Proceedings of9th International Conference on Artificial Neural Networks(ICANN99) pp 575ndash580 IEE UK September 1999
[7] A Smola and B Scholkopf Learning with Kernels MIT PressCambridge UK 2002
[8] C Y Yeh CWHuang and S J Lee ldquoAmultiple-kernel supportvector regression approach for stock market price forecastingrdquoExpert Systems with Applications vol 38 no 3 pp 2177ndash21862011
[9] W-J Lin and S-S Jhuo ldquoA fast luminance inspector forbacklight modules based on multiple kernel support vector
Mathematical Problems in Engineering 11
regressionrdquo IEEE Transactions on Components Packaging andManufacturing Technology vol 4 no 8 pp 1391ndash1401 2014
[10] Z Zhong and T R Carr ldquoApplication of mixed kernels function(MKF) based support vector regressionmodel (SVR) for CO2 ndashReservoir oil minimum miscibility pressure predictionrdquo Fuelvol 184 pp 590ndash603 2016
[11] G Bloch F Lauer G Colin and Y Chamaillard ldquoSupportvector regression from simulation data and few experimentalsamplesrdquo Information Sciences vol 178 no 20 pp 3813ndash38272008
[12] J Zhou B Duan J Huang and N Li ldquoIncorporating priorknowledge and multi-kernel into linear programming supportvector regressionrdquo Soft Computing vol 19 no 7 pp 2047ndash20612015
[13] F Lauer and G Bloch ldquoIncorporating prior knowledge insupport vector machines for classification A reviewrdquo Neuro-computing vol 71 no 7-9 pp 1578ndash1594 2008
[14] C Pan Y Dong X Yan and W Zhao ldquoHybrid model for mainand side reactions of p-xylene oxidation with factor influencebased monotone additive SVRrdquo Chemometrics and IntelligentLaboratory Systems vol 136 pp 36ndash46 2014
[15] O Naghash-Almasi and M H Khooban ldquoPI adaptive LS-SVRcontrol scheme with disturbance rejection for a class of uncer-tain nonlinear systemsrdquo Engineering Applications of ArtificialIntelligence vol 52 pp 135ndash144 2016
[16] Y Wang and L Zhang ldquoA combined fault diagnosis method forpower transformer in big data environmentrdquo MathematicalProblems in Engineering vol 2017 Article ID 9670290 6 pages2017
[17] H Dai B Zhang andWWang ldquoAmultiwavelet support vectorregression method for efficient reliability assessmentrdquo Reliabil-ity Engineering and System Safety vol 136 pp 132ndash139 2015
[18] P-Y Hao ldquoPairing support vector algorithm for data regres-sionrdquo Neurocomputing vol 225 pp 174ndash187 2017
[19] W-CHong ldquoChaotic particle swarmoptimization algorithm ina support vector regression electric load forecasting modelrdquoEnergy Conversion and Management vol 50 no 1 pp 105ndash1172009
[20] G Wang T R Carr Y Ju and C Li ldquoIdentifying organic-richMarcellus Shale lithofacies by support vector machine classifierin the Appalachian basinrdquo Computers and Geosciences vol 64pp 52ndash60 2014
[21] W Wen Z Hao and X Yang ldquoA heuristic weight-setting strat-egy and iteratively updating algorithm for weighted least-squares support vector regressionrdquoNeurocomputing vol 71 no16-18 pp 3096ndash3103 2008
[22] H Xiong Z Chen H Qiu H Hao and H Xu ldquoAdaptiveSVR-HDMR metamodeling technique for high dimensionalproblemsrdquo International Conference onModelling Identificationand Control AASRI Procediapp vol 3 pp 95ndash100 2012
[23] M Lichman Auto MPG Data Set UCI Machine LearningRepository httparchiveicsuciedumldatasetsAuto+MPG2013
[24] M Lichman Computer Hardware Data Set UCI MachineLearningRepository httparchiveicsuciedumldatasetsCom-puter+Hardware 2013
[25] A Tsanas and A Xifara ldquoAccurate quantitative estimation ofenergy performance of residential buildings using statisticalmachine learning toolsrdquo Energy and Buildings vol 49 pp 560ndash567 2012
[26] R Rinnan and A Rinnan ldquoApplication of near infrared reflec-tance (NIR) and fluorescence spectroscopy to analysis ofmicro-biological and chemical properties of arctic soilrdquo Soil Biologyand Biochemistry vol 39 no 7 pp 1664ndash1673 2007
Submit your manuscripts athttpswwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
8 Mathematical Problems in Engineering
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 10
04
05
06
07
08
09
1
Noise intensity
Aver
age v
alue
(a)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 1002468
1012141618
Noise intensity
Aver
age v
alue
(b)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
0005
001
0015
002
0025
003
0035
Noise intensity
Stan
dard
dev
iatio
n
(c)
PDISVRSVR
HWSVRMKSVR
0 1 2 3 4 5 6 7 8 9 100
50
100
150
200
250
Noise intensity
Stan
dard
dev
iatio
n
(d)
Figure 5 Four modelsrsquo prediction results in polynomial kernel with different noise (a) is the average value of correlation coefficient 119903 (b) isthe average value of MSE (c) is the standard deviation of correlation coefficient 119903 and (d) is the standard deviation of MSE
Table 4 Testing results of SVR algorithms with rough error (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR119903
Average value 083278 079399 075874 08137Standard deviation 0000377 0004657 0009121 0003568
MSEAverage value 04473 0727 148208 062433Standard deviation 0003168 0076291 1462867 0028654
could affect outlier detection Thus the influence from twotrivial outliers on the predictability of the PDISVR algorithmwas reduced and the influence from one strong outlier waseliminated By contrast the effect of outliers remained in theHWSVR algorithm
43 Example 3 To test our regressionmodel in amore realis-tic way we imported six more realistic datasets from the UCIMachine Learning Repository [23ndash25] Department of FoodScience University of Copenhagen database [26] and somereal chemical industrial process [14] See Table 5 for moredetailed information In these datasets four out of five datawere used for training and one-fifth of the data was used for
testing The hyperparameters used in this example are alsoobtained by the process introduced in Section 33
As shown in Tables 6ndash8 the proposed PDISVR obtainedthe best predictive ability in the majority of the criterion Forexample in the case of Auto-MPG the proposed PDISVRwas best achieved with both standards Thus the proposedPDISVR is appropriate for the Auto-MPG dataset In thedatasets for crude oil distillation and computer hardwarethe proposed PDISVR only obtained the best correlationcoefficient r and could not establish a suitablemodel at a pointwhere the probability distribution was low thus increasingthe MSE Therefore the use of PDISVR requires validation
Mathematical Problems in Engineering 9
Table 5 Details of the experimental datasets
Data Number of instances Number of attributes Dataset information
Combustion side reaction 144 6The combustion side reaction is the process by which
the solvent and reactants are oxidized to carbondioxide carbon monoxide and water
Auto-MPG dataset 398 8 Data concerning city-cycle fuel consumption
Computer hardware dataset 209 9 Relative CPU performance in terms of cycling timememory size and other parameters
Soil samples measured byNear-Infrared Spectroscopy(NIR)
108 1050 This data is about the soil organic matter (SOM) fromnorthern Sweden (681210N 181490E)
Energy efficiency 768 9This study looked into assessing the heating load andcooling load requirements of buildings (that is energy
efficiency) as a function of building parameters
Crude oil distillation process 30 1 Data related to different components with differentboiling points
Table 6 Comparative results of previous SVR models in real datasets (linear kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 07141 06790 06797 06939MSE 00051 00050 00046 00044
Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802
Computer hardware 119903 09533 08750 08748 09012MSE 79872 83737 83815 65772
NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858
Energy efficiency 119903 1 1 1 09999MSE 00047 11092 00221 00818
Crude oil distillation process r 09372 07684 09372 09927MSE 10162 9724 14510 81628
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 010531
X = 17Y = 0
X = 50
Y = 000036228
Figure 6 Weights of the training sample for PDISVR
through additional research and dataset message Moreoverin the sample of crude oil boiling point there are far lessquantity of data and the PDISVR cannot differentiate the
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 027292
X = 17Y = 031429
X = 50
Y = 0075054
Figure 7 Weights of the training sample for HWSVR
noise according to its probability distribution Thus theproposed PDISVR is applied to improve the SVR in the caseof large datasets
10 Mathematical Problems in Engineering
Table 7 Comparative results of previous SVR models in real datasets (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 06664 0425 06031 05827MSE 00038 00095 06031 00578
Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482
Computer hardware 119903 07815 08597 08647 08851MSE 27961 19986 15355 13160
NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235
Energy efficiency 119903 1 09796 09901 09997MSE 0000269 41875 3019 00474
Crude oil distillation process 119903 09899 09739 09913 09573MSE 477425 787604 11198 3176248
Table 8 Comparative results of previous SVR models in real datasets (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 05796 06853 08078 05518MSE 00081 00106 03546 01237
Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559
Computer hardware 119903 08337 08571 08519 08423MSE 125509 195298 198641 139781
NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073
Energy efficiency 119903 1 09964 09988 09867MSE 00221 04598 01676 41448
Crude oil distillation process 119903 09571 08071 09807 09913MSE 3218016 1129353 543637 11054000
5 Conclusion
In traditional SVR modeling the deviation between themodel outputs and the real data is the only way to representthe influence of noise Other information such as possibilitydistribution information is not emphasized Therefore weproposed a special method that uses the possibility dis-tribution information to modify the basic error parameterand slack variables of SVR Given that these parametersare weighted by the probability distribution of the modeloutput points they can be adjusted by SVR itself and nolonger require optimization by any intelligent optimizationalgorithmThe proposed algorithm is superior to other SVR-based algorithms in dealing with noisy data and outliers insimulation and actual datasets
Conflicts of Interest
The authors declare that they have no conflicts of interest
References
[1] VN VapnikTheNature of Statistical LearningTheory Springer2000
[2] S Yin and J Yin ldquoTuning kernel parameters for SVM based onexpected square distance ratiordquo Information Sciences vol 370-371 pp 92ndash102 2016
[3] J A K Suykens J Vandewalle and B de Moor ldquoOptimalcontrol by least squares support vector machinesrdquo NeuralNetworks vol 14 no 1 pp 23ndash35 2001
[4] J A K Suykens J De Brabanter L Lukas and J VandewalleldquoWeighted least squares support vector machines robustnessand sparce approximationrdquo Neurocomputing vol 48 pp 85ndash105 2002
[5] H-S Tang S-T Xue R Chen and T Sato ldquoOnline weightedLS-SVM for hysteretic structural system identificationrdquo Engi-neering Structures vol 28 no 12 pp 1728ndash1735 2006
[6] A Smola B Schoelkopf and G Raetsch ldquoLinear programsfor automatic accuracy control in regressionrdquo in Proceedings of9th International Conference on Artificial Neural Networks(ICANN99) pp 575ndash580 IEE UK September 1999
[7] A Smola and B Scholkopf Learning with Kernels MIT PressCambridge UK 2002
[8] C Y Yeh CWHuang and S J Lee ldquoAmultiple-kernel supportvector regression approach for stock market price forecastingrdquoExpert Systems with Applications vol 38 no 3 pp 2177ndash21862011
[9] W-J Lin and S-S Jhuo ldquoA fast luminance inspector forbacklight modules based on multiple kernel support vector
Mathematical Problems in Engineering 11
regressionrdquo IEEE Transactions on Components Packaging andManufacturing Technology vol 4 no 8 pp 1391ndash1401 2014
[10] Z Zhong and T R Carr ldquoApplication of mixed kernels function(MKF) based support vector regressionmodel (SVR) for CO2 ndashReservoir oil minimum miscibility pressure predictionrdquo Fuelvol 184 pp 590ndash603 2016
[11] G Bloch F Lauer G Colin and Y Chamaillard ldquoSupportvector regression from simulation data and few experimentalsamplesrdquo Information Sciences vol 178 no 20 pp 3813ndash38272008
[12] J Zhou B Duan J Huang and N Li ldquoIncorporating priorknowledge and multi-kernel into linear programming supportvector regressionrdquo Soft Computing vol 19 no 7 pp 2047ndash20612015
[13] F Lauer and G Bloch ldquoIncorporating prior knowledge insupport vector machines for classification A reviewrdquo Neuro-computing vol 71 no 7-9 pp 1578ndash1594 2008
[14] C Pan Y Dong X Yan and W Zhao ldquoHybrid model for mainand side reactions of p-xylene oxidation with factor influencebased monotone additive SVRrdquo Chemometrics and IntelligentLaboratory Systems vol 136 pp 36ndash46 2014
[15] O Naghash-Almasi and M H Khooban ldquoPI adaptive LS-SVRcontrol scheme with disturbance rejection for a class of uncer-tain nonlinear systemsrdquo Engineering Applications of ArtificialIntelligence vol 52 pp 135ndash144 2016
[16] Y Wang and L Zhang ldquoA combined fault diagnosis method forpower transformer in big data environmentrdquo MathematicalProblems in Engineering vol 2017 Article ID 9670290 6 pages2017
[17] H Dai B Zhang andWWang ldquoAmultiwavelet support vectorregression method for efficient reliability assessmentrdquo Reliabil-ity Engineering and System Safety vol 136 pp 132ndash139 2015
[18] P-Y Hao ldquoPairing support vector algorithm for data regres-sionrdquo Neurocomputing vol 225 pp 174ndash187 2017
[19] W-CHong ldquoChaotic particle swarmoptimization algorithm ina support vector regression electric load forecasting modelrdquoEnergy Conversion and Management vol 50 no 1 pp 105ndash1172009
[20] G Wang T R Carr Y Ju and C Li ldquoIdentifying organic-richMarcellus Shale lithofacies by support vector machine classifierin the Appalachian basinrdquo Computers and Geosciences vol 64pp 52ndash60 2014
[21] W Wen Z Hao and X Yang ldquoA heuristic weight-setting strat-egy and iteratively updating algorithm for weighted least-squares support vector regressionrdquoNeurocomputing vol 71 no16-18 pp 3096ndash3103 2008
[22] H Xiong Z Chen H Qiu H Hao and H Xu ldquoAdaptiveSVR-HDMR metamodeling technique for high dimensionalproblemsrdquo International Conference onModelling Identificationand Control AASRI Procediapp vol 3 pp 95ndash100 2012
[23] M Lichman Auto MPG Data Set UCI Machine LearningRepository httparchiveicsuciedumldatasetsAuto+MPG2013
[24] M Lichman Computer Hardware Data Set UCI MachineLearningRepository httparchiveicsuciedumldatasetsCom-puter+Hardware 2013
[25] A Tsanas and A Xifara ldquoAccurate quantitative estimation ofenergy performance of residential buildings using statisticalmachine learning toolsrdquo Energy and Buildings vol 49 pp 560ndash567 2012
[26] R Rinnan and A Rinnan ldquoApplication of near infrared reflec-tance (NIR) and fluorescence spectroscopy to analysis ofmicro-biological and chemical properties of arctic soilrdquo Soil Biologyand Biochemistry vol 39 no 7 pp 1664ndash1673 2007
Submit your manuscripts athttpswwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 9
Table 5 Details of the experimental datasets
Data Number of instances Number of attributes Dataset information
Combustion side reaction 144 6The combustion side reaction is the process by which
the solvent and reactants are oxidized to carbondioxide carbon monoxide and water
Auto-MPG dataset 398 8 Data concerning city-cycle fuel consumption
Computer hardware dataset 209 9 Relative CPU performance in terms of cycling timememory size and other parameters
Soil samples measured byNear-Infrared Spectroscopy(NIR)
108 1050 This data is about the soil organic matter (SOM) fromnorthern Sweden (681210N 181490E)
Energy efficiency 768 9This study looked into assessing the heating load andcooling load requirements of buildings (that is energy
efficiency) as a function of building parameters
Crude oil distillation process 30 1 Data related to different components with differentboiling points
Table 6 Comparative results of previous SVR models in real datasets (linear kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 07141 06790 06797 06939MSE 00051 00050 00046 00044
Auto-MPG 119903 09055 08578 08992 08554MSE 190023 258051 237999 204802
Computer hardware 119903 09533 08750 08748 09012MSE 79872 83737 83815 65772
NIRSOIL 119903 08033 07335 07712 07657MSE 18061 22303 20851 23858
Energy efficiency 119903 1 1 1 09999MSE 00047 11092 00221 00818
Crude oil distillation process r 09372 07684 09372 09927MSE 10162 9724 14510 81628
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 010531
X = 17Y = 0
X = 50
Y = 000036228
Figure 6 Weights of the training sample for PDISVR
through additional research and dataset message Moreoverin the sample of crude oil boiling point there are far lessquantity of data and the PDISVR cannot differentiate the
0 10 20 30 40 50 60 70 800
01
02
03
04
05
06
07
08
09
1
Sample number
Wei
ght o
f dat
a
X = 48Y = 027292
X = 17Y = 031429
X = 50
Y = 0075054
Figure 7 Weights of the training sample for HWSVR
noise according to its probability distribution Thus theproposed PDISVR is applied to improve the SVR in the caseof large datasets
10 Mathematical Problems in Engineering
Table 7 Comparative results of previous SVR models in real datasets (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 06664 0425 06031 05827MSE 00038 00095 06031 00578
Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482
Computer hardware 119903 07815 08597 08647 08851MSE 27961 19986 15355 13160
NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235
Energy efficiency 119903 1 09796 09901 09997MSE 0000269 41875 3019 00474
Crude oil distillation process 119903 09899 09739 09913 09573MSE 477425 787604 11198 3176248
Table 8 Comparative results of previous SVR models in real datasets (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 05796 06853 08078 05518MSE 00081 00106 03546 01237
Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559
Computer hardware 119903 08337 08571 08519 08423MSE 125509 195298 198641 139781
NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073
Energy efficiency 119903 1 09964 09988 09867MSE 00221 04598 01676 41448
Crude oil distillation process 119903 09571 08071 09807 09913MSE 3218016 1129353 543637 11054000
5 Conclusion
In traditional SVR modeling the deviation between themodel outputs and the real data is the only way to representthe influence of noise Other information such as possibilitydistribution information is not emphasized Therefore weproposed a special method that uses the possibility dis-tribution information to modify the basic error parameterand slack variables of SVR Given that these parametersare weighted by the probability distribution of the modeloutput points they can be adjusted by SVR itself and nolonger require optimization by any intelligent optimizationalgorithmThe proposed algorithm is superior to other SVR-based algorithms in dealing with noisy data and outliers insimulation and actual datasets
Conflicts of Interest
The authors declare that they have no conflicts of interest
References
[1] VN VapnikTheNature of Statistical LearningTheory Springer2000
[2] S Yin and J Yin ldquoTuning kernel parameters for SVM based onexpected square distance ratiordquo Information Sciences vol 370-371 pp 92ndash102 2016
[3] J A K Suykens J Vandewalle and B de Moor ldquoOptimalcontrol by least squares support vector machinesrdquo NeuralNetworks vol 14 no 1 pp 23ndash35 2001
[4] J A K Suykens J De Brabanter L Lukas and J VandewalleldquoWeighted least squares support vector machines robustnessand sparce approximationrdquo Neurocomputing vol 48 pp 85ndash105 2002
[5] H-S Tang S-T Xue R Chen and T Sato ldquoOnline weightedLS-SVM for hysteretic structural system identificationrdquo Engi-neering Structures vol 28 no 12 pp 1728ndash1735 2006
[6] A Smola B Schoelkopf and G Raetsch ldquoLinear programsfor automatic accuracy control in regressionrdquo in Proceedings of9th International Conference on Artificial Neural Networks(ICANN99) pp 575ndash580 IEE UK September 1999
[7] A Smola and B Scholkopf Learning with Kernels MIT PressCambridge UK 2002
[8] C Y Yeh CWHuang and S J Lee ldquoAmultiple-kernel supportvector regression approach for stock market price forecastingrdquoExpert Systems with Applications vol 38 no 3 pp 2177ndash21862011
[9] W-J Lin and S-S Jhuo ldquoA fast luminance inspector forbacklight modules based on multiple kernel support vector
Mathematical Problems in Engineering 11
regressionrdquo IEEE Transactions on Components Packaging andManufacturing Technology vol 4 no 8 pp 1391ndash1401 2014
[10] Z Zhong and T R Carr ldquoApplication of mixed kernels function(MKF) based support vector regressionmodel (SVR) for CO2 ndashReservoir oil minimum miscibility pressure predictionrdquo Fuelvol 184 pp 590ndash603 2016
[11] G Bloch F Lauer G Colin and Y Chamaillard ldquoSupportvector regression from simulation data and few experimentalsamplesrdquo Information Sciences vol 178 no 20 pp 3813ndash38272008
[12] J Zhou B Duan J Huang and N Li ldquoIncorporating priorknowledge and multi-kernel into linear programming supportvector regressionrdquo Soft Computing vol 19 no 7 pp 2047ndash20612015
[13] F Lauer and G Bloch ldquoIncorporating prior knowledge insupport vector machines for classification A reviewrdquo Neuro-computing vol 71 no 7-9 pp 1578ndash1594 2008
[14] C Pan Y Dong X Yan and W Zhao ldquoHybrid model for mainand side reactions of p-xylene oxidation with factor influencebased monotone additive SVRrdquo Chemometrics and IntelligentLaboratory Systems vol 136 pp 36ndash46 2014
[15] O Naghash-Almasi and M H Khooban ldquoPI adaptive LS-SVRcontrol scheme with disturbance rejection for a class of uncer-tain nonlinear systemsrdquo Engineering Applications of ArtificialIntelligence vol 52 pp 135ndash144 2016
[16] Y Wang and L Zhang ldquoA combined fault diagnosis method forpower transformer in big data environmentrdquo MathematicalProblems in Engineering vol 2017 Article ID 9670290 6 pages2017
[17] H Dai B Zhang andWWang ldquoAmultiwavelet support vectorregression method for efficient reliability assessmentrdquo Reliabil-ity Engineering and System Safety vol 136 pp 132ndash139 2015
[18] P-Y Hao ldquoPairing support vector algorithm for data regres-sionrdquo Neurocomputing vol 225 pp 174ndash187 2017
[19] W-CHong ldquoChaotic particle swarmoptimization algorithm ina support vector regression electric load forecasting modelrdquoEnergy Conversion and Management vol 50 no 1 pp 105ndash1172009
[20] G Wang T R Carr Y Ju and C Li ldquoIdentifying organic-richMarcellus Shale lithofacies by support vector machine classifierin the Appalachian basinrdquo Computers and Geosciences vol 64pp 52ndash60 2014
[21] W Wen Z Hao and X Yang ldquoA heuristic weight-setting strat-egy and iteratively updating algorithm for weighted least-squares support vector regressionrdquoNeurocomputing vol 71 no16-18 pp 3096ndash3103 2008
[22] H Xiong Z Chen H Qiu H Hao and H Xu ldquoAdaptiveSVR-HDMR metamodeling technique for high dimensionalproblemsrdquo International Conference onModelling Identificationand Control AASRI Procediapp vol 3 pp 95ndash100 2012
[23] M Lichman Auto MPG Data Set UCI Machine LearningRepository httparchiveicsuciedumldatasetsAuto+MPG2013
[24] M Lichman Computer Hardware Data Set UCI MachineLearningRepository httparchiveicsuciedumldatasetsCom-puter+Hardware 2013
[25] A Tsanas and A Xifara ldquoAccurate quantitative estimation ofenergy performance of residential buildings using statisticalmachine learning toolsrdquo Energy and Buildings vol 49 pp 560ndash567 2012
[26] R Rinnan and A Rinnan ldquoApplication of near infrared reflec-tance (NIR) and fluorescence spectroscopy to analysis ofmicro-biological and chemical properties of arctic soilrdquo Soil Biologyand Biochemistry vol 39 no 7 pp 1664ndash1673 2007
Submit your manuscripts athttpswwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
10 Mathematical Problems in Engineering
Table 7 Comparative results of previous SVR models in real datasets (radial basis function kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 06664 0425 06031 05827MSE 00038 00095 06031 00578
Auto-MPG 119903 05594 0614 06261 06563MSE 204277 261598 1697247 220482
Computer hardware 119903 07815 08597 08647 08851MSE 27961 19986 15355 13160
NIRSOIL 119903 07278 06707 05565 05666MSE 1113 13023 16011 35235
Energy efficiency 119903 1 09796 09901 09997MSE 0000269 41875 3019 00474
Crude oil distillation process 119903 09899 09739 09913 09573MSE 477425 787604 11198 3176248
Table 8 Comparative results of previous SVR models in real datasets (polynomial kernel)
Criterion PDISVR SVR HWSVR MKSVR
Combustion side reaction 119903 05796 06853 08078 05518MSE 00081 00106 03546 01237
Auto-MPG 119903 09124 09338 09341 08604MSE 18213 67933 68144 48559
Computer hardware 119903 08337 08571 08519 08423MSE 125509 195298 198641 139781
NIRSOIL 119903 06576 06342 06468 06429MSE 27513 31047 29189 30073
Energy efficiency 119903 1 09964 09988 09867MSE 00221 04598 01676 41448
Crude oil distillation process 119903 09571 08071 09807 09913MSE 3218016 1129353 543637 11054000
5 Conclusion
In traditional SVR modeling the deviation between themodel outputs and the real data is the only way to representthe influence of noise Other information such as possibilitydistribution information is not emphasized Therefore weproposed a special method that uses the possibility dis-tribution information to modify the basic error parameterand slack variables of SVR Given that these parametersare weighted by the probability distribution of the modeloutput points they can be adjusted by SVR itself and nolonger require optimization by any intelligent optimizationalgorithmThe proposed algorithm is superior to other SVR-based algorithms in dealing with noisy data and outliers insimulation and actual datasets
Conflicts of Interest
The authors declare that they have no conflicts of interest
References
[1] VN VapnikTheNature of Statistical LearningTheory Springer2000
[2] S Yin and J Yin ldquoTuning kernel parameters for SVM based onexpected square distance ratiordquo Information Sciences vol 370-371 pp 92ndash102 2016
[3] J A K Suykens J Vandewalle and B de Moor ldquoOptimalcontrol by least squares support vector machinesrdquo NeuralNetworks vol 14 no 1 pp 23ndash35 2001
[4] J A K Suykens J De Brabanter L Lukas and J VandewalleldquoWeighted least squares support vector machines robustnessand sparce approximationrdquo Neurocomputing vol 48 pp 85ndash105 2002
[5] H-S Tang S-T Xue R Chen and T Sato ldquoOnline weightedLS-SVM for hysteretic structural system identificationrdquo Engi-neering Structures vol 28 no 12 pp 1728ndash1735 2006
[6] A Smola B Schoelkopf and G Raetsch ldquoLinear programsfor automatic accuracy control in regressionrdquo in Proceedings of9th International Conference on Artificial Neural Networks(ICANN99) pp 575ndash580 IEE UK September 1999
[7] A Smola and B Scholkopf Learning with Kernels MIT PressCambridge UK 2002
[8] C Y Yeh CWHuang and S J Lee ldquoAmultiple-kernel supportvector regression approach for stock market price forecastingrdquoExpert Systems with Applications vol 38 no 3 pp 2177ndash21862011
[9] W-J Lin and S-S Jhuo ldquoA fast luminance inspector forbacklight modules based on multiple kernel support vector
Mathematical Problems in Engineering 11
regressionrdquo IEEE Transactions on Components Packaging andManufacturing Technology vol 4 no 8 pp 1391ndash1401 2014
[10] Z Zhong and T R Carr ldquoApplication of mixed kernels function(MKF) based support vector regressionmodel (SVR) for CO2 ndashReservoir oil minimum miscibility pressure predictionrdquo Fuelvol 184 pp 590ndash603 2016
[11] G Bloch F Lauer G Colin and Y Chamaillard ldquoSupportvector regression from simulation data and few experimentalsamplesrdquo Information Sciences vol 178 no 20 pp 3813ndash38272008
[12] J Zhou B Duan J Huang and N Li ldquoIncorporating priorknowledge and multi-kernel into linear programming supportvector regressionrdquo Soft Computing vol 19 no 7 pp 2047ndash20612015
[13] F Lauer and G Bloch ldquoIncorporating prior knowledge insupport vector machines for classification A reviewrdquo Neuro-computing vol 71 no 7-9 pp 1578ndash1594 2008
[14] C Pan Y Dong X Yan and W Zhao ldquoHybrid model for mainand side reactions of p-xylene oxidation with factor influencebased monotone additive SVRrdquo Chemometrics and IntelligentLaboratory Systems vol 136 pp 36ndash46 2014
[15] O Naghash-Almasi and M H Khooban ldquoPI adaptive LS-SVRcontrol scheme with disturbance rejection for a class of uncer-tain nonlinear systemsrdquo Engineering Applications of ArtificialIntelligence vol 52 pp 135ndash144 2016
[16] Y Wang and L Zhang ldquoA combined fault diagnosis method forpower transformer in big data environmentrdquo MathematicalProblems in Engineering vol 2017 Article ID 9670290 6 pages2017
[17] H Dai B Zhang andWWang ldquoAmultiwavelet support vectorregression method for efficient reliability assessmentrdquo Reliabil-ity Engineering and System Safety vol 136 pp 132ndash139 2015
[18] P-Y Hao ldquoPairing support vector algorithm for data regres-sionrdquo Neurocomputing vol 225 pp 174ndash187 2017
[19] W-CHong ldquoChaotic particle swarmoptimization algorithm ina support vector regression electric load forecasting modelrdquoEnergy Conversion and Management vol 50 no 1 pp 105ndash1172009
[20] G Wang T R Carr Y Ju and C Li ldquoIdentifying organic-richMarcellus Shale lithofacies by support vector machine classifierin the Appalachian basinrdquo Computers and Geosciences vol 64pp 52ndash60 2014
[21] W Wen Z Hao and X Yang ldquoA heuristic weight-setting strat-egy and iteratively updating algorithm for weighted least-squares support vector regressionrdquoNeurocomputing vol 71 no16-18 pp 3096ndash3103 2008
[22] H Xiong Z Chen H Qiu H Hao and H Xu ldquoAdaptiveSVR-HDMR metamodeling technique for high dimensionalproblemsrdquo International Conference onModelling Identificationand Control AASRI Procediapp vol 3 pp 95ndash100 2012
[23] M Lichman Auto MPG Data Set UCI Machine LearningRepository httparchiveicsuciedumldatasetsAuto+MPG2013
[24] M Lichman Computer Hardware Data Set UCI MachineLearningRepository httparchiveicsuciedumldatasetsCom-puter+Hardware 2013
[25] A Tsanas and A Xifara ldquoAccurate quantitative estimation ofenergy performance of residential buildings using statisticalmachine learning toolsrdquo Energy and Buildings vol 49 pp 560ndash567 2012
[26] R Rinnan and A Rinnan ldquoApplication of near infrared reflec-tance (NIR) and fluorescence spectroscopy to analysis ofmicro-biological and chemical properties of arctic soilrdquo Soil Biologyand Biochemistry vol 39 no 7 pp 1664ndash1673 2007
Submit your manuscripts athttpswwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Mathematical Problems in Engineering 11
regressionrdquo IEEE Transactions on Components Packaging andManufacturing Technology vol 4 no 8 pp 1391ndash1401 2014
[10] Z Zhong and T R Carr ldquoApplication of mixed kernels function(MKF) based support vector regressionmodel (SVR) for CO2 ndashReservoir oil minimum miscibility pressure predictionrdquo Fuelvol 184 pp 590ndash603 2016
[11] G Bloch F Lauer G Colin and Y Chamaillard ldquoSupportvector regression from simulation data and few experimentalsamplesrdquo Information Sciences vol 178 no 20 pp 3813ndash38272008
[12] J Zhou B Duan J Huang and N Li ldquoIncorporating priorknowledge and multi-kernel into linear programming supportvector regressionrdquo Soft Computing vol 19 no 7 pp 2047ndash20612015
[13] F Lauer and G Bloch ldquoIncorporating prior knowledge insupport vector machines for classification A reviewrdquo Neuro-computing vol 71 no 7-9 pp 1578ndash1594 2008
[14] C Pan Y Dong X Yan and W Zhao ldquoHybrid model for mainand side reactions of p-xylene oxidation with factor influencebased monotone additive SVRrdquo Chemometrics and IntelligentLaboratory Systems vol 136 pp 36ndash46 2014
[15] O Naghash-Almasi and M H Khooban ldquoPI adaptive LS-SVRcontrol scheme with disturbance rejection for a class of uncer-tain nonlinear systemsrdquo Engineering Applications of ArtificialIntelligence vol 52 pp 135ndash144 2016
[16] Y Wang and L Zhang ldquoA combined fault diagnosis method forpower transformer in big data environmentrdquo MathematicalProblems in Engineering vol 2017 Article ID 9670290 6 pages2017
[17] H Dai B Zhang andWWang ldquoAmultiwavelet support vectorregression method for efficient reliability assessmentrdquo Reliabil-ity Engineering and System Safety vol 136 pp 132ndash139 2015
[18] P-Y Hao ldquoPairing support vector algorithm for data regres-sionrdquo Neurocomputing vol 225 pp 174ndash187 2017
[19] W-CHong ldquoChaotic particle swarmoptimization algorithm ina support vector regression electric load forecasting modelrdquoEnergy Conversion and Management vol 50 no 1 pp 105ndash1172009
[20] G Wang T R Carr Y Ju and C Li ldquoIdentifying organic-richMarcellus Shale lithofacies by support vector machine classifierin the Appalachian basinrdquo Computers and Geosciences vol 64pp 52ndash60 2014
[21] W Wen Z Hao and X Yang ldquoA heuristic weight-setting strat-egy and iteratively updating algorithm for weighted least-squares support vector regressionrdquoNeurocomputing vol 71 no16-18 pp 3096ndash3103 2008
[22] H Xiong Z Chen H Qiu H Hao and H Xu ldquoAdaptiveSVR-HDMR metamodeling technique for high dimensionalproblemsrdquo International Conference onModelling Identificationand Control AASRI Procediapp vol 3 pp 95ndash100 2012
[23] M Lichman Auto MPG Data Set UCI Machine LearningRepository httparchiveicsuciedumldatasetsAuto+MPG2013
[24] M Lichman Computer Hardware Data Set UCI MachineLearningRepository httparchiveicsuciedumldatasetsCom-puter+Hardware 2013
[25] A Tsanas and A Xifara ldquoAccurate quantitative estimation ofenergy performance of residential buildings using statisticalmachine learning toolsrdquo Energy and Buildings vol 49 pp 560ndash567 2012
[26] R Rinnan and A Rinnan ldquoApplication of near infrared reflec-tance (NIR) and fluorescence spectroscopy to analysis ofmicro-biological and chemical properties of arctic soilrdquo Soil Biologyand Biochemistry vol 39 no 7 pp 1664ndash1673 2007
Submit your manuscripts athttpswwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of
Submit your manuscripts athttpswwwhindawicom
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical Problems in Engineering
Hindawi Publishing Corporationhttpwwwhindawicom
Differential EquationsInternational Journal of
Volume 2014
Applied MathematicsJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Mathematical PhysicsAdvances in
Complex AnalysisJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
OptimizationJournal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Operations ResearchAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Function Spaces
Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014
International Journal of Mathematics and Mathematical Sciences
Hindawi Publishing Corporationhttpwwwhindawicom Volume 201
The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Algebra
Discrete Dynamics in Nature and Society
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Decision SciencesAdvances in
Journal of
Hindawi Publishing Corporationhttpwwwhindawicom
Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014
Stochastic AnalysisInternational Journal of