intrusion detection system based on enhanced pls feature...

International Journal of Science, Engineering and Technology Research (IJSETR), Volume 3, Issue 6, June 2014

1655ISSN: 2278 – 7798 All Rights Reserved © 2014 IJSETR

Intrusion Detection System based on Enhanced PLS Feature Extraction with Hybrid

classification Method1S.M.Kannathal,

1PG Scholar Department of Computer Science and Engineering,

Avinashilingam Institute for Home Science and Higher Education for WomenCoimbatore, Tamil Nadu, India.

Abstract -Computer technology and the popularity of internet has been increased, which leads to the attention of network security. Today network security has become a challenging task in order to protect security goals. Intrusion prevention techniques like encoding and authentication alone for not enough and detection techniques are also needed. Intrusion detection, a network security mechanism for monitoring, preventing and resisting intrusions, plays a very important role in

ensuring network security. However many IDS are deployed, an efficient system is needed for intrusion detection. This paper uses Single Value Decomposition (SVD) to enhance Partial Least Square (PLS) feature extraction, a hybrid classifier and performance is evaluated using KDD cup 99 dataset.

Keywords – Feature Extraction, KDD99, Partial Least Square, Single Value Decomposition.

1. INTRODUCTIONWith the extensive use of Internet the

possibilities of exposing sensitive information to attackers increases. Intrusion is a group of activity which try to encompass the privacy, rejection of resources or illegal use of resources in other words, any act that knowingly deviates from the normal behavior is considered as intrusion.Intrusion Detection System (IDS) is used for detecting various intrusions in network environment and to prevent information from malicious attackers. Detection is not introduced to replace prevention-based techniques such as authentication and access device as an alternative, it is planned to balance existing security measures and detect actions that bypass the security monitoring and control component of the system. Intrusion detection is the process of monitoring the events occurring in a computer system or network and analyzing them for signs of intrusions, denoted as attempts to attack or to enhance the security mechanisms of a computer or network. A good IDS identifies all possible intrusions and recommends actions to stop the attacks. When an intruder attacks a system, the ideal response of the system is to stop the activity. The design of IDS is based on the architecture that is durable and can survive when there is an outbreak.

IDS can be classified into generally as misuse intrusion detection and anomaly intrusion detection systems.Misuse detection catches the intrusions in terms of the characteristics of known attacks or system vulnerabilities. Misuse Detection based on known attack actions. Disadvantage cannot detect new or unknown attacks. Anomaly detection detect any action that significantly deviates from the normal

behavior Anomaly Detection based on the normal behavior. Anomaly detection is about finding the normal usage patterns from the examination data, though misuse detection is about training and matching the intrusion patterns using the examination data.

Data mining techniques are used to build an efficient IDSs. In spite of the assurance of improved detection performance and generalization ability of data mining based IDSs, there are some integral complications in the implementation and deployment of these system. We can group these difficulties into three general categories: accuracy, competence, and usability. Typically, data mining based IDSs (especially anomaly detection systems) have higher false positive rates than traditional hand-crafted signature based methods. This prevents them from being able to process audit data and detect intrusions on-line. Finally, these systems involve large amounts of training data and are significantly more complex than traditional systems. The significant theme of our approach is to apply data mining techniques to intrusion detection. The process of (automatically) extracting models from large stores of data is referred to be Data mining. The current fast development in data mining has made available a wide range of algorithms, drawn from various fields such as pattern recognition, machine learning, and etc. Dimensionality reduction is required to reduce the data. Some of the dimensionality reduction technique are feature extraction, feature selection. In feature selection only the required features are chosen according to the objective function, where as in feature extraction all the feature are used by transforming them. The transformed feature contains



all original features in their combination. This paper proposes enhanced Partial Least Square (PLS) method for feature extraction with a hybrid classifierand their performance is evaluated.

This paper is ordered as follows: Section 2 discusses about some related work; Section 3 about the dataset description; Section 4 provides the overview of the proposed framework; Section 5 describes the methodology; Section 6 givesexperimental result and discussion; and finally Section 7 gives the conclusions.

2. RELATED WORK

Gan Xu-Sheng et al [1] recommended a combined algorithm to increase the ability of identifying abnormality intrusions, based on Partial Least Square (PLS) feature extraction and Core Vector Machine (CVM) algorithms. By the feature extraction of PLS algorithmprincipal elements are first extracted from the data to build the feature set. Then the anomaly intrusion detection model for the feature set is built by CVM algorithm in processing large-scale sample data. PLS algorithm has roles such as dimension reduction, de-noising and multi-correlation elimination between independent variables.

Wenying Feng et al [2] proposed a machine-learning based data classification algorithm that is applied to network. The basic task is to categorize network actions (in the network log) as normal or abnormal however reducing misclassification. A new algorithm Combining Support Vector with Ant Colony (CSVAC) applied to the intrusion detection problem for generating classifiers with clustering. D. Dasgupta et al [3] proposedan approach that does notdepend on structured illustration of the data and uses only positive data to construct a normal profile of the system. It is a general approach which can be applied to various abnormality detection problems. Ming-Xiang He [4] proposed an algorithm with the significance of weighted average of attributes and a null set as initial point, a reduction results from adding the large prominence of attribute from all uncertain attribute sets progressively. If there are many of same importance of attributes, we can select any one when adding the big one from surplus conditional attribute sets.

Artificial intelligence technique are used for detecting intrusions suggested by H. Debara, et al, [5]. Explicit knowledge was necessary to build an expert system, which is not available, Artificial intelligence is used as alternate solution for treating problems. S. Mukkamala et al [6] proposed a model in neural networks as a section of an intrusion

detection system. A new approach to train support vector machines or neural networks to learn the normal behavior and attack patterns was. In that variations from normal behavior are selected as attacks. It demonstrates that both SVMs and neural networks are capable of making highly precise attack/normal categorizations. Scientists regularly use Partial Least Square (PLS)[7] for grouping and there is considerable proof to recommend that it achieves well in that role. With PLS in this way has experimental support due to the association between PLS and Canonical Correlation Analysis (CCA) and the association in turn, between CCA and Linear Discriminant Analysis (LDA). PLS is to be chosen over PCA while refinement is the goal and dimension reduction is required. This paper applies artificial bee colony for anomaly-based intrusion detectionsystems. M. Aldwairi [8]proposed a new anomaly based intrusion detection approach based on intelligent foraging behavior of bee swarm.

3. DATA SET DESCRIPTIONThe KDD99 data set, the most widely used

data set in the evaluation of anomaly detection, was selected. This data set was prepared by Lee and Stolfo et al. It was built based on the data produced from the 1998 DARPA Intrusion Detection Evaluation program. The KDD data [11] set consists of nearly 4,900,000 distinct connection vectors, each of which contains 41 features (34 continuous features and 7 discrete features). Since the data amount of KDD99 data set is too large, we chose the sample data randomly from Kddcup.data_10_percent.gz as the experiment data. The KDD training dataset consist of 10% of original dataset that is approximately 494,020 and is labeled with exact one specific attack type i.e., either normal or an attack. Deviations from ‘normal behavior’, everything that is not ‘normal’, are considered attacks. [10] Attacks labeled as normal are records with normal behavior. The simulated attack falls in one of the following four categories [9]:

1. Denial of Service Attack (DOS): In this type, the attacker makes some memory resources or computing too busy or too full to handle legitimate request, or deny legitimate user’s access. DOS contains many attacks, some of them are: 'neptune', 'back', 'smurf', 'pod', 'land', and 'teardrop'.

2. Users to Root Attack (U2R): In this type, the attacker starts out with right of entry to a normal user account on the system and is able to exploit some vulnerability to obtain root access to the system. U2R contains many attacks some of them are: 'buffer overflow', 'load module', 'rootkit' and 'perl'

3. Remote to Local Attack (R2L): In this type,the attacker sends packets over a network but who does not have an account on machine, exploits some threat



to gain local access as a user of that machine. R2L contain the attacks:'warezmaster', 'warezclient', ' multihop', ' ftp_write', 'spy', 'imap', 'guess_passwd' and 'phf'

4. Probing Attack (PROBE): In this type, the attacker try to gather information about network of computers for the apparent purpose of circumventing its security. PROBE contains the attacks: 'portsweep', 'satan', 'nmap', and 'ipsweep'. TCP, UDP, and ICMPare the protocols that are considered in KDD dataset.

4. OVER VIEW OF THE PROPOSED WORKThe principle for a better intrusion detection

system is to detect new attacks with high accuracy. High false positives and negatives, unable to handle increasing traffic rates, from which Existing system suffers. The proposed work analysis is based on the KDD CUP 99 dataset. Preprocessing is done for eliminating redundant, inconsistent data. After preprocessing dimensionality reduction is done by feature extraction method. Features are transformed by PLS feature extraction to achieve efficiency and further reduced by using Single value decomposition. The transformed features are then classified by a hybrid classifier.

5. METHODOLOGYThis paper proposes enhanced PLS feature

extraction method for dimensionality reduction. KDD CUP 99 data set has been used for experimental analysis

a) ENHANCED PLS USING SINGLE VALUE DECOMPOSITION (SVD) :

Partial Least Square feature extraction method reduces the dimensionality of the data, but processing takes greater time since large iterations are involved. In order to avoid to large number iterations Single Value Decomposition technique (SVD) is used as enhancement after PLS feature extraction technology.

Single Value Decomposition is done after extracting principal elements from PLS method, the resultant contains three matrix; from that the matrix t is taken which contains all the information of variance. TheSVD theorem states that The SVD equation for an (m × n) singular matrix A is:

A = USVT (5.1)vectors in this space are thus, also mutually independent, and thus a solution for x may be now calculated.

Anxp= Unxn Snxp VTpxp

(5.2)

where T – transpose of a matrix

Snxp -singular matrix,

Unxn - left orthogonal matrix

VTpxp – right orthogonal matrix

The steps followed in SVD process are:

1. Find AAT which gives the matrix U which is the left orthogonal matrix , such that UUT = I

2. Find ATA which gives the matrix V which is the right orthogonal matrix, such that VVT =I

3. Find the eigenvalues and eigenvectors of U and V.

4. The eigenvectors of ATA make up the columns of V, the eigenvectors of AAT make up the columns of U. Also, the singular values in S are square roots of eigenvalues from AAT or ATA.

5. The singular values are the diagonal entries of the S matrix and are arranged in descending order.



Algorithm 1: Partial Least Square Feature Extraction Technique

b) CORE VECTOR MACHINE (CVM) WITH PARTICLE SWARM OPTIMIZATION (PSO)

The proposed CVM-PSO system for classification, initially aims at optimizing the accuracy of CVM classifier by detecting the subset of best informative features and estimating the best values for regularization of kernel parameters for CVM model. In order to achieve this PSO based optimized framework is used.

Particle swarm optimization is used in CVM in order to choose the boundary value which will give accurate results. CVM is used to avoid QP problem in SVM, which is modified as MBE. PSO is used to optimize the radius of the ball forming efficient clustering which will cover allpoints. The number of iteration specifies the split of dataset and number of particles denote the particles taken for calculation.

Input: n instances and each has 41 features.

Method:

1. Divide independent (p) and dependent (q) features using their correlation dependency value

2. Calculate a linear combination of independent (t1) and dependent features (u1) which contains the

independent and dependent column as matrix.

for X = i…,.n

calculate max variance of t1

for Y = j…,.n

calculate max variance u1

for X = k…,.n Y = l…n

calculate correlation

for X = p…,.n Y = o…n

calculate covariance

endfor

endfor

endfor

endfor

if m< A

Calculate principle element for each feature in X, Y

Endif



6. EXPERIMENTAL RESULTSDISSCUSSION

Four training sets, named PROBE, U2R, R2L, DOS combined with 10% normal data are selected from Kddcup.data_10_percent.gz, are constructed as well as their corresponding four test sets with random samples. The sample data are preprocessed for removing redundancy and inconsistent data. After that enhanced Partial Least Square (PLS) using Single Value Decomposition (SVD) method is used for feature extraction. The number of attributes or features selected after applying enhanced PLS feature extraction using SVD method are illustrated in Table 1.

Table 1. Number features present after applying Enhanced PLS – SVD feature extraction

Data subsets based on category of

attacks

Features selected in

PLS method

Features selected in PLS-SVD

method

DOS+10%normal 29 19

PROBE+10%normal 29 23

R2L+10%normal 26 24

U2R+10%normal 33 21

The performance is evaluated with classifiers accordingly, using performance metrics like

1. Accuracy – Calculated in percentage2. Precision – Calculated in percentage3. Recall – Calculated in percentage4. Detection rate – Calculated in

percentage

5. False alarm rate – Calculated in percentage

6. Execution rate – Calculated in milliseconds

(A) –CVM_PLS

(B) –CVM _PLS-SVD

(C) –CVM-PSO_PLS

(D) –CVM-PSO_PLS-SVD

(E) –SVM-PSO_PLS

(F) –SVM-PSO_PLS-SVD

The performance is evaluated with classifiers accordingly, as after feature extraction and before feature extraction. The performance is found to be good in CVM-PSO_PLS-SVD classifier.

Enhanced PLS Feature Extraction using SVD with Hybrid classifier

1. INPUT : Dataset with n features2. OUTPUT: Significant features3. BEGIN– Divide the features as dependent and independent (X, Y)– Compute covariance by equation (1). – Extract Principal component elements (PCA) based on Eigen vector.– Apply Neutralization.– Find transpose of the matrix obtained A– Find U, V, where U=AAT,V=ATA– Find S– Specify Number of particles and iterations and apply CVM-PSO– Evaluate their performance4. END


ISSN: 2278 – 7798 All Rights Reserved © 2014 IJSETR

Accuracy of CVM_PSO is for all data sets.

Precision of PLS_SVD –CVM and PLS_SVDdatasets except PLS_SVD-be better.

Recall PLS_SVD-CVM_PSO is found to be comparatively lesser than other methods for allU2L+10% normal.

020406080

100120

ACCU

RACY

INPE

RCEN

TAG

E

020406080

100120

PREC

ISIO

NIN

PERC

ENTA

GE

020406080

100120

RECA

LLIN

PERC

ENTA

GE

020406080

100120

DET

ECTI

ON

RATE

INPE

RCEN

TAG

E


7798 All Rights Reserved © 2014 IJSETR

Figure 1 Comparison of Accuracy

Figure 2 Comparison of Precision

PLS_SVD-CVM_PSO is found to be higher for Dos+10%normal, for all other

Figure 3 Comparison of Recall

CVM_PSO is found to be comparatively lesser than other methods for all

DATASET

ACCURACY

(A)(B)(C)(D)(E)(F)

DATASET

PRECISION

(A)(B)(C)(D)(E)(F)

DATASET

RECALL

(A)(B)(C)(D)(E)(F)

DATASET

DETECTION RATE

(A)(B)(C)(D)(E)(F)


1660

PLS_SVD –found to be better

+10%normal, for all other CVM-PLS and SVM_PSOfound to

CVM_PSO is found to be comparatively lesser than other methods for all datasets except


ISSN: 2278 – 7798 All Rights Reserved © 2014 IJSETR

Figure 4

Detection rate of PLS_SVD-CVM_PSO is found to be comparatively higher than other methods for all datasets.

Figure 5

False alarm rate of PLS_SVD-CVM_PSO is found to be comparatively lesser than other methods for all datasets.

Execution time after reducing the datasets by SVD method f

7. CONCLUSIONIn order to solve the problem of anomaly intrusion detection, a combined intrusion detection algorithm was proposed based on PLS algorithm and classifiers. PLS algorithm has roles in dimension reduction, de-noising and multielimination between independent variables. Performance comparison of data subsets with PLSfeature extraction and after applying enhanced (PLS-SVD) method are analyzed. PLSreduces data efficiently which leads to higher accuracy while comparing with PLScomparison is done for datasets with different classifiers SVM-PSO, SVM, CVM and proposed hybrid classifier CVM-PSO.Among them CVM-PSO are found to be more efficient. with PS-SVD provides a significant improvement on

01020304050

FALS

EAL

ARM

RATE

INPE

RCEN

TAG

E

050000

100000150000200000250000

MIL

LlSE

CON

DS


7798 All Rights Reserved © 2014 IJSETR

Figure 4 Comparison of Detection rate

CVM_PSO is found to be comparatively higher than other methods for all datasets.

Figure 5 Comparison of False alarm rate

CVM_PSO is found to be comparatively lesser than other methods for all datasets.

Figure 6 Execution time

Execution time after reducing the datasets by SVD method for all datasets and classifiers found to be

In order to solve the problem of anomaly intrusion detection, a combined intrusion detection algorithm was proposed based on PLS algorithm and hybrid classifiers. PLS algorithm has roles in dimension

noising and multi-correlation elimination between independent variables.

nce comparison of data subsets with PLSenhanced PLS

method are analyzed. PLS-SVD method reduces data efficiently which leads to higher

PLS. Performance comparison is done for datasets with different

PSO, SVM, CVM and proposed .Among them CVM and

und to be more efficient. CVM-PSO improvement on

accuracy, detection rate and execution timewith other algorithms.

8. REFERENCES 1) Gan Xu-Sheng, et al, Anomaly intrusion detection

based on PLS feature extraction and core vector machine,

Knowledge-Based Systems, vol. 40, pp.1

2) Abraham, C. Grosan, C.M. Vide, Evolutionary design

of intrusion detection programs, International Journal of Network

Security, vol. 4, pp.328–333, March 3, 2007.

3) S.X. Wu, W. Banzhaf, The use of computational

intelligence in intrusion detection systems: a review, Applied Soft

Computing vol. 10, pp. 1–35, January 1, 2010.

DATASET

FALSE ALARM RATE

(A) (B)(C)(D)(E)(F)

DATASET

EXECUTION TIME

(A) (B)(C)(D)(E)(F)


1661

CVM_PSO is found to be comparatively higher than other methods for all datasets.

CVM_PSO is found to be comparatively lesser than other methods for all datasets.

and classifiers found to be lesser.

rate and execution time compared

REFERENCES Sheng, et al, Anomaly intrusion detection

based on PLS feature extraction and core vector machine,

Based Systems, vol. 40, pp.1-6, 2013.

Abraham, C. Grosan, C.M. Vide, Evolutionary design

of intrusion detection programs, International Journal of Network

333, March 3, 2007.

S.X. Wu, W. Banzhaf, The use of computational

ms: a review, Applied Soft

35, January 1, 2010.



4) Jiawei Han and Micheline Kamber, Data Mining:

Concepts and Techniques, Morgan Kufmann, 3rd edition, 2011.

5) Ahmed Youssef and Ahmed Emam, Network Intrusion

Detection using Data Mining and Network Behaviour Analysis,

International Journal of Computer Science & Information

Technology (IJCSIT), vol 3, no. 6, Dec 2011.

6) S.A.Joshi, Varsha S.Pimprale, Network Intrusion

Detection System (NIDS) based on Data Mining, International

Journal of Engineering Science and Innovative Technology

(IJESIT)vol 2,no. 1, January 2013.

7) Ming-Xiang He, A Intrusion Detection Method Based

on Neighborhood Rough Set, TELKOMNIKA, vol. 11 no. 7, pp.

3736 -3741, 2013.

8) C. J. Lin, Trust Region Newton Methods for Large-

Scale Logistic Regression, Journal of Machine Learning Research

vol. 9, pp. 627-650, 2008.

9) M. Barker, W. Rayens, Partial Least Squares for

discrimination, Journal of chemo metrics vol. 17, pp.166-173,

2003.

10) Z. M. Yang et al., Feature Selection Based on Linear

Twin Support Vector Machines, Proceeding in Computer Science

vol. 17, pp.1039 - 1046, 2013.

11) Y. J. Lee et al., Anomaly Detection via Online

Oversampling Principal Component Analysis, IEEE transactions

on knowledge and data engineering, vol.7 no.25, pp.1461-1470,

2013.

12) M. Zhu, J. Song, An Embedded Backward Feature

Selection Method for MCLP Classification Algorithm,

proceeding in computer science, vol. 17, pp. 1047-1054, 2013.

13) Abdolhossein Sarrafzadeh et al., ReliefF Based Feature

Selection In Content-Based Image Retrieval, proceedings of the

International Multiconference of engineers and Computer

Scientists 2012, vol I, IMECS 2012, March 14-16, 2012, Hong

Kong.

14) Asha Gowda Karegowda, A.S.Manjunath and

M.A.Jayaram, Comparative Study of Attribute Selection using

Gain Ratio and Correlation Based Feature Selection, International

Journal of Information Technology and Knowledge Management

July-December 2010, vol. 2, no. 2, pp. 271-27.

15) Hiep-Thuan Do, Nguyen-Khang Pham and Thanh-Nghi

Do, A Simple, Fast Support Vector Machine Algorithm for Data

Mining, Fundamental & Applied IT Research Symposium 2005.

16) I.W. Tsang et al., Simpler Core Vector Machines with

Enclosing Balls, in proceedings of the Twenty-Fourth

International conference on machine Learning (ICML), corvilis,

Oregon, USA, June 2007.

17) P. Schere, Using SVM and Clustering Algorithms in

IDS Systems, pp. 108-119, 2011.

18) M. Aldwairi, Application of artificial bee colony for

intrusion detection systems in Security and Communication

Networks Wiley Online Library, (2012).

19) Wenying Feng et al., Mining network data for intrusion

detection through combining SVMs with ant colony networks,

Future Generation Computer Systems, pp. 1-14, 2013.

20) Xuemei Li, Ming Shao, Lixing Ding, Gang Xu, Jibin

Li, Particle Swarm Optimization-based LS-SVM for Building

Cooling Load Prediction , Journal of Computers, vol 5, no. 4,

pp.614-621, Apr 2010.

21) W. Tsang, Core Vector Machines: Fast SVM Training

on Very Large Data Sets, Journal of Machine Learning Research

vol. 6, pp. 363–392, 2005.

22) Rong-En Fan et al., Working Set Selection Using

Second Order Information for Training Support Vector Machines,

Journal of Machine Learning Vol. 6, pp. 1889-1918, 2005.

23) An Efficient Classification Mechanism Using Machine

Learning Techniques For Attack Detection From Large Dataset

International Journal of Innovative Research in Science,

Engineering and Technology, vol. (1)2, pp. 230-236, 2012.

24) YasharMaali and Adel Al-Jumaily, Hierarchical

Parallel PSO-SVM Based Subject-Independent Sleep Apnea

Classification, Neural Information Processing vol. 7666, pp. 500-

507, 2012.

25) Qinghua He et al.,Classification of Electronic Nose

Data in Wound Infection Detection Based on PSO-SVM

Combined with Wavelet Transfor, Intelligent Automation, Soft

Computing vol. 18, no. 7, pp. 967-979, 2012.



SM.Kannathal is a P.G student in the Department of Computer science and engineering, Avinashilingam Institute of Home Science and Higher Education for women, Faculty of Engineering, India. Presented a research paper titled “Intrusion Detection Based on Partial Least Square Feature Extraction with Classifiers” in the International conference on Communication and Computer Networks of the future COMNET 2014 at P.S.G College of Technology, Coimbatore sponsored by Computer Society of India (CSI).

intrusion detection system based on enhanced pls feature...

Documents