Anomalous: A Cloud Based Approach to Anomaly

Detection and Visualisation

Dylan Brown Henk Joubert Justin van der Merwe

June 11, 2012

1 Project Description

Intrusion detection faces several challengeswhen scaling up to handle thousands ofnodes [14] . A cloud based solution is pro-posed to handle the vast quantities of datarequired for analysis by the Intrusion De-tection System. Integration of data collec-tion, anomaly detection, and ultimately vi-sualization of the results will aid adminis-trators in effectively identifying anomalousactivity within their systems. This combi-nation of techniques provides a novel andapplicable solution for the current trend insystem architectures.

2 Problem Statement

The problem faced is incorporating threeseparate modules, which has previouslybeen used as stand alone systems, into onesystem which can provide benefits whichare greater than the sum of its parts. Theproblem is important as these three sep-arate systems need to be able to keep intouch with modern system architecture.This system would be of greatest benefitsto clients which have widespread branchesand wish to be able to use a singular solu-tion (Software As A Service) that will scalewith data from all their branches.

2.1 Cloud Collection

Can we scale collection of securityevents from a large distributed en-vironment reporting to a centralizedconsole for analysis? There are fewcomprehensive open source Security Infor-mation and Event Management (SIEM) so-lutions that can scale to large networks.Timofte [16] notes that open source toolsare gaining popularity among all types ofusers, although it is difficult to determineusage figures hence it is an open questionas to whether such tools are good enoughfor corporate deployment.

2.2 Intrusion Detection

Which intrusion detection techniquescales best to the cloud based systemand how can it be optimised for thecloud context? Data integrity and se-curity are increasing concerns in this datacentric age we find ourselves in [6]. As aresult of the increased occurrences of unau-thorised access to sensitive data [11] intru-sion detection systems (IDS) are becom-ing increasingly important for both pre-vention and analysis of network intrusions.Various intrusion detection approaches ex-ist, ranging from statistically based ap-proaches, through knowledge based ap-proaches (whereby detection is based off a

1

set of predefined rules), to machine learn-ing based approaches [3]. In our contextwe aim to investigate the application of In-trusion Detection in a system where datais collected by the Cloud. In order to makethis as effective as possible we need to de-cide which Intrusion detection approach isbest suited to the cloud based approach andhow we can optimise it to increase its effi-ciency. The most important requirementsfor clients of our system would be the ac-curacy of the Intrusion Detection System(IDS). The IDS must provide a solution tothe clients which ensures maximum secu-rity of data whilst remaining computation-ally efficient and usable for system admin-istrators. The usability will be provided bythe visualization tool.

2.3 Visualization

To identify network intrusions, system ad-ministrators are tasked with analysing tex-tual data, often in the form of event logfiles generated by Operating Systems or In-trusion Detection Systems [10]. Such filestypically reach 50000 events daily, render-ing manual analysis infeasible [5]. Informa-tion Visualisation offers a viable approachto analysing event log data by improvinginterpretation efficiency, as well as provid-ing an informative overview of the networkactivity recorded in these log files [1].

In order to provide an effective visualisa-tion, several questions need to be asked:

Does the visualisation enable efficientexploration of large security relatedlogs files? As noted by Musa et al. in[13], this is the main objective of secu-rity event visualisation. In the prototypingphase, we plan on keeping in mind the vastamount of data inherent to collections of se-

curity events. This will allow us to addressthis question early on.

Does the visualisation scale with net-work size? Previous work has shown atrade off between scalability in terms ofnetwork size compared to detail and num-ber of queries offered on the other. We planon maintaining a balance between these as-pects.

Does the visualisation meet the re-quirements of the intended users, sys-tem administrators? As highlighted byKomlodi et al. [10], the visualisation re-quirements identified through user studiesare:

• Providing an overview of the networkin a way that allows intrusions to beidentified

• Allowing the user to zoom in on finerdetails to diagnose intrusions

• Providing response when intrusionsare identified

These requirements will serve as a basisfor the design of the Visualisation compo-nent.

3 Procedures and Methods

3.1 Implementation Details

In order to effectively manage code and col-laborate effectively Git with GitHub will beused. This is ideal as it provides a versioncontrol system which all members are fa-miliar with.

The proof of concept will be hosted onGitHub in a private repository during de-velopment and made public at the end of

2

the project. It should be modular in orderto collect data from various sources, andimplemented in a language that is crossplatform. Languages under considerationare Python and Java.

For the Visualisation component, severalVisualisation libraries are being considered.Prefuse and JUNG are the current consid-erations for a Java implementation, whileGraphViz and VTK are current considera-tions for both Java and Python.

3.2 System Design

The system design can be summarised bythe following diagram. Here the modu-lar design can easily be seen. The threemodules are the Intrusion Detection Sys-tem (IDS), the Cloud Collection systemand the Visualization. These modules to-gether form a unique system which can beof benefit in many situations. The CloudCollection module provides data from var-ious client branches to both the IDS andVisualization. This data is formatted intoa previously agreed upon standard. Thiswill allow data to be grouped accordinglyby the IDS and Visualization. The IDSwill then use the data provided by the Col-lection module to train the machine learn-ing techniques used and then detect theanomalies within the target systems. Thisanomaly data will then be combined withthe Collected data and visualized by theVisualization module in a manner which ishelpful to system administrators.

3.2.1 Cloud Collection

A 3-layer hierarchical approach ( recom-mended by Grzech [9]) will be taken withregard to collecting IDS data :

Figure 1: System design diagram and workallocation

The sensor layer where all data collec-tion occurs. This is represented by theleaves in the tree structure.

The filter layer where the sensor datais collected from the local network and ag-gregated to reduce the amount of networktraffic generated by the system.

The reporting layer , effectively theroot of the tree, where aggregate data isarchived and analyzed. This will be im-plemented as an interface that allows theIntrusion Detection and Visualization com-ponents to query this data store.

A prototype will collect and normalizedata from Snort and Linux system log files.If this is task is implemented quickly thenadditional sources can be integrated. Thereporting layer will be accessed througha web interface hosted on Amazon WebServices for elastic computation and stor-age requirements. There are two optionsfor implementing the database using AWS:Amazon RDS for a relational databasestore, and Amazon SimpleDB as a nosqlalternative. Both options provide low la-tency when interfacing with EC2 instancesin the same region.

3

3.2.2 Intrusion Detection

Of the various IDS approaches anomaly de-tection is the preferred approach becausein the outsourced cloud computing contextthe advantages of reduced manual devel-opment, protection against unseen attacksand a large dataset from which to train theanomaly detection system outweigh the dis-advantages of unseen legitimate behaviourand high false alarm rates. Anomaly detec-tion can also be subdivided into it’s varioustechniques, as such the problem of choosingthe right technique is important. For thiscontext neural networks are a good fit dueto the fact that neural networks will pro-vide a supervised approach which shouldscale well.

The major drawback lies in the factthat it is sometimes difficult to trackwhy a particular detection decision hasbeen made. This drawback is nullifiedby the combination of the IDS withthe visualisation component which helpssystem administrators sort through thefalse alarms. Neural networks do howeverprovide defence against unseen attackswhich is important in this context.

A prototype for the IDS would include abasic implementation which can train itselfbased off a limited data set. This will thenbe expanded to cope with the vast quan-tities of data provided by the cloud collec-tion system. In testing the first compari-son must be done in terms of the IDS falsealarm rate compared to existing systemsfalse alarm rate. Analysis must be thenperformed in terms of how well it scaleswith increasing data sizes. This combina-tion of metrics is important in determiningif the system can be said to offer a ben-efit when used in the cloud based context.From these metrics it can be decided if neu-

ral networks are the correct approach or ifanother approach should be tested.

The challenges that arise from the cre-ation of this prototype are likely to be re-lated to the implementation challenges ofneural networks. These will range from theinherent difficulty in applying neural net-works to the IDS context, to the challengeof designing a system that can scale withthe expected large data in future iterations.

3.2.3 Visualisation

The Visualisation component will serve asa visual and interactive interface to the nor-malized data retrieved from the Cloud Col-lection component, overlaid with the out-put retrieved from the Intrusion Detectioncomponent.

As discussed by Livnat et al. in [12], eachsecurity event recorded should contain theevent type, time of occurrence, and networknode where the event occurred. In order tocorrelate the multitude of seemingly unre-lated events and event types, we will fo-cus on displaying the connections betweenthese three fields of each event. For ex-ample, suspicious activity of different eventtypes may exhibit a pattern if the eventsoccurred at the same network node, andat a similar time. In this case, we wouldaim at illustrating the possible connectionsbetween the two event types. Focusing onthese three fields may simplify aggregationof data in the collection component.

As much of the user interaction wouldtake place in this part of the system, usercentered design is key to its success. Amock-up prototype focusing on the visual-isation’s design will be produced, followedby a functional prototype. The design pro-totype will use the security visualisationrequirements highlighted in the problemstatement as a basis for the design, with

4

a particular focus on the network overviewrequirement. The scalability of the designto suit networks of varying sizes will be con-sidered in this phase. Furthermore, the fol-lowing visualisation techniques and princi-ples discussed by Card et al. in [1] will betaken into account:

• Visual Information Seeking mantra:Providing the user with an overview ofthe data, allowing the user to zoom inon items of interest and filter out un-interesting data, and allowing the userto select an item to provide them withfiner details

• Focus + Context: Providing the userwith both an overview (context) anddetailed information (focus) simulta-neously

This prototype will be presented to theproject supervisors, Andrew Hutchinsonand Michelle Kuttel. The expertise ofeach supervisor will provide us with knowl-edgeable feedback on both the visualisa-tion’s design, and its use in network secu-rity. The feedback received will be takeninto account, and our design altered appro-priately. Several complications may arisewhen implementing the design. A possi-ble issue is the scalability of the design,which may only be noticeable when usedwith real data. By considering these chal-lenges early in the design phase, we hope toavoid the likelihood of redesigning the visu-alisation late into the project. If certain is-sues are found to be unavoidable, compen-sations will considered. For example, thevisualisation may need to resort to panningand zooming in order to fit larger networksinto the same space, although this will beavoided.

The functional prototype will follow thedesign phase and use the altered design as

a basis for implementation. A horizontalprototyping approach will be followed, fo-cusing more on user interaction than lowerlevel system functionality. This prototypewill then be tested using a network admin-istrator, providing us with an indication ofhow effectively the design addresses the re-quirements the intended end users.

Once implemented, the visualisation sys-tem will be evaluated using non-expertusers with a general understanding of net-working. Possible candidates include com-puter science students. Although theseusers will not have the technical knowledgeand experience of network administrators,they will nevertheless provide us with anunderstanding of how effective and intu-itive the visualisation and its interface are.The advantage to utilising non-expert usersis the accessibility and availability of suchcandidates. We will thus be able to ob-tain an adequate number of willing candi-dates. Several factors will be consideredduring these tests:

• Does the visualisation allow the userto zoom in on details of interest?

• Does the visualisation allow the userto filter out information on demand?

• How effectively was the user able tofocus on finer details, whilst keepingall the network’s activity in context?

• Is response provided when the userrecognises suspicious activity?

To evaluate these factors, each user willfirst be asked to interact with the system,and notes taken on their usage of the sys-tem and navigation around the visualisa-tion’s interface. Subsequently, each userwill be interviewed to provide insight intothe factors highlighted above.

5

In addition to user testing, the scalabilityour visualisation’s will be measured. Oursystem will be tested on varying sizes of col-lected event data from the Cloud Collectioncomponent. This will provide us with anindication of our system’s scalability withrespect to the number of events visualised.To determine the system’s ability to handlelarge network sizes, our visualisation will betested using as large an amount of networknodes as provided by the Cloud Collectioncomponent. Our system’s scalability willcompared to that of existing systems. Thefactors to be considered in this comparisonare as follows:

• How many network nodes can be dis-played in the visualisation’s overviewof the network

• How many events can be displayed inthe visualisation’s overview of the net-work

• In how much detail can each node bedisplayed in the overview of the net-work

• How much detail can be displayedabout each node once zoomed in or fo-cused on

4 Ethical, Professional andLegal Issues

For user testing of the visualisation system,we will require ethical clearance in order toperform usability tests with both a systemadministrator in the prototype phase andnon-expert users in the testing phase.

The software artefacts developed for thisproject will be released under the BSD li-cence due to its permissiveness.

5 Related Work

5.1 Cloud Collection

Debar et al. [4] discuss the IDMEF (In-trusion Detection Message Exchange For-mat [2] ) used for communication and inter-operability between different security ven-dors and Timofte mentions the adoptionthereof by several open source tools, includ-ing snort.

5.2 Intrusion Detection

Whilst there has been much work donewithin the field of Intrusion Detection, thisfield is lagging behind in its application tomodern cloud based architecture. Debar etal [3] provide a taxonomy of intrusion de-tection systems which highlights the vari-ous strengths and weaknesses of the intru-sion detection approaches. Garcia-Teodoroet al [6] focus on the anomaly based ap-proach to intrusion detection. Garcia-Teodoro et al [6] show that the weakness insome approaches stems from computationaldifficulties encountered with attempting todesign a system which can cope with un-seen anomalous behaviour.

5.3 Visualisation

Although still a relatively young area of re-search, security visualisation has seen var-ied approaches. Work in the area hasconcentrated on visualising either packettraces, network flows, or event data [8].Raw packets, are the lowest level of net-work data, and are useful understandingnetwork behaviour. However, packet databecomes too large to be stored systemically.Network flows are aggregations of packettraces allowing them to be stored on a sys-tematic basis for later analysis [8]. Eventdata describes network activity at a higher

6

level, and are usually recorded in log filesgenerated by operating systems and intru-sion detection systems [15]. The Visualisa-tion component of the project lends itselfto the event data classification, as this isthe data made available by the Cloud Col-lection component.

Two notable visualisation systems show-ing advances in the area are VisAlert [12]and TNV [10]. Livnat et al. state thateach security event recorded should containwhat (the event type), when (the time ofoccurrence), and where (the network nodewhere the event occurred). Their systemdrew correlations between these three fieldsas connections between two domains (theinner circle domain and the outer ring do-main in Figure 2).

Figure 2: The VisAlert visualisation sys-tem

A distinguishing feature of TNV, a sys-tem proposed by Komlodi et al. in [10],is its application of the focus + contextinformation visualisation technique. Thistechnique aims to provide the user withboth an overview (context) and detailed in-

formation (focus) simultaneously [1]. Thecenter of the main view represents the fo-cused time intervals with wider columns,while the left and right sides (representingearlier and later time intervals respectively)represents the context [7]. This techniqueallows TNV to better cope with even num-bers of scale without sacrificing detail, twoaspects common to recorded security data.

Figure 3: The TNV visualisation system

6 Anticipated Outcomes

6.1 Cloud Collection

The system aims to present a prototype fora highly scalable SIEM solution. The de-sign should be modular and allow for easyintegration with existing IDS technology.The prototype will be evaluated on the abil-ity to handle a large number of sensors in areliable manner while maintaining the qual-ity of the resulting information.

7

6.2 Intrusion Detection

The IDS is expected to provide a new ap-plication of IDS which is both well suitedand useful in todays modern system archi-tecture. The result needs to scale well withlarge data. The question is how well willthis scale in terms of detection and falsealarm rate? The expected results are thatcertain machine learning techniques, suchas neural networks, will scale well both interms of false alarm rate and detection rate.The expected effect is a shift in thinkingin terms of scaling solutions for ID to thecloud and specifically a solution which willprovide a solution which could be used toeffectively detect anomalies from the cloud.

6.3 Visualization

The Visualisation component hopes to con-tribute to the young field of security visual-isation. With the exception of works suchas TNV [7], current work in the field doesnot maintain a strong focus on user cen-tered design. As the system is ultimatelyintended for use by system administrators,we plan on differentiating from much of thecurrent work by taking into account thevisualisation requirements of these users.Furthermore, the integration of the CloudCollection and Intrusion Detection compo-nents will allow the visualisation to explorea multitude of events and alerts. This pro-vides us with the opportunity to display amore comprehensive overview of the net-work alerts in an organisation.

As the visualisation component serves asthe system’s interface to end users, the ef-fectiveness of this component is importantto the success of the system. The mainconcern for the visualisation component’ssuccess is how well the user requirementsfor a security visualisation system are met.

Hence, the user testing planned and high-lighted in the System Design section willserve as an indicator of the success of theVisualisation component.

7 Project Plan

7.1 Work Allocation

• Henk Joubert - Cloud Collection

• Dylan Brown - Intrusion Detection

• Justin van der Merwe - Visualisation

7.2 Risks

7.2.1 Group member departure

Risk : LowSeverity : LowEven though the Visualization andAnomaly Detection components rely ondata from the Collection module, the im-pact of this happening has been mitigatedby the acquisition of anonamized datacollected by a masters student.

7.2.2 Neural networks do not scalewell with the large data

Risk : LowSeverity : MediumIf neural networks do not scale well then asimpler approach can be used or the datacan be scaled down.

7.2.3 Neural networks prove too dif-ficult to implement

Risk : MediumSeverity : MediumNeural networks are a difficult conceptwhich the group members have no experi-ence in. As such the IDS can be simplifiedto another technique which is simpler in

8

concept (such as specification detection, ora simpler machine learning technique, suchas clustering).

7.2.4 Data needed to test scalabilityof IDS is not available

Risk : MediumSeverity : LowIf the data required to test scalability is notavailable due to the cloud collection sys-tem not being completed or available thenthe already provided data from the Mastersstudent can be used to do limited scalabil-ity testing.

7.2.5 Output data from IDS cannotbe properly integrated intothe Visualisation component

Risk : LowSeverity : MediumIf the IDS output data cannot be inte-grated, the Visualisation component caninstead display output based solely on thecollected security events. Additionally, theIDS component will be able to display itsoutput in an alternative readable format.

7.2.6 Too many queries are requiredto display the collected data inthe Visualisation

Risk : HighSeverity : LowIf too many queries are inherent to the datacollected, filtering and zooming techniquescan be applied to show varying levels of de-tail according to the user’s situational in-terests

7.2.7 The visualisation cannot sup-port the network size

Risk : MediumSeverity : MediumThis can be mitigated by applying a focus+ context approach, where unfocused net-work nodes can still be visualised, at unfo-cused or lower levels of detail. If it cannotbe avoided, panning and zooming function-ality will be applied to visualise the entirenetwork.

7.2.8 Loss of data

Risk : LowSeverity : Medium – CatastrophicThe loss of source code or other assets canoften be the death of any project. This ismitigated by redundant copies of data onseveral machines, cloud storage and versioncontrol, hence the time lost should neverexceed a few hours of work.

7.2.9 Not enough resources to testcollection scalability

Risk : MediumSeverity : MediumData can be faked, or duplicated to stressthe system similar to large scale experi-ment.

7.3 Resources Required

High performance computing resources willbe required to test the scalability and fea-sibility of the IDS for the cloud based sys-tem.For user testing of the visualizationsystem administrators will be required totest usability and give feedback on the sys-tem’s benefits.

9

7.4 Deliverables

Project Proposal 21/05Project Website 12/06Prototype 25/08Final Implementation 21/09Experimentation results 03/10Draft of Report 24/10Final report Due 31/10Poster Due 03/11Final Project Website 07/11Individual Reflection 11/11

7.5 Milestones

Proposal 10/06Presentation 24/05Design chapter complete 25/07

Prototype 1 complete 20/07Henk: Collection and storageDylan: Training phaseJustin: Design prototype

Prototype 1 testing complete 25/07

Prototype 2 complete 25/08Henk: AggregationDylan: Detection phaseJustin: Functional prototype

Prototype 2 experiments due 30/08

Implementation complete 21/09Henk: OptimisationDylan: OptimisationJustin: Component Integration

Implementation experiments due 23/09Implementation chapter complete 03/10Final report due 31/10Poster due 03/11Project demonstration 05/11Project website due 07/11Individual reflection due 11/11Final project presentation 14/11

7.6 Timeline

Refer to figure 7.6 for the Gantt chart.

10

Figure 4: Timeline

11

References

[1] Card, S. K., Mackinlay, J. D.,and Shneiderman, B., Eds. Read-ings in information visualization: us-ing vision to think. Morgan KaufmannPublishers Inc., San Francisco, CA,USA, 1999.

[2] Debar, H., Curry, D., and Fe-instein, B. The intrusion detectionmessage exchange format. Tech. rep.,IETF, 3 2007.

[3] Debar, H., Dacier, M., and We-spi, A. Towards a taxonomy ofintrusion-detection systems. Com-puter Networks 31, 8 (1999), 805 – 822.

[4] Debar, H., and Wespi, A. Ag-gregation and correlation of intrusion-detection alerts. In Proceedings of the4th International Symposium on Re-cent Advances in Intrusion Detection(London, UK, UK, 2001), RAID ’00,Springer-Verlag, pp. 85–103.

[5] Erbacher, R., Walker, K., andFrincke, D. Intrusion and mis-use detection in large-scale systems.Computer Graphics and Applications,IEEE 22, 1 (jan/feb 2002), 38 –47.

[6] Garcıa-Teodoro, P., Dıaz-Verdejo, J., Macia-Fernandez,G., and Vazquez, E. Anomaly-based network intrusion detection:Techniques, systems and challenges.Computers and Security 28, 1–2(2009), 18 – 28.

[7] Goodall, J., Lutters, W., Rhein-gans, P., and Komlodi, A. Pre-serving the big picture: visual net-work traffic analysis with tnv. InVisualization for Computer Security,

2005. (VizSEC 05). IEEE Workshopon (oct. 2005), pp. 47 – 54.

[8] Goodall, J. R. Introduction to vi-sualization for computer security. InVizSEC 2007, J. R. Goodall, G. Conti,and K.-L. Ma, Eds., Mathematics andVisualization. Springer Berlin Heidel-berg, 2008, pp. 1–17.

[9] Grzech, A. P. Optimal monitoringsystem for a distributed intrusion de-tection system.(report). Artificial Lifeand Robotics 14, 3 (2009), 453.

[10] Komlodi, A., Goodall, J. R., andLutters, W. G. An information vi-sualization framework for intrusion de-tection. In CHI ’04 extended abstractson Human factors in computing sys-tems (New York, NY, USA, 2004),CHI EA ’04, ACM, pp. 1743–.

[11] Kumar, S., and Spafford, E. Apattern matching model for misuse in-trusion detection. In In Proceedings ofthe 17th National Computer SecurityConference (1994).

[12] Livnat, Y., Agutter, J., Moon,S., Erbacher, R., and Foresti,S. A visualization paradigm for net-work intrusion detection. In Informa-tion Assurance Workshop, 2005. IAW’05. Proceedings from the Sixth AnnualIEEE SMC (june 2005), pp. 92 – 99.

[13] Musa, S., and Parish, D. J. Visu-alising communication network secu-rity attacks. In Proceedings of the 11thInternational Conference InformationVisualization (Washington, DC, USA,2007), IV ’07, IEEE Computer Soci-ety, pp. 726–733.

12

[14] Shaikh, S. A., Chivers, H., No-bles, P., Clark, J. A., and Chen,H. Towards scalable intrusion detec-tion. Network Security 2009, 6 (2009),12 – 16.

[15] Shiravi, H., Shiravi, A., andGhorbani, A. A. A survey of visu-alization systems for network security.IEEE Transactions on Visualizationand Computer Graphics 99, PrePrints(2011).

[16] Timofte, J. Intrusion detection us-ing open source tools. InformaticaEconomica Journal XII, 2 (2008), 75–80.

13