an overview of the parallax battlemind v1.5 for computer network

An Overview of the Parallax BattleMind v1.5

for Computer Network Defence

Duncan Grove Alex Murray Damien Gerhardy Benjamin Turnbull

Troy Tobin Christopher Moir

Defence Science Technology OrganisationPO Box 1500, Edinburgh, South Australia 5111

Email: {alex.murray, damien.gerhardy, benjamin.turnbull, troy.tobin,christopher.moir}@dsto.defence.gov.au

Abstract

BattleMind (BM) version 1.5 is the first of a series ofArtificial Intelligence systems for semi-automaticallyunderstanding, planning and conducting ComputerNetwork Defence. It makes use of a wide range ofexisting techniques including classification and fea-ture extraction, semantic web technologies, data fu-sion, ontologies, first order predicate logic based for-ward and backward chained reasoning, hierarchicaltask network planning and supervised learning. Novelcontributions of our work compared to other AI basedCND tools are: (1) explicitly modelling people andorganisations as well as computers and networks aspart of the overall system, and elements of the busi-ness processes that link them; and (2) using a broadrange of high level data sources rather than just tra-ditional low level data sources such as packet capture.

1 Introduction

Parallax is a high intensity research project being con-ducted by DSTO to research methods to rapidly ad-vance the state of the art for Cyber Security utilising avariety of approaches. As part of this project, a seriesof BattleMind (BM) prototypes are being constructed.This paper provides an overview of BM version 1.5,a prototype system making heavy use of a variety ofArtificial Intelligence (AI) based techniques to explorenew methods for Computer Network Defence (CND).Classic CND systems, on the other hand, are typicallysignature and/or anomaly based [8].

Unlike other CND systems which do use AI basedtechniques, BM considers more than just the com-puter and the network. While computers and net-works are an integral part of BM’s world view, it ex-ploits the profusion of electronic data sources thathave become available in recent times to explicitlymodel these plus people, organisations, and the busi-ness processes that link them; and it does so forboth Cyberspace and parts of the physical world withwhich it intermingles. This provides an approach for asystematic framework for identifying complex vulner-abilities that involve the manifold interactions of a di-verse range of domains including hardware/software,social networks, logistics, supply chains and more.

Copyright c©2013, Commonwealth of Australia. This pa-per appeared at the 11th Australasian Information SecurityConference (AISC 2013), Adelaide, South Australia, January-February 2013. Conferences in Research and Practice in In-formation Technology (CRPIT), Vol. 138, Clark Thomborsonand Udaya Parampalli, Ed. Reproduction for academic, not-for-profit purposes permitted provided this text is included.

2 Architecture

The high level architecture for BM is shown in Fig-ure 1.

Specialised Data Collection agents are envisagedto be distributed throughout the world’s informationprocessing systems. Although they may vary by thedata they operate on – being simple or highly intel-ligent, cost-effective or high-performance – all havethree things in common. First, they are placed wherethey have access to data. Second, they extract basicfacts from these data sources into longitudinal groupsof semi-structured observations, which we call glims.Each glim summarises the extracted information us-ing its native terminology and semantics but in a stan-dard syntax. Finally, each agent transmits these glimsto the Glim Bridge for ingestion further into the sys-tem.

The Glim Bridge converts glims into a series of(subject, predicate, object) tuples and sendsthem to a triple store formatted Knowledge Base(KB) for further processing. It translates each scopedrange of a Glim’s hierarchical structure into a sepa-rate named object, then uses the hierarchy and ex-plicit linkage information to encode the relationshipsbetween those objects. For leaf nodes utilising nativeJSON types like strings and numbers, plus a smallnumber of extension types like dates and times, GlimBridge is also responsible for converting these to acanonical form that is compatible with the KB.

Glim Space is a staging area for information com-ing into the KB. It acts as a short term memory forraw observations, before later parts of the system havebeen able to digest and incorporate them into thesystem’s World Model. Although the information issemantically equivalent to the observations from theData Collectors, it has by now been syntactically reg-ularised, allowing further processing to focus on thecontent and ignore its format.

The World Model is a digital representation of thereal world, currently including things like people, or-ganisational structure, computers and networks, andbusiness processes like project tasking, procurementand travel – plus the rich linkages between all of theseelements. This knowledge is reconstructed over timeby specialised data fusion components that transformthe information entering Glim Space into a more se-mantically consistent form.

The World Model makes Reasoners and Plannersrelatively more simple to implement and computa-tionally efficient. An intent is that many questionsabout the current state of the world can be answeredby simple lookup since the answers have already beenforward chained into the World Model. Answeringmore complex questions that require meta-level anal-yses and planning across multiple World Model struc-

Proceedings of the Eleventh Australasian Information Security Conference (AISC 2013), Adelaide, Australia

31

Figure 1: BM High Level Architecture

tures may also be simplified since inferencing can fo-cus on the higher level concepts rather than having torepeatedly backwards chain knowledge from unstruc-tured facts.

The User Interface provides a mechanism for theuser to: ask questions and receive answers about thestate of the world in order to facilitate human under-standing; and generate plans for future actions that auser could carry out to achieve the stated goal. Giventhe complex and subtle nature of cyber defence, webelieve that the “working” of BM’s reasoning andplanning engines must be presented to the user, notjust the final answers. Rather than present the userwith overwhelming detail, however, our User Inter-face factors out repetitive chains of logic so the usercan focus on classes of deductions in general, not justspecific cases. This allows the user to steer BM to-wards good answers, and also provides scope for thesystem to apply supervised Machine Learning (ML)to automatically find the better answers more quickly.

3 Prototype System

3.1 Data Collectors, Glims and the GlimBridge

Data Collection Agents gather information from awide variety of different document types. For exam-ple, our Classifier and Feature EXtraction (CFEX)agent uses Naive Bayes to classify incoming docu-ments and then applies regular expressions to ex-tract salient information; our Computer System In-formation (CSI) agent extracts hardware, software,operating system, network and user information di-rectly from a Windows or Linux computer; our Hard-ware Ontology Scraper (HOS) agent spiders computercomponent manufacturer and supplier websites fordetails about various types of computer components;and our Network Analysis agent extracts output fromexisting tools such as nmap [7] and tshark [1].

JSON Formatted Glims encode the output ofData Collection agents. They use a number of con-ventions for naming and referencing a given objectand informing Glim Bridge about the type of datacontained within an element. A small but internally

consistent part of a glim produced by CSI is shownin Listing 2. While the original glim contained over18,000 lines (including a detailed enumeration and de-scription of the hardware, software and users of thecomputer), only 43 lines relevant to examples pre-sented in the paper are included.

Glim Bridge accepts incoming JSON glims asHTTP POST requests from Data Collection agents.It is responsible for processing each glim and sendingthe contents to the KB. In processing a JSON glim,named objects are merged and typed values areprocessed into the World Model’s fixed ontology.

3.2 Converting Glims into Knowledge

Each data fusion component intimately understandsthe semantics of the incoming glim, intimately un-derstands the semantics of related parts of the WorldModel, and can extract relevant information fromGlim Space and update the World Model accordingly.Key parts of solving the third part of this probleminclude: entity extraction or business process iden-tification so the relevant World Model structure canbe updated; conflict resolution to deal with incominginformation that contradicts the World Model; modelgeneration to create knowledge from partial observa-tions combined with existing knowledge; and onto-logical bridging from dynamic Glim Space conceptsto the fixed World Model ontology.

Entity Matching of objects in a glim to enti-ties in the World Model is based upon their typeand attributes. Since we used Cyc [5] to implementBM v1.5’s KB (see Section 3.3) we used its #$isapredicate to match type, which indicates that an ob-ject is an instance of a particular (#$Collection).The other attributes of an object (for example#$firstName #$surname and #$dateOfBirth) arethen used to match against existing entities with thesame or similar attributes. As explained in Section 2,the attribute names within a glim are dependent onthe original information source (and the agent whichcreates the glim). In the example being considered,the agent chose to encode the surname of each per-son using the label surname. However, in the stan-dard Cyc ontology this concept is represented by the

CRPIT Volume 138 - Information Security 2013

32

1 {

2 "glim" : "CSI -AGENT -44 B960A70B71C827 -3 D4F7655C198192" ,

3 "isa" : "#$PxCSIAgentOutput" ,

4 "pxhasCSIReport" :

5 {

6 "_name" : "CSIReport -460 C6FA4E724FECD -EF46F9B00ECC695" ,

7 "isa" : [ "#$PxCSIReport" , "#$InformationBearingThing" ],

8 "pxhasIdentifiers" :

9 {

10 "_name" : "Identifiers -4 AF761F84F7913CA -DA7A80E0AE20D882" ,

11 "isa" : "#$PxIdentifiers" ,

12 "pxIdentifierHostname" : "apm -vm.dsto.defence.gov.au"

13 },

14 "pxhasSoftwareList" :

15 {

16 "_name" : "SoftwareList -4 D098670311419FC -F8506F38AC780FB9" ,

17 "pxWin32_Product" :

18 [

19 {

20 "_name" : "Win32_Product -44 C17DEA168B6B77 -EC0FCC1658F2C7A0" ,

21 "isa" : "#$Win32_Product" ,

22 "pxrsiname" : "Adobe Reader 9.3.2",

23 "pxrsivendor" : "Adobe Systems Incorporated" ,

24 "pxrsiversion" : "9.3.2"

25 }

26 ]

27 },

28 "pxhasUser" :

29 {

30 "_name" : "UserList -4204 A39396907FFF -4921 D3B1A46C60BF" ,

31 "pxWin32_Account" :

32 [

33 {

34 "_name" : "Win32_NetworkLoginProfile -449944 D55DE64CBA -19 FE7B65CEA7AEBE" ,

35 "isa" : "#$Win32_NetworkLoginProfile" ,

36 "pxrsicaption" : "MurrayA" ,

37 "pxrsifullname" : "Murray , Alex" ,

38 "pxrsiname" : "DSTO\\murraya" ,

39 }

40 ]

41 }

42 }

43 }

Figure 2: A small part of a CSI glim showing software and user information.

#$familyName predicate. When matching entitiesthese discrepancies between the ontology used by thevarious agents and the standardised World Model on-tology must be reconciled. We solved this problemwith a mixture of two approaches: manual ontologybridging by analysts with expert knowledge; and anautomatic comparison of a glim’s entities against theattributes of all similarly typed entities in the WorldModel, searching for minimum Levenshtein edit dis-tance [6].

Entity Merging of the matched entities then pro-ceeds to unify new knowledge from the glim with theexisting entity in the KB. In general this is a complexprocedure since there may be conflicting assertionsthat need to be resolved, however the data sources weused were generally consistent and so in most caseswe were able to simply add attributes from the glimentity to the World Model entity. In Cyc this wasachieved by making assertions using the same subjectand predicate as the glim entity statements but tar-geting the World Model entity to simply “copy over”those assertions. For the few cases where glim in-formation conflicted with existing information in theWorld Model, we instead applied rules that attemptedto select the most authoritative statement. For sim-plicity we chose to assume the most recent observa-tion was the most authoritative, but rules for makingmore intelligent decisions can be envisaged.

Business Process Modelling is then performedusing the information from a wide variety of ingesteddocuments, allowing BM to model and track the life-cycle of various business processes based upon theirelectronic “paper trail”. BM currently treats businessprocesses in terms of sequences of actions and branch-ing alternatives, while also taking some account ofthe relationships between interlinked processes. Thisintroduces a predictive capability that makes subse-quent reasoning about the future substantially moretractable and accurate. Although machine learn-ing techniques could perhaps automatically constructsuch models, BM currently relies on humans to anal-yse and map out the important relationships between

various states and events. This effort is minimised,however, since core business processes change slowlyover time.

3.3 Cyc Knowledge Base and World ModelOntology

Through the glim processing chain, BM constructs arichly interconnected, structured KB representing thecurrent state of the world. BM’s current scope coverscomputers, networks, people and business processes,so the BM ontology provides a way to describe theseaspects of the world. We used Cyc [5] to implementBM v1.5’s KB and World Model Ontology because itcame pre-populated with a large general knowledgeontology and a substantial number of forward andbackward chained reasoning modules; and for testingits suitability for later versions of BM.

In order to encode the BM World Model, we devel-oped Cyc ontologies for: people, as individuals or aspart of an organisational hierarchy; and for comput-ers and networks, both as physical items constructedfrom components and also as logical entities withusers running software. While Cyc’s existing generalpurpose ontologies were sufficient for modelling someideas, we found the existing ontology to be outdated,incomplete or incongruous for specifying many con-cepts of interest to us. We therefore expanded Cyc’sontology with over 1,400 concepts and 730 predicatesfor use by our World Model.

3.4 Planning

Inspired by SHOP [11], the BM v1.5 Planner usesHierarchical Task Networks (HTN) to decompose ac-tions into progressively more specific subactions. Al-though the planning engine is domain-independent,templates for solving particular problems rely onexpertly encoded domain knowledge using HTNs.These are composed from complex (i.e. decompos-able) actions and simple (i.e. atomic) actions at the


33

leaf nodes that actually query or modify the WorldModel.

Actions are encoded using our custom ActionDefinition Language exemplified in Figures 3aand 3b. Figure 3a shows a simple action foridentifying the author of a DSTO publication,which it achieves by querying the World Modelusing the search-condition. This triggers a Cycquery whose results are used to populate variablesin the effect-name, effect and state-bindingdefinitions. Figure 3b shows a complex action foridentifying any people associated with a paper. Thisincludes constructs for specifying complex action de-compositions, namely orFunc, seqFunc or andFunc.The post-condition specifies any conditions thatmust be satisfied after the decomposition has beenevaluated for the entire plan to remain valid. Finally,the stateBindingAlias specifies how input andoutput parameters and state-bindings are mappedbetween different sub-actions. Here, #$px-pl-pubof identify-people-associated-with-papermaps to the parameter #$px-pl-paper forpx-identify-paper-author.

The BM Planner applies Monte Carlo [4] (i.e.sampling-based) methods when evaluating the ef-fects of hypothetical actions represented by the HTN.Where the planner encounters an orFunc branch ormultiple bindings to World Model entities it makes arandom choice from the set of potential sub-actions.This enables the planner to rapidly sample a widerange of plans from the state space. It also simpli-fies the planner logic since no state needs to be main-tained in order to resume part way through previouslyevaluated plans. When a satisfiable plan is found, theplanner outputs a trace such as that shown in Fig-ure 3c.

Finally, BM also calculates a quantitative assess-ment of the utility of a plan using the plan scoringfunction:

Si =1

Li

×

Li∑

j=1

ajbj

where Si is the score for plan i, Li is the decomposed“length” of plan i, aj is the score for action j in plani, and bj is the score for the binding j in the context ofaction j. The plan scoring function is a plan-length-scaled sum of the scores for the depth first expansionof a given plan across the product of each sub-plan’saction score, aj , and its bindings score, bj . A largerscore indicates a better plan. The scaling factor 1/Li

ensures longer plans do not accumulate a higher scorethan shorter plans simply by virtue of their length.

3.5 Web Based User Interface

As shown in Figure 4, the top of the User Interfaceincludes two auto-completing text entry boxes for theuser to select a top level HTN plan, i.e. the goal (forexample identify-risk-of-information-loss)and specify any input parameters. This causes thePlanner to search for solutions to the goal. Assolutions are found they are dynamically merged intothe views being presented. This allows the user tobegin evaluating potential solutions as soon as theyare found, yet benefit from improved answers as theybecome available.

The User Interface has two views: a tree view anda state transition view. The tree view shows an HTNaction tree decomposition where nodes represent ac-tions and edges represent the decomposition of thoseactions. Complex actions can be expanded to showtheir child actions and different line styles are used

to differentiate between sequence, and, and or oper-ations. In the state transition view actions are nowedges while the nodes represent the @state that theylead to. Importantly, our system visualises hierarchi-cal (i.e. decomposable) actions as opposed to hier-archical states. It presents the user with a dynamiccanvas where they can pan and zoom into and out ofsubactions that are placed inside their parents, allow-ing the user to explore the solution at whatever levelof abstraction is appropriate at the time. The statetransition view also shows actions and bindings or-dered by rank so that those associated with the bestplans are shown first; and includes + and - buttons sothat users can tune the scoring of actions and bind-ings during plan ranking.

4 Use Cases

With participants’ permission we collected a rangeof traditional low level CND data sources from asmall enclaved network including traffic flow statis-tics, nmap [7] scans and host based Computer Sys-tem Information; full concurrence of all users was ob-tained and data from other parties was not captured.We were also able to obtain project tasking, publi-cations, procurement, travel and corporate directoryinformation from corporate Management InformationSystems. Finally, we scraped public information fromthe internet related to computer vulnerabilities, ex-ploits, and computer components and configurations.

Similar to other work on vulnerability analysisthrough automated planning [12] we then encoded anHTN to determine vectors an adversary could use toextract intellectual property from our organisation.The HTN first identifies where information related toa topic may be stored by searching the World Model’sknowledge of tasks, publications, procurement andtravel activities related to a topic of interest. It thenidentifies associated people, for example the membersof a task, authors of a paper, purchaser of equipmentand so on. Finally it cross references these peopleagainst the computer and networking entities thatthey use. A second stage then ascertains how theinformation might be lost from an identified source.For example, the HTN encodes social engineering ap-proaches to elicit the information from a person bycrafting believable emails based on a number of sce-narios including the person’s position in the organisa-tion or their recent purchasing activity. Alternatively,it correlates the software installed on a computer ofinterest with known vulnerabilities from Mitre’s CVEdatabase [10] and exploits from Metasploit’s exploitdatabase [9].

Figure 3c shows a single trace that combines so-cial engineering and software vulnerabilities for aplausible attack an adversary could use to elicitinformation about “Minisec”. The bindings forthis trace target #$AlexMurray, an author of#$Publication-36803, and identifies vulnerability#CVE-2011-0611 in the Adobe Flash Player on hiscomputer which is exploitable by Metasploit exploit#$PxExploit flash10o. It then assists in crafting aconvincing spear fishing email appearing to be froma service provider he has previously dealt with in#$Procurement-3739 to provide a delivery mecha-nism for the exploit1. Upon activation, this wouldprovide the adversary with access to Mr Murray’scomputer as a launchpad for extracting informationon the topic of interest.

1Again, it is emphasised that these actions are to assist in dis-covering vulnerabilities and thus countermeasures.


34

1 ( (:simple-action . "px-identify-paper-author ")

2 (:label . "Identify an author of a published paper")

3 (:parameters . (#$px-pl-paper ))

4 (:pre-condition . (#$and (#$px-pl-paper ?INPUT )(#$isa ?INPUT #$PxDSTOPublicationProcess )))

5 (:search-condition . (#$and (#$px-pl-paper ?PAPER)

6 (#$isa ?PAPER #$PxDSTOPublicationProcess)

7 (#$pxpublicationAuthor ?PAPER ?AUTHOR )))

8 (:effect-name . (@PaperAuthor ?AUTHOR ))

9 (:effect . nil)

10 (:state-binding . (( #$px-pl-author ?AUTHOR )))))

(a) A simple action for identifying an author of a DSTO publication.

1 ( (:complex-action . "identify-people-associated-with-paper ")

2 (:label . "Identify people associated with a specific published paper")

3 (:parameters . (#$px-pl-pub ))

4 (:pre-condition . (#$and (#$px-pl-pub ?INPUT)(#$isa ?INPUT #$PxDSTOPublicationProcess )))

5 (:orFunc . (nil))

6 (:seqFunc . (identify-paper-author ))

7 (:andFunc . (nil))

8 (:post-condition . (( px-identify-paper-author ( #$px-pl-author ?PERSON ))))

9 (:effect-name . (@PaperAssoc ?PERSON ))

10 (:effect . nil)

11 (:state-binding . (( #$px-pl-paper-associates ?PERSON )))

12 (:stateBindingAlias . ((( px-identify-people-associated-with-paper #$px-pl-pub)

13 (px-identify-paper-author #$px-pl-paper ))))))

(b) A complex action for identifying people associated with a publication.

1 ("px-identify-risk-of-information-loss " (@INFORMATIONLEAKAGE (#$AlexMurray ))

2 (#$SEQ (("px-identify-source-of-information " (@INFOSOURCE (#$AlexMurray ))

3 (#$SEQ (("px-identify-people-with-access-to-information"

4 (@PERSONACCESSINFO (#$AlexMurray ))

5 (#$SEQ (("px-identify-people-who-have-published-paper-on-information"

6 (@PAPERPEOPLE (#$AlexMurray ))

7 (#$SEQ (("px-identify-paper-relating-to-information "

8 (@PAPER (#$Publication-36803 )))

9 ("px-identify-people-associated-with-paper "

10 (@PAPERASSOC (#$AlexMurray ))

11 (#$SEQ (("px-identify-paper-author "

12 (@PAPERAUTHOR (#$AlexMurray )))))))))))))))

13 ("px-identify-information-leakage-path " (@IDENTIFYINFOLEAKAGE (#$AlexMurray ))

14 (#$SEQ (("px-identify-information-leakage-through-person"

15 (@INFORMATION-LEAKAGE-FOUND (#$AlexMurray ))

16 (#$SEQ (("px-identify-social-engineering-threat "

17 (@SOCIAL-ENGINEERING-THREAT (#$AlexMurray ))

18 (#$SEQ (("px-identify-spear-phishing-email-threat "

19 (@SPEAR-PHISHING-THREAT (#$AlexMurray ))

20 (#$SEQ (("px-identify-spear-phishing-exploit-threat "

21 (@SPEAR-PHISHING-EXPLOIT-THREAT (#$AlexMurray ))

22 (#$SEQ (("px-get-currently-used-computer-vulnerable-to-spear-phishing"

23 (@SPEAR-PHISHING-VULNERABLE-COMPUTER ( #$PxComputer10A50B42 ))

24 (#$SEQ (("px-identify-computers-currently-used-by-person"

25 (@CURRENTCOMPUTER ( #$PxComputer10A50B42 )))

26 ("px-get-vulnerability"

27 (@EMAIL-ATTACHMENT-VULNERABILITY (#$CVE-2011-0611 ))))))

28 ("px-identify-company-used-for-requisition "

29 (@PROCUREMENT-DETAILS

30 ("Supplier name: ACME Computing , Email: [email protected]"))

31 (#$SEQ (("px-get-requested-procurement"

32 (@PROCUREMENT (#$Procurement-3739 )))

33 ("px-get-procurement-supplier-details "

34 (@SUPPLIER-DETAILS

35 ("ACME Computing , [email protected]"))))))

36 ("px-identify-persons-details"

37 (@PERSONS-DETAILS ("Alex , Murray ,

38 [email protected]")))

39 ("px-identify-spear-phishing-email "

40 (@SPEAR-PHISHING-EMAIL

41 ("Dear Alex Murray ,

42 We would like to thank you for your recent purchase by

43 attaching our new and exclusive specials list.

44 These discounts apply to all orders received before the

45 end of the month.

46 Kind regards ,

47 ACME Computing")))

48 ("px-identify-email-attachment"

49 (@EMAIL-ATTACHMENT ("march-specials.pdf

50 (Vulnerability: #$CVE-2011-0611 ,

51 Exploit: #$PxExploit_MSF___flash10o)"

52 )))))))))))))))))))))

(c) An trace for satisfying the HTN identify-risk-of-information-loss.

Figure 3: Example simple and complex planning action definitions, and plan output.


35

Figure 4: User Interface showing a unified action tree view (left) and unified state transition diagram view(right).

This simple example demonstrates how an ex-pertly constructed HTN coupled with a richly inter-linked World Model can automatically identify com-plex and potentially novel vulernabilities for CND at-tention. One obvious remedial action would be topatch the vulnerable Adobe Flash Player; but onecan also envisage other proactive safeguards such astargeted user education.

5 Evaluation

Data Fusion. Entity matching and merging is rela-tively ad hoc in our current system. Entities are onlymatched based on their simple attributes, ignoringany wider context that could facilitate better deci-sions. For example, BM currently uses names, contactphone numbers and email addresses to match people.If only some of these attributes are available BM tendsto overfit, for example potentially mismatching “MarcAnderson” to “Marcus Anderson” if it is unaware ofdistinct personalities. Even where it did know of bothpeople, a reference to Marc Anderson in the contextof a trip to a conference on cybersecurity, for example,could be used to substantially reduce any ambiguity.

Similar to this is the problem of dealing withdata sources that represent the same concept indifferent ways. For example, in some cases “[Dr]Duncan Grove”, “Grove, D A”, “DSTO\groved”,or “duncan.grove@[dsto.]defence.gov.au” may inter-changeably be used to identify the same person (letalone when they denote different concepts like name,credential or email address); and similar difficultiesoccur for things, places, dates, times, etc. Althoughthe algorithms we employed were able to match suchvariations to World Model concepts, they usually re-quired hand-tuning to perform well.

Ontology and Plan Library. Due to limitationswith Cyc’s existing ontology, BM uses a substantialamount of custom ontology. This includes people,computer hardware and software, organisations and

business processes. For example, our current ontologyrelates these concepts in the following ways: peopleare linked as users of computers, members of an or-ganisation and actors within business processes; whilecomputers are composed of instances of hardware andsoftware components. While this enables a rich WorldModel for describing the types of scenarios discussedin Section 4, it would need to be extended for otheruses. The other significant limitation of our ontologyis that it does not currently have a detailed conceptof time or uncertainty.

Performance and Scalability. While our workbenefited from Cyc as an extant and integrated plat-form, we found its performance, scalability and sta-bility inadequate for our needs. Our use of the sys-tem was characterised by relatively slow insertion andqueries, often taking ≈ 1ms to assert a single triple;inefficient queries compared with manual query de-composition; and instability when performing querieswhile simultaneously inserting new data. We note,however, that our use of Cyc as a repository for largeamounts of data runs counter to its intended purposeand that our experiences may not necessarily hold inless data intensive scenarios.

The Planner’s use of Monte Carlo sampling per-mitted a simple implementation and produced diverseplans but its current implementation is quite slow.Although we have implemented a mechanism for userfeedback to select promising plans as discussed in Sec-tion 3.5, the Planner does not implement an auto-matic mechanism to capitalise on previous discoveryof viable plans. Heuristics such as ant colony optimi-sation [2] could help the Planner use existing plansas starting points for quickly identifying similar butmore optimal plans.

Machine Learning. The Planner’s supervisedlearning mechanism for plan ranking simply accountsfor the user’s indicated preference for particular ac-tions and bindings, using this as an informal summa-tion of perceived utility across many potential met-


36

rics including cost versus benefit, complexity, likeli-hood of success and so on. Furthermore, it definesthe context of an action or binding using only its im-mediate parent, so there is no way to indicate that abinding is good in one circumstance but bad in an-other if their immediate parents are the same. FinallyBM v1.5’s HTN Planner is not well suited to identi-fying novel plans, merely finding ways to apply pre-conceived plans in newly encountered but generallywell understood situations; but this could be tackledby incorporating classical planning and/or machinelearning to identify novel combinations of HTNs forsolving unanticipated problems.

6 Related Work

DSTO’s Shapes Vector [3] (SV) uses a data drivenintelligent agent framework for collecting and pro-cessing information, performing inference, and pro-viding cyber situational awareness. Although BM cansolve similar problems to SV, it takes a very differ-ent approach to its knowledge management and in-ference architecture, making use of modern semanticweb technologies to construct a rich World Model us-ing first order predicate logic. This has advantagesand disadvantages. Where SV’s architecture is moretuned for near real time performance, BM v1.5 optsfor a flexible predicate logic based architecture that ispotentially capable of producing high quality knowl-edge, but doing so more slowly.

CycSecure [13] is also similar to BM v1.5. How-ever, although its Cyc [5] foundation provides a richand detailed ontology, the CycSecure extensions ap-pear to have a limited model for cyber entities andonly a small set of rules supporting very narrow rea-soning. Its scope also seems limited to cyber resourcesand functions, ignoring broader organisational func-tions and business processes, which is a major dif-ferentation of BM v1.5 compared with extant sys-tems. Another novel contribution is BM’s use of busi-ness process modelling, for example as a concise re-porting mechanism for providing situational aware-ness.

7 Conclusion

We have described BM version 1.5, one of a series ofAI systems being developed for semi-automatic Com-puter Network Defence. Although BM v1.5 uses awide range of existing techniques, their application toCND is unusual, as is considering not just computersand networks but also the people, organisations andbusiness processes that link them.

Acknowledgements

The authors would like to thank DSTO DistinguishedFellow (Cyber) and Parallax Director, Dr Mark An-derson, and also the Chief Engineer Parallax, ChrisNorth, for their oversight, guidance and support.

References

[1] G. Combs. tshark - Dump and analyze networktraffic. http://www.wireshark.org/docs/man-pages/tshark.html, 2012.

[2] V. Dorigo, M. Maniezzo and A. Colorni. Antsystem: optimization by a colony of cooperatingagents. Systems, Man, and Cybernetics, Part

B: Cybernetics, IEEE Transactions on, 26(1):29–41, Feb 1996.

[3] D. Engelhardt and M. Anderson. A distributedmulti-agent architecture for computer securitysituational awareness. In Information Fusion,2003. Proceedings of the Sixth International Con-ference of, volume 1, pages 193 – 200, 2003.

[4] W. K. Hastings. Monte Carlo sampling meth-ods using markov chains and their applications.Biometrika, 57(1):97–109, 1970.

[5] D. Lenat. CYC: A large-scale investment inknowledge infrastructure. Communications ofthe ACM, 1995.

[6] V. I. Levenshtein. Binary codes capable of cor-recting deletions, insertions, and reversals. SovietPhysics Doklady, 10(8):707–710, 1966.

[7] G. Lyon. Nmap network scanning: official Nmapproject guide to network discovery and securityscanning. Insecure.Com, LLC, 2009.

[8] J. Mchugh. Intrusion and intrustion detection.International Journal of Information Security,1:14–35, 2001.

[9] Metasploit. Exploits database.http://www.metasploit.com/modules, 2012.

[10] Mitre. CVE database.http://cve.mitre.org/, 2012.

[11] D. Nau, T.-C. Au, O. Ilghami, U. Kuter, J. W.Murdock, D. Wu, and F. Yaman. SHOP2: AnHTN planning system. Journal of Artificial In-telligence Research, 20:379–404, dec 2003.

[12] M. Roberts, A. Howe, I. Ray, M. Urbanska, Z. S.Byrne, and J. M. Weidert. Personalized vulner-ability analysis through automated planning. InWorking Notes of IJCAI 2011, Workshop Secu-rity and Artificial Intelligence (SecArt-11),, 042011.

[13] B. Shepard, C. Matuszek, C. Fraser, W. Wecht-enhiser, D. Crabbe, Z. Gundordu, J. Jantos,T. Hughes, L. Lefkowitz, M. Witbrock, D. Lenat,and E. Larson. A knowledge-based approach tonetwork security: applying cyc in the domain ofnetwork risk assessment. In Innovative Applica-tions of Artificial Intelligence Conference, 2005.


37

http://www.wireshark.org/docs/man-

pages/tshark.html

http://www.metasploit.com/modules

http://cve.mitre.org/


38

an overview of the parallax battlemind v1.5 for computer network

Documents