Detecting & PreventingMisuse of Privilege

PI Meeting 1/27/05

Bob Balzer (Teknowledge)

Howie Shrobe (MIT)

• Updates since Kickoff

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

LegacyApp

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

LegacyApp

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

MITTeknowledge

Distinguishing AWDRAT & PMOP

• AWDRAT– Detecting misbehaving software

• Hijacks, overprivledged scripts, trap doors, faults

• PMOP– Detecting misbehaving operators

• Malicious intent, operator error

• For integrated SRS system need both capabilities– Have had extensive discussions on integrating

both projects together - headstart on workshop :-)

MAF

CAF

Proposed MI

Approved MI

Targeting TNL

JEESEDC JW

CHWChem

Hazard

SPI TAP

CHI

Combat

Ops

AODB AS

LOC

Weather

Hazard

WH

WLC

ATO

EDC

CHW

Chem

Hazard

CHA

External

JBI DemVal Dataflow(via Publish/Subscribe)

What We’ve Got

• End-To-End Demonstration (demo shortly)– Working Prototypes of PMOP components– Working models & rules of target application– Working integration of PMOP components

The Good – The Bad – The Ugly

End-To-End Demonstration• Block Harmful Operations

• Differentiate– Operator Error

– Malicious Intent

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

JBIDemVal

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

What We’ve Got

• End-To-End Demonstration (demo shortly)– Working Prototypes of PMOP components– Working models & rules of target application– Working integration of PMOP components

The Good – The Bad – The Ugly

• Architecture Visualizer (demo shown in AWDRAT)

– Event-Sequence diagrams– Architecture dataflow

What We’re Missing

• Realistic Rules (Domain Knowledgeable)– Would be created by SMEs in real deployment

• Comprehensive Rule Set– Would be created by SMEs in real deployment

• Instrumentation of the GUI actions– Just Mission Building/Editing methods currently

instrumented– GUI actions will be instrumented by 4/1/05

The Good – The Bad – The Ugly

Accommodations

• Java code base– Created wrapper infrastructure for Java

• Planning Application (harm is in future)– Defined Harm as publishing harmful plan

• Available JBI components to wrap– Detailed on next slide

The Good – The Bad – The Ugly

Canned ComponentPublishes fixed output

Legacy ComponentCode Not Available

Table Lookup

MAF

CAF

Proposed MI

Approved MI

Targeting TNL

JEESEDC JW

CHWChem

Hazard

SPI TAP

CHI

Combat

Ops

AODB AS

LOC

Weather

Hazard

WH

WLC

ATO

EDC

CHW

Chem

Hazard

CHA

External JBI DemVal Dataflow(via Publish/Subscribe)

The Good – The Bad – The Ugly

Differences from AWDRAT

• Harm Detector instead of Architecture Diff• Client Reconstitution inactive

M

M

Mediation Cocoon

M

M

JBIServer

PMOP Execution Architecture

JBIClient

Harm Rules

Harm Detector

Scripted PMOP Driven from History ScriptsNominalHarmful: Takeoff Before LandingHarmful: Missing Leg (landing not collocated with takeoff)

Visualizer

Scripts

Script Driver

History

ClientReconstitution

ArchitectureVisualizer

M

M

Mediation Cocoon

M

M

JBIServer

JBIClient

Mixed Initiative PMOP• One Client Live (with human operator)• Others Scripted

DetectingHarmful Actions

Demo

Determining Intent

• Determining that an insider is/has been taking malicious action is a task for human security agents and managers.

• Our automated system takes the action of raising an “alarm”, based on:– Degree of harm in the action– Probability of Malicious intent

• And provides the initial evidence

Degree of Harm

• We are interested in examining harm done by maliciously or accidentally creating a defective plan of action, such as an Air Tasking Order.

• We base our calculation of harm on a static analysis of the probable consequences of a plan.

• How the error happened is only used as evidence of intent.

Categories of Harmful Plans

• Plan results in direct damage – e.g.:– Friendly fire incident– Political harm from attacking non-combatants

• Plan results in a denial of resources – e.g.– Wasting munitions and sorties– Creating confusion– Putting valuable personnel under suspicion.

Factors used to Determine Intent

• The harm is more likely to be intentional:1. If the plan defect depends on a more deliberate, more conscious

process

2. If the actions can be fit into a larger plan of action

3. If there are related historical errors for the operator in question.

4. If the action involves coordination with others (inside or outside)

• Even type 1 involves analyzing the trace of actions

• Others involve keeping historical “Case Book”

Evidence of a Deliberate Process

• If the defect in the plan occurs through a plan editing, rather than plan creation step.

• If there is evidence of information hiding.

• If there is evidence of tampering with logs or other monitors.

Processing of MAF/CAF Traces

• Parse XML of traces• Accumulate parsed trace into “User Actions”

– Event creation followed by setInformation methods -> Single Event creation

• Follow though sequence of User Actions simulating effect on plan, detecting when harmful effect is created.

• Edited in harmful effect flagged as definite malicious

Raw Trace

missing-leg 5 6**end-of-messages**<trace><MethodEntermethodClass="mil.af.rl.jbi.client.ExtensibleMappingClient.toolsets.MissionPlan.MissionEventObject" thread="0"/> <MethodReturn methodClass="mil.af.rl.jbi.client.ExtensibleMappingClient.toolsets.MissionPlan.MissionEventObject" thread="0"> <this class="mil.af.rl.jbi.client.ExtensibleMappingClient.toolsets.MissionPlan.MissionEventObject"

printer="1"/></MethodReturn> <MethodEnter methodName="setInformation" methodClass="mil.af.rl.jbi.client.ExtensibleMappingClient.toolsets.MissionPlan.MissionEventObject" methodSignature="(Ljava/lang/String;Ljava/lang/String;)V" thread="0" arg0="EVTTYPE" arg1="TO"> <this class="mil.af.rl.jbi.client.ExtensibleMappingClient.toolsets.MissionPlan.MissionEventObject" printer="1"/></MethodEnter> ....

Parsed

(("missing-leg 5 6")

(ENTER :NAME CONSTRUCTOR :CLASS

"mil.af.rl.jbi.client.ExtensibleMappingClient.toolsets.MissionPlan.MissionEventObject")

(RETURN :NAME CONSTRUCTOR :CLASS

"mil.af.rl.jbi.client.ExtensibleMappingClient.toolsets.MissionPlan.MissionEventObject" :THIS

("MissionEventObject" "1"))

(ENTER :NAME "setInformation" :CLASS

"mil.af.rl.jbi.client.ExtensibleMappingClient.toolsets.MissionPlan.MissionEventObject" :ARG0

"EVTTYPE" :ARG1 "TO" :THIS ("MissionEventObject" "1"))

(RETURN :NAME "setInformation" :CLASS

"mil.af.rl.jbi.client.ExtensibleMappingClient.toolsets.MissionPlan.MissionEventObject" :ARG0

"EVTTYPE" :ARG1 "TO" :THIS ("MissionEventObject" "1"))

...

Reconstructed

(("missing-leg 5 6") (EVENT :THIS ("MissionEventObject" "1") :EVTTYPE "TO" :EVTCD "I" :EVTSEQID "1" :LOCID "KBLV-1" :LATITUDE "-89.804" :LONGITUDE "38.671" :TIMEON "2004-05-27T19:25:23Z" :TIMEOFF "2004-05-27T19:25:23Z" :ALT "0" :AMCPURPCD "A" :EVTSUBTYPE "-" :SUBTYPECALLSIGN

"-" :SUBTYPEFREQ "-" :SUBTYPEMSNCD "-") (EVENT :THIS ("MissionEventObject" "2") :EVTTYPE "REFUEL" :EVTCD "T" :EVTSEQID "2" :LOCID

"PATRIOT-2" :LATITUDE "3.164" :LONGITUDE "52.031" :TIMEON "2004-05-28T03:05:20Z" :TIMEOFF "2004-05-

28T03:05:20Z" :ALT "280" :AMCPURPCD "Z" :EVTSUBTYPE "-" :SUBTYPECALLSIGN "-" :SUBTYPEFREQ

"-" :SUBTYPEMSNCD "-") (EVENT :THIS ("MissionEventObject" "3") :EVTTYPE "LDG" :EVTCD "I" :EVTSEQID "3" :LOCID "LIPA-3" :LATITUDE "12.070" :LONGITUDE "46.230" :TIMEON "2004-05-28T04:45:20Z" :TIMEOFF "2004-05-28T04:45:20Z" :ALT "0" :AMCPURPCD "A" :EVTSUBTYPE "-" :SUBTYPECALLSIGN

"-" :SUBTYPEFREQ "-" :SUBTYPEMSNCD "-")...

Interpreted

MISSING-LEG Between event 5 and 6

CREATING event 1 Take Off 05/27/2004 19:25:23 KBLV -89.80 38.67

CREATING event 2 Refuel 05/28/2004 03:05:20 PATRIOT 3.16 52.03

CREATING event 3 LDG 05/28/2004 04:45:20 LIPA 12.07 46.23

CREATING event 4 Take Off 05/28/2004 07:20:20 LIPA 12.07 46.23

CREATING event 5 LDG 05/28/2004 08:35:20 LICZ 14.73 37.62

CREATING event 6 Take Off 05/28/2004 11:35:20 LICZ 14.73 37.44

CREATING event 7 LDG 05/28/2004 17:15:20 OEKH 47.70 24.08

EDITING event 6 Take Off 05/28/2004 11:35:20 LICZ 5.43 47.64

Editing event after its creation

Not leaving from where you landed 5 6 14.726 37.617 5.4346514 47.63672

Editing over existing leg causes error - Malicious

...

MALICIOUS

DetectingMalicious Intent

Demo

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

LegacyApp

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

What are we trying to do?• Block Harmful Operations

• Differentiate– Operator Error

– Malicious Intent

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

LegacyApp

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

How will you show success?• Block Harmful Operations

• Differentiate– Operator Error

– Malicious Intent

• Red-TeamExperiment

Block Harmful Operations

Differentiate– Operator Error

– Malicious Intent

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

LegacyApp

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

What are implicationsof success?

• Systems can be protectedfrom insider attacks

from operator error

from zero-day attacks

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

LegacyApp

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

What is technical approach?• Observe effect of operator

action in system model• Match harmful

actions against– Errorful Operator Plans– Attack Plans

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

LegacyApp

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

What is new?• Observe effect of operator

action in system model• Match harmful

actions against– Errorful Operator Plans– Attack Plans

BehaviorAuthorizer

M

M

M

M

Mediation Cocoon

LegacyApp

BehaviorMonitor

OperatorAction

OperationalSystemModel Predicted

State

HarmAssessment

BenignOperatorAction

HarmfulOperatorAction

GUI

IntentAssessment

OperatorError

MaliciousInsider

What is hard?• Modeling System

to predict effect• Modeling Operator

to differentiate– Operator Error– Malicious Intent

Technology for SRS Integration

• Behavior Monitor/Authorizer– What code is doing– What human operator is doing

• Operational Models– Software Components– Human Operators

• Harm Detector– Rule driven

• Intent Determination