reactively adaptive malware what is it? how do we detect it? dr. bhavani thuraisingham cyber...

Reactively Adaptive MalwareWhat is it?

How do we detect it?

Dr. Bhavani ThuraisinghamCyber Security Research and Education Institute

https://csi.utdallas.edu

The University of Texas at Dallas

April 19, 2013

1FEARLESS engineering

FEARLESS engineering

Outline

• Analogies

• Malware: What is it?

• Our Solutions– Profs. Thuraisingham, Khan, Hamlen, Lin, Makris,

Cardenas, Kantarcioglu

• Directions – Holistic Interdisciplinary Treatment

FEARLESS engineering

Analogies: The Human Body• Humans infected with virus and

bacteria

• Virus replicates itself and spreads throughout the body

• Attacks vital organs

• Doctor conducts tests and detects the problem

• Medicine is given to slow the progress of the disease

• Patient’s condition may improve or the patient may die

FEARLESS engineering

Analogies: An Organization

• Bad person joins the organization and pretends to be a good person

• He/she monitors what is going on and spies on the organization

• Conveys vital information to the adversary – insider threat

• Builds a network of bad people

• Takes over the organization

FEARLESS engineering

What is a Malware?• It’s a piece of software that is malicious and

carries out bad things

• It infects a vulnerable and neglected machine

• It attacks the various components of the machine– the operating system (vital organs), applications (limbs) and hardware (bone)

• It spreads across a network of machines

• It cripples the machines and the network

• It conveys vital information to the enemy – the hacker

• It takes over the network and carries out its agenda

Victim Network

What does it look like?Example: Melissa Virus March 26, 1999

The Virus-Antivirus Arms Race• Malware (e.g., viruses)

– Rogue programs that carry out malicious actions on victim machines

• Vandalism (delete files, carry out phishing scams, etc.)• reconnaissance & secret exfiltration (cyber-warfare /

hacktivism)• Sabotage (e.g., attacks against power grids)

– Randomly mutate themselves automatically as they propagate

• Harder to detect since no two samples look identical• Antivirus defenses

– Defenders manually reverse-engineer many malware samples

– Find mutation patterns– Build defenses to automatically detect & quarantine all

mutants

FEARLESS engineering

FEARLESS engineering

Incidents Reported 1990-2001

Incidents Reported to Computer Emergency Response Team/Coordination Center (CERT/CC)

0

10000

20000

30000

40000

50000

60000

90 91 92 93 94 95 96 97 98 99 00 01

Everything changed with Code Red attack in 2001

FEARLESS engineering

Problem is much worse now!

FEARLESS engineering

Our Malware Team

Adversarial Mining SolutionsProfessor Murat Kantarcioglu

Data Mining Solutionsfor MalwareProfessor Latifur Khan

Reactively Adaptive Malware and SolutionsProfessor Kevin Hamlen

Android Malware andSolutionsProfessor Zhiqiang Lin

Hardware Malwareand SolutionsProfessor Yiorgos Makris

Smart Grid Malwareand SolutionsProfessor Alvaro Cardenas

Data Mining Solutions

Data Mining

Knowledge Discoveryin Databases

Knowledge Extraction

Data Pattern Processing

The process of discovering meaningful new correlations, patterns, trends and nuggets by sifting through large amounts of attack data, often previously unknown, using pattern recognition technologies and machine learning statistical and mathematical techniques.

FEARLESS engineering

Thuraisingham, Data Mining: Technologies, Techniques, Tools and Trends, CRC Press 1998

Training and Testing

Testing Data

DGSOT: Dynamically Growing Self-Organizing Tree Our novel solution

FEARLESS engineering

TrainingData

Enhancements to current

data mining approachesHierarchical Clustering (DGSOT)

Testing

Data Mining Classification

ModelTraining

GoodClass

BadClass

• Supported by US Air Force 2005-2008

– PI: Thuraisingham, Co-PI: Khan

• Extract features

✗Binary n-gram features

✗Assembly n-gram features

Report Results: Example

• HFS = Hybrid Feature Set (Binary and Assembly)• BFS = Binary Feature Set• AFS = Assembly Feature Set

FEARLESS engineering

Reactively Adaptive Malware: What is it?

• Next-generation Malware Technology

– Malware that mutates NON-randomly

– LEARNS and ADAPTS to antivirus defenses fully automatically in the wild

– Immune to conventional antivirus defenses

– Supported by the U.S. Air Force; 2010-2013

• PI: Hamlen, Co-PI: Khan

FEARLESS engineering

FEARLESS engineering

Data Mining-based Anti-antivirus[Hamlen & Khan]

Antivirus Signature Database

Signature Q

uery Interface

Signature Inference

Engine

Signature Approximation

Model

Obfuscation Generation

Obfuscation Function

Malware Binary

Obfuscated Binary

Testing propagate

“Frankenstein”[Mohan & Hamlen, USENIX WOOT, 2012]

• Stitch together code harvested from benign binaries to re-implement malware on each propagation.

• Many offensive advantages:– resulting malware is 100% metamorphic

• no common features between mutants

– statistically indistinguishable from benign-ware• everything is plaintext code (no cyphertexts)

– no runtime unpacking• evades write-then-execute protections

– obfuscation is targeted and directed• evolves to match infected system’s notion of

“benign”

FEARLESS engineering

Frankenstein Press Coverage• Presented at USENIX Offensive Technologies (WOOT) mid-August 2012• Thousands of news stories in August/September

– The Economist, New Scientist, NBC News, Wired UK, The Verge, Huffington Post, Live Science, …

FEARLESS engineering

Solution we are exploring: SNODMAL Solution we are exploring: SNODMAL Stream Based Novel Class DetectionStream Based Novel Class Detection

• Divide the data stream into equal sized chunks– Train a classifier from each data chunk– Keep the best L such classifier-ensemble

Data chunks

Classifiers

D1

C1

D2

C2

D3

C3

Ensemble C1 C2 C3

D4

Prediction

D4

C4C4

C4

D5D5

C5C5

C5

D6

Labeled chunk

Unlabeled chunk

Addresses infinite lengthand concept-drift

Note: Di may contain data points from different classes

FEARLESS engineering

Smartphones can also beinfected with malware!

FEARLESS engineering

Our Solution – Combine Static Analysis with Dynamic Analysis

FEARLESS engineering

• Static Analysis– Data mining solutions

• Dynamic Analysis– Platform– Android & I-Phone– Reverse engineering

• Level– System call– Operating systems– Network

• Supported by US Air Force 2012-2016– Technical Leads Lin and Khan

Remote Server

Mal App

Network Behavior

App Behavior

We cannot forget about

HardwareDo you Trust

Your Chips?Yiorgos Makris

(yiorgos.makris@utdallas.edu)

The Hacker in Your Hardware, Villasenor, Scientific American 2010

The Hunt for the Kill SwitchAdee, IEEE Spectrum, 2008 3500 counterfeit Cisco networking

components recovered

2012 Phobos-Grunt Mission Fails Due to Counterfeit Non Space-Rated Chips

Research Supported by:

Our Solution to Hardware Trojan

FEARLESS engineering

That’s not all – Attacks to Critical Infrastructures

Attacks Maroochy Shire 2000

Threats

HVAC 2012

Stuxnet 2010

Smart Meters 2012

Obama administrationdemonstrates attack to power grid in Feb. 2012

DHS and INL study impact of cyber-attacks on generator

FEARLESS engineering

New Attack-Detection Mechanisms by Incorporating “Physical Constraints” of the System

• 1st Step: Model the Physical World • 2nd Step: Detect Attacks– Compare received signal from

expected signalPhysical World

System ofDifferential Equations

Model

• 3rd Step: Response to Attacks • 4th Step: Security Analysis Missed Detections

Study stealthy attacks False Positives

Ensure safety of automated response

[Alvaro Cárdenas, et.al. AsiaCCS, 2011]

FEARLESS engineering

It never ends!We need to mine the adversary

• Adversary changes its behavior to avoid being detected

• Data Miner and the Adversary are playing games

• Remember, malware detection is a two class problem?

•Good class (e.g., benign program)

•Bad class (e.g., malware)

• Adapt your classifier to changing adversary behavior

• Questions?–How to model this game? Does this game ever end?–Is there an equilibrium point in the game?

FEARLESS engineering

Our Solution: Game Playing• Adversarial Stackelberg Game

– Adversary chooses an action

– After observing the action, data miner chooses a counteraction

– Game ends with payoffs to each player

• Adversary may use malware obfuscation

• Change has some cost to the adversary

• We need data mining techniques to handle the changes by the adversary

• Funded by the US Army; 2012-2015

– PI: Kantarcioglu, Co-PI: Thuraisingham

FEARLESS engineering

FEARLESS engineering

Where do we go from here:Holistic Treatment

Three actors interacting with each other:

•The Doctor

– The Defender/Analyst

•The Patient

– The User /Soldier

•The Virus/Bacteria

– The Malware/Attacker

Together with ECS, SOM, EPPS and BBS, we are proposing an Interdisciplinary approach.