deeplocker - black hat briefings · deeplocker – ai-powered concealment and unlocking e a 1 a 2 a...

DeepLocker Concealing Targeted Attacks with AI Locksmithing Dhilung Kirat, Jiyong Jang, Marc Ph. Stoecklin IBM Research

2

Cognitive Cyber Security Intelligence (CCSI) IBM Research

Dhilung Kirat Jiyong Jang Marc Ph. Stoecklin

3

AI-aided attacks Profile a target to increase success [2]

Craft an email to bypass filters [1]

Mutate malware to bypass AV [3, 4]

Execute machine-speed/creative attacks [5, 6]

AI

[1] S. Palka et al., “Fuzzing Email Filters with Generative Grammars and N-Gram Analysis”, Usenix WOOT 2015 [2] A. Singh and V. Thaware, “Wire Me through Machine Learning”, Black Hat USA 2017 [3] J. Jung et al., “AVPASS: Automatically Bypassing Android Malware Detection System”, Black Hat USA 2017 [4] H. Anderson, “Bot vs. Bot: Evading Machine Learning Malware Detection”, Black Hat USA 2017 [5] DARPA Cyber Grand Challenge (CGC), 2016 [6] D. Petro and B. Morris, “Weaponizing Machine Learning: Humanity was Overrated Anyway”, DEF CON 2017

4

[1] S. Palka et al., “Fuzzing Email Filters with Generative Grammars and N-Gram Analysis”, Usenix WOOT 2015 [2] A. Singh and V. Thaware, “Wire Me through Machine Learning”, Black Hat USA 2017 [3] J. Jung et al., “AVPASS: Automatically Bypassing Android Malware Detection System”, Black Hat USA 2017 [4] H. Anderson, “Bot vs. Bot: Evading Machine Learning Malware Detection”, Black Hat USA 2017 [5] DARPA Cyber Grand Challenge (CGC), 2016 [6] D. Petro and B. Morris, “Weaponizing Machine Learning: Humanity was Overrated Anyway”, DEF CON 2017

AI-aided attacks Profile a target to increase success [2]

Craft an email to bypass filters [1]

Mutate malware to bypass AV [3, 4]

Execute machine-speed/creative attacks [5, 6]

AI

AI-embedded attack

AI

AI capability embedded inside malware itself

5

Malware concealment – Locksmithing

Decryption code

Encryption Hide payload

⎯⎯⎯ Packers

Evasive malware Avoid being analyzed

⎯⎯⎯ VM check

Processes check

Decryption code

Checks for sandbox

Targeted attack Disclose only at a target

⎯⎯⎯ Target attributes check

Decryption code

Checks for target

Obfuscation Mutate payload

⎯⎯⎯ Polymorphism Metamorphism

1980 1990 2000 2010

6

AI Locksmithing

© 2003 Warner Bros. Pictures All Rights Reserved

7

Unleashing DeepLocker – AI Locksmithing

8

DeepLocker – Overview

Existing malware Benign-looking

malware

Benign application

Target attributes

AI

Execution at target Malicious behavior

AI DeepLocker

Execution at non-target Benign behavior

Distribution & execution

AI-powered concealment

Evading static, dynamic, manual analysis

Security analysis

9

DeepLocker Deep Dive

10

Is this not a sandbox?

Yes No

Start

Traditional targeted attack

End

Execute Attack!

Is this a target?

RegKeyExists(“HKLM\SOFTWARE\SIEMENS\STEP7”) Stuxnet

11

Is this a target?

AI-powered targeted attack

Start

End

Execute Attack!

AI

12

What is a Deep Neural Network (DNN)?

Output Layer Input Layer Hidden Layers

f ( wixii=1

n

∑ )

w1

w2

wn

Sigmoid()tanh()ReLU()

x1

x2

xn

y

13

Input Layer Output Layer Hidden Layers

Deep Convolutional Neural Network

[1] Krizhevsky, Alex, et. al. "Imagenet classification with deep convolutional neural networks.” NIPS 2012.

AlexNet (2012) [1]

8 layers, 622K neurons, 60 million parameters

14

Target

Target attributes Audio, visual

User activity

Physical environment

Geolocation

Sensors

Software environment

15

Target detection Reference

Face Extracted

Face

�

Yes No

Template matching requires a template to match to.

Start

End

Execute Attack!

DNN

16

Derivation of an unlocking key

Start

End

Execute Attack!

DeepLocker

17

DeepLocker – AI-Powered Concealment and Unlocking

Ea1

a2

a3a4

a5

Concealed payload

a2a1

a3

Target attributes Secret key Target Concealment

encryption

Payload Concealment Malicious payload

Conc

ealm

ent

DeepLocker

Malicious payload

a1

a3a2

Concealed payload Payload Unlocking Recovered key Input attributes Target Detection

decryption

Unlo

ckin

g

18

AI-powered concealment

No decryption key available in malware

sample to reverse engineer!

DeepLocker

19

Key generation

Face Detection Model

Bucketization

High-dimensional facial features Key Target

0.9

0.0

0.8

0.1

0.9

1

0

1

0

1

Unstable key!

20

Key generation

Face Detection Model

Key Generation Model

Key Target

0

1

1

0

1

Class 1

Class 2

Class 3

Class 4

Class 5

þ ý

þ ý þ

0.9

0.0

0.8

0.1

0.9

High-dimensional facial features

21

Analysis of the key generation model

Target Detection

Dataset: Labeled Faces in the Wild (LFW) http://vis-www.cs.umass.edu/lfw/

22

DeepLocker – AI-powered concealment

1 Target Class Concealment

Does not reveal what it is looking for (e.g., faces, organization, or a completely obscure object specific to the target environment)

2 Target Instance Concealment

If the target class is an individual, it does not reveal who it is looking for

3 Malicious Intent Concealment

Payload is fully encrypted concealing how the final attack is executed

23

Attacking DeepLocker – AI Lock Picking

24

Ways to counter

AI

Payload execution

AI-powered concealment

DeepLocker

25

Ways to counter

AI

Payload execution

AI-powered concealment

Code attestation Block sensor access

AI usage monitoring Host-based monitoring

Brute-force key Brute-force attributes

Deceptive resources Deceptive attributes

Code analysis AI lock picking

26

?

27

Partial occlusion Occlude a portion of the image to see how the embedding is affected (deconvnet) [1]

Neural attention model Heatmap using the degree of excitation of neurons in each layer (excitation backprop) [2]

Debug neural networks Fuzzing for neural networks (coverage-guided fuzzing) [3]

Reverse engineering AI models

[1] M. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” ECCV 2014 [2] J. Zhang et. al., “Top-down neural attention by excitation backprop,” ECCV 2016 [3] A. Odena and I. Goodfellow, “TensorFuzz: Debugging neural networks with coverage-guided fuzzing,” arXiv 2018

28

Takeaways

DeepLocker is a demonstration of the potential of

AI-embedded attacks

Current defenses will become obsolete and

new defenses are needed

Rapid democratization of AI has made AI-powered attacks an imminent threat

Thank you

Dhilung Kirat dkirat@us.ibm.com

Jiyong Jang j jang@us.ibm.com

Marc Ph. Stoecklin mpstoeck@us.ibm.com IBM Research