ryan riley, dongyan xu, xuxian jiang · 2020. 8. 26. · kernel malware attacks operating system...

47
+ Junghwan Rhee, * Ryan Riley, + Dongyan Xu, ** Xuxian Jiang + Department of Computer Science, Purdue University * Department of Computer Science and Engineering, Qatar University ** Department of Computer Science, North Carolina State University 1

Upload: others

Post on 04-Oct-2020

0 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

+Junghwan Rhee, *Ryan Riley, +Dongyan Xu, **Xuxian Jiang

+ Department of Computer Science, Purdue University

* Department of Computer Science and Engineering, Qatar University

** Department of Computer Science, North Carolina State University

1

Page 2: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Background

Allocation-driven mapping

Evaluation

Discussion

Conclusion

Demo

2

Page 3: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Kernel malware attacks operating system kernels.

e.g., kernel rootkits

Attack goals

Hide processes, files, etc.

Provide hidden services, backdoors, etc.

Attack techniques

Hijack system services (e.g., system calls)

Directly manipulate kernel data (DKOM)

Hijack hooks by overwriting function pointers (KOH)

3

Operating system kernel

User applications

Page 4: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Kernel malware attacks operating system kernels.

e.g., kernel rootkits

Attack goals

Hide processes, files, etc.

Provide hidden services, backdoors, etc.

Attack techniques

Hijack system services (e.g., system calls)

Directly manipulate kernel data (DKOM)

Hijack hooks by overwriting function pointers (KOH)

4

ROOT

Operating system kernel

User applications

Page 5: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Kernel memory mapping has been used for kernel integrity checking and kernel malware detection.

Existing approaches

Type-projection mapping: kernel objects identification by recursively traversing pointers from global objects

▪ Static: memory snapshots as input

▪ Dynamic: memory traces as input

5

Page 6: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Type-projection mapping using memory snapshots

SBCFI [CCS 2007]

Gibraltar [ACSAC 2008]

KOP [CCS 2009]

Type-projection mapping using memory traces

Rkprofiler [RAID 2009]

PoKeR [Eurosys 2009]

6

Page 7: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

7

X *next

X *prev

Static memory Dynamic memory

struct X {struct X *next;struct X *prev;

}

s1

A memory snapshot

Address Address

Value

Data type definition of X

Page 8: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

8

X *next

X *prev

Static memory Dynamic memory

struct X {struct X *next;struct X *prev;

}

a1

a1

a2

a3

X *next

X *prev

a2

X *next

X *prev

a3

X *next

X *prev

s1

s1

Address Address

ValueValue

Data type definition of X

A memory snapshot

Page 9: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

9

X *next

X *prev

Static memory Dynamic memory

struct X {struct X *next;struct X *prev;

}

a1

a1

a2

a3

X *next

X *prev

a2

X *next

X *prev

a3

X *next

X *prev

s1

s1

The map of kernel objects is subject to the manipulation by malware.

Address Address

ValueValue

Data type definition of X

Page 10: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

X3X1

X1, X2, and X3 : kernel objects allocated in the same address with the same data type.

A malware analyzer based on asynchronous mapping may not be able to differentiate X1, X2, and X3.

Memory maps from snapshots

Dynamicmemory

status

Timet1 t2

X1 X3X2

10

Malware analysis using an asynchronous mapping

allocfreefree alloc

Benign Benign?

Page 11: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

X2 X3X1

X1, X2, and X3 : kernel objects allocated in the same address with the same data type.

A malware analyzer based on asynchronous mapping may not be able to differentiate X1, X2, and X3.

Memory maps from snapshots

Dynamicmemory

status

Timet1 t2

X1 X3X2

11

Malware analysis using an asynchronous mapping

allocfreefree alloc

Benign BenignUnder attack

Page 12: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

X2 X3X1

X1, X2, and X3 : kernel objects allocated in the same address with the same data type.

A malware analyzer based on asynchronous mapping may not be able to differentiate X1, X2, and X3.

Memory maps from snapshots

Dynamicmemory

status

Timet1 t2

X1 X3X2

12

≠≠

Malware analysis using an asynchronous mapping

allocfreefree alloc

Benign BenignUnder attack?

Page 13: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Kernel objects are identified by

transparently capturing kernel

memory function calls.

The memory ranges are extracted

from function arguments and return

values.

Call stack information (allocation call

site) is used to derive data types.

13

p = kmalloc (size, …);

p

p + size

p = kmalloc (…);Allocationcall site

struct T {… };

Runtimecall stack

Source code

* An memory allocation call site: code address of a memory allocation call

Page 14: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Lifetime of a dynamic kernel object

Allocation DeallocationUsage

14

Page 15: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Advantages

Un-tampered view▪ Tolerant to the manipulation of memory content

Lifetime of a dynamic kernel object

Allocation DeallocationUsage

15

Page 16: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Advantages

Un-tampered view▪ Tolerant to the manipulation of memory content

Temporal view▪ Lifetime of dynamic data is tracked to differentiate objects at

the same memory location

Lifetime of a dynamic kernel object

Allocation DeallocationUsage

16

Page 17: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

17

Kernel stack

Registers

Kernel memory

Gu

est

OS

Kernel source code

Kernel object map

a = kmalloc (size, flag);

VM

M

Allocation

* An memory allocation call site: code address of a memory allocation call

, ,size

A map entry for an object

Page 18: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

18

Kernel stack

Registers

Kernel memory

Gu

est

OS

Kernel source code

Kernel object map

a = kmalloc (size, flag);

VM

M

Allocation

* An memory allocation call site: code address of a memory allocation call

, ,sizea+a

A map entry for an object

Page 19: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

19

Kernel stack

Registers

Kernel memory

Gu

est

OS

Kernel source code

Kernel object map

a = kmalloc (size, flag);

VM

M

Allocation

* An memory allocation call site: code address of a memory allocation call

Call site

, ,sizea+a Call siteRuntime identifier:

a memory allocation call site*

A map entry for an object

Page 20: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

20

Kernel stack

Registers

Kernel memory

Gu

est

OS

Kernel source code

Kernel object map

kfree (a);

VM

M

Deallocation

* An memory allocation call site: code address of a memory allocation call

, ,sizea+a Call site

A map entry for an object

Page 21: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

21

Debugging Information

Modified Compiler

Static analysis

Kernel source code

Extracted code

elements

Memory allocation call sites*

Allocation code

statements

Data types

A: a = kmalloc (size, flag);

D: struct X *a;

T: struct X {int a*;

};

An assignment statement

A declaration of a pointer

A type definition

* An memory allocation call site: code address of a memory allocation call

, … , Call site

X

Page 22: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

LiveDM : Live Dynamic kernel memory Map

Supported guest OS kernels Redhat 8, Debian Sarge, Fedora Core 6

Virtual machine monitor : QEMU

Knowledge of kernel memory functions is assumed.

Type resolution Debugging symbols for translation of allocation call

sites Modified gcc compiler to extract code elements

22

Page 23: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Effectiveness

Performance

Applications

Hidden object detector (un-tampered view)

Temporal malware behavior monitor (temporal view)

23

Page 24: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

24

A list of core dynamic kernel objects (OS: Debian Sarge)Total dynamic kernel objects: 29488

A D TType resolution Identified

instances

kernel/fork.c:248

Allocation statementkernel/fork.c

248 tsk =

kmem_cache_alloc(...);

Page 25: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

25

A list of core dynamic kernel objects (OS: Debian Sarge)Total dynamic kernel objects: 29488

A D TType resolution Identified

instances

kernel/fork.c:248 kernel/fork.c:243

Allocation statementkernel/fork.c

248 tsk =

kmem_cache_alloc(...);

Declaration of a pointerkernel/fork.c

243 struct

task_struct *tsk;

Page 26: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

26

A list of core dynamic kernel objects (OS: Debian Sarge)Total dynamic kernel objects: 29488

A D TType resolution Identified

instances

kernel/fork.c:248 kernel/fork.c:243 task_struct

Allocation statementkernel/fork.c

248 tsk =

kmem_cache_alloc(...);

Declaration of a pointerkernel/fork.c

243 struct

task_struct *tsk;

Type definitioninclude/linux/sched.h

390 struct task_struct {

...

};

Page 27: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Manual analysis: convert allocation call sites to data types (similar to validation methods of KOP [Carbone et. al., CCS 2009] and Laika [Cozzie et. al., OSDI 2008])

27

Manual Analysis

Kernel source code

Static analysis

Extracted code

elements

Memory allocation call sites

Data types

Data types

Debugging Information

=

Allocation code

statements

Page 28: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Benchmarks

Kernel compile, UnixBench, nbench

Overhead

Slowdown compared to unmodified QEMU (worst in benchmarks): 42% for Linux 2.4, 125% for Linux 2.6

Mainly caused by the capture of dynamic objects

Near-zero overhead for CPU-intensive benchmarks

Non-production application scenarios

Honeypot, malware profiling, kernel debugging

28

Page 29: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Hidden object detector

Periodic comparison of an allocation-driven map and memory content

Allocation-driven map Memory content

29

Page 30: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Hidden object detector

Periodic comparison of an allocation-driven map and memory content

Allocation-driven map Memory content

30

Page 31: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Hidden object detector Periodic comparison of an allocation-driven map

and memory content 10 kernel rootkits are tested and all detected. Agnostic to the injection of malware code Non-code injection attacks (hide_lkm and fuuld) are

detected.

Allocation-driven map Memory content

31

Page 32: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Temporal Malware Behavior Monitor

Systematically visualize malware influence via the manipulation of dynamic kernel memory

Steps

32

Time

A log of kernelcontrol flow

A log of memoryaccesses

Allocation-driven map log

Causes

Effects

Page 33: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Temporal Malware Behavior Monitor

Systematically visualize malware influence via the manipulation of dynamic kernel memory

Steps

33

Time

A log of kernelcontrol flow

A log of memoryaccesses

Allocation-driven map log

Causes

Effects

The list of kernel objects manipulated by adore-ng rootkit

T3

Page 34: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

34

Time(Billions of instructions)

0.1 0.2 0.40.3

Memory accessesto T3’s address

(+:read, x :write)

Kernel control flow

Page 35: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

35

Time(Billions of instructions)

0.1 0.2 0.40.3

Memory accessesto T3’s address

(+:read, x :write)

Kernel control flow

Before attack After attack

Page 36: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

36

Time(Billions of instructions)

0.1 0.2 0.40.3

Memory accessesto T3’s address

(+:read, x :write)

Kernel control flow

Allocation-drivenmap log

T1‘s lifetime

T2’s lifetime

T3’s lifetime

T4’s lifetime

T5’s lifetime

Before attack After attack

The time range relevant to the attack

Page 37: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Malware analysis is guided to the attack victim objects (e.g., T3).

37

Time(Billions of instructions)

0.1 0.2 0.40.3

Memory accessesto T3’s address

(+:read, x :write)

Kernel control flow

Allocation-drivenmap log

T1‘s lifetime

T2’s lifetime

T3’s lifetime

T4’s lifetime

T5’s lifetime

Before attack After attack

The time range relevant to the attack

Page 38: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Ker

nel

mem

ory

ad

dre

ss

Memory address offset Memory address offset

kernel modulesproc_dir_entrytask_struct (PCB) rootkit ext3

Before the rootkit attack After the rootkit attack

T3 T3

38

Kernel object maps

Page 39: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Ker

nel

mem

ory

ad

dre

ss

Memory address offset Memory address offset

kernel modulesproc_dir_entrytask_struct (PCB) rootkit ext3

Before the rootkit attack After the rootkit attack

T3 T3

PCB statusuid = euid = 500

suid = fsuid = 500gid = egid = 500

fsgid = 500cap_effective

= cap_inheritable= cap_permitted

= 0User credentials

PCB statusuid = euid = 0

suid = fsuid = 0gid = egid = 0

fsgid = 0cap_effective

= cap_inheritable= cap_permitted

= 0xffffffffRoot credentials

39

Privilege escalation attack

Kernel object maps

Page 40: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Before the rootkit attack

40

Kernel control flow graphs

Execution time

Page 41: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Before the rootkit attack

41

Hook invocation

Kernel control flow graphs

Execution time

Read of a function pointer

Page 42: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Execution time

Before the rootkit attack After the rootkit attack

42

Kernel control flow graphs

Execution time

Page 43: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Execution time

Before the rootkit attack After the rootkit attack

43

Kernel control flow graphs

Read of a function pointer

Hijacked hook

activity

Redirection

Execution time

Page 44: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Memory objects of 3rd party drivers, malware

Source code is required to derive data types.

Memory aliasing (type casting)

Allocation-driven map does not have aliasing problem by avoiding the evaluation of pointers.

Allocation using generic pointers : 0.1% of total objects

Attack cases towards memory functions

44

Page 45: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Un-tampered and temporal views of dynamic kernel objects can be enabled for malware analysis.

Kernel data hiding attacks can be detected by using an un-tampered view.

Temporal view can guide a malware analyzer to attack victim objects by tracking data lifetime.

45

Page 46: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

Main technique: Live kernel object map Live status is dumped to a GUI every 5 seconds. Dynamic changes of the map are illustrated.

Applications: Hidden PCB and module detector HP rootkit hides processes. modhide rootkit hides kernel modules (drivers). Data hiding attacks are checked every 5 seconds.

URL: http://www.cs.purdue.edu/homes/rhee/pubs/raid2010_livedm.avi

Note: some parts of a video clip are trimmed to reduce its play time.

46

Page 47: Ryan Riley, Dongyan Xu, Xuxian Jiang · 2020. 8. 26. · Kernel malware attacks operating system kernels. e.g., kernel rootkits Attack goals Hide processes, files, etc. Provide hidden

47