efficient detection of split personalities in malware
DESCRIPTION
Efficient Detection of Split Personalities in Malware. Davide Balzarotti, Marco Cova, Christoph Karlberger, Christopher Kruegel, Engin Kirda and Giovanni Vigna NDSS 2011 Feb. OUTLINE. Introduction and Related Work Our Approach Implementation Evaluation Conclusion. Introduction. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/1.jpg)
Efficient Detection of Split Efficient Detection of Split Personalities in MalwarePersonalities in Malware
Davide Balzarotti, Marco Cova, Christoph Karlberger, Christopher Kruegel, Engin Kirda
and Giovanni Vigna NDSS 2011 Feb.
1
![Page 2: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/2.jpg)
OUTLINEOUTLINEIntroduction and Related WorkOur ApproachImplementationEvaluationConclusion
2
![Page 3: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/3.jpg)
IntroductionIntroductionMalware detection
◦ Static◦ Dynamic
Sandboxes(Anubis, CWSandbox, Joebox, Norman Sandbox)
Counterattack◦ Attacks on Virtual Machine Emulators◦ CPU semantics, timing attacks◦ Environment attacks
Processes, drivers, or registry values
3
![Page 4: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/4.jpg)
SoluSolu11:Transparent malware :Transparent malware analysisanalysisCobra
◦ Code blocks◦ Replace instruction with a safe version
Ether◦ Hardware virtualization
More difficult to detect by malicious code.Great, but slow.
4
![Page 5: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/5.jpg)
Solu2:Detect different Solu2:Detect different behavesbehaves“Emulating Emulation-Resistant
Malware”, 2009◦ Reference system vs. emulated
environment◦ Compare execution path◦ Use Ether to produce the reference trace
But executing the same program twice can lead to different execution runs.
5
![Page 6: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/6.jpg)
OUTLINEOUTLINEIntroduction and Related WorkOur ApproachImplementationEvaluationConclusion
6
![Page 7: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/7.jpg)
Our approachOur approachRecording and Replaying
◦ Reference system vs. emulated environment
◦ system call trace: types and arguments
If there is a different behavior◦ Rerun it in a transparent framework(Ether)
Detect malware reliably and efficiently
7
![Page 8: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/8.jpg)
Reliability Two systems are execution-
equivalence if all program that◦ Start from the same initial state◦ Same inputs on both systems => Same runtime behavior => Same sequence of the system calls?
Assume no race condition
8
![Page 9: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/9.jpg)
Reliability(cont.)If our reference system and the
analysis system are execution-equivalence, any difference in the observed behavior => split-personality
Also, this discrepancy is the result of CPU semantics or timing attacks
9
![Page 10: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/10.jpg)
Making Systems Making Systems Execution-Execution-EquivalenceEquivalenceSame OS environment
Same address space layout of a process at load time
Same inputs to a program◦ Run program on the reference system in
log mode◦ Run program on the analysis system in
replay mode◦ System call matching
10
![Page 11: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/11.jpg)
ReplayReplay ProblemProblemA number of system calls are not safe
to replay◦ Allocating memory, spawning threads
Only replay for those system calls that read data from the environment◦ other system calls are passed directly to the
underlying OS
Delay cause additional system calls◦ WaitForSingleObject()
11
![Page 12: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/12.jpg)
System Call MatchingSystem Call Matching
12
![Page 13: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/13.jpg)
OUTLINEOUTLINEIntroduction and Related WorkOur ApproachImplementationEvaluationConclusion
13
![Page 14: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/14.jpg)
ImplementationImplementationA kernel driver
◦ Trap all the system calls Hook “System Service Descriptor Table”
◦ Each system call, two handler, log and replay
A user-space application◦ Start and control the driver◦ Start the process that has to be analyzed◦ Store the data generated during the
logging phase
14
![Page 15: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/15.jpg)
Practical aspectsPractical aspectsHandles consistency
◦ Live handles and replayed handles◦ Check a list of all replayed handles
Networking◦ NtDeviceIOControlFile()◦ Device-dependent parameters
15
![Page 16: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/16.jpg)
Practical aspects(cont.)Practical aspects(cont.)Deferred results
◦ STATUS_PENDING◦ NtWaitForSingleObject()
Thread Management◦ NtCreateThread()◦ Each thread has a new log
16
![Page 17: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/17.jpg)
LimitationsLimitationsMemory Mapped Files
◦ DLLs◦ Create file with memory-mapped
Remove the system calls
Multiple processesRandom numbers
◦ KsecDDInter-process communication and
asynchronous callsPostponing check
17
![Page 18: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/18.jpg)
OUTLINEOUTLINEIntroduction and Related WorkOur ApproachImplementation EvaluationConclusion
18
![Page 19: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/19.jpg)
EvaluationEvaluationMicrosoft Windows XP Service Pack 3VMware virtual machineAnubis system(Qemu)
1. Log and Replay six programs(success)2. SDBot(fail)
◦ spawning new process, like NtCreateProcess◦ six different versions to detect
VMware(success) Red Pill, Scoopy, VMDetect, and SourPill
19
![Page 20: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/20.jpg)
Evaluation(cont.)Evaluation(cont.)3. Real Malware with no VM-checks
20
![Page 21: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/21.jpg)
Evaluation(cont.)Evaluation(cont.)4. Real malware with VM-checks
21
![Page 22: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/22.jpg)
PerformancePerformanceDepends on the type of operation
◦ Average 1% overhead
Compresses a 1KB-long random fileCMD: 7za.exe a test.zip 1KB_rand_file
Anubis: 4.267 secEther: 77.325 secOur Vmware reference system: 1.640
sec22
![Page 23: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/23.jpg)
OUTLINEOUTLINEIntroduction and Related WorkOur ApproachImplementation EvaluationConclusion
23
![Page 24: Efficient Detection of Split Personalities in Malware](https://reader035.vdocuments.us/reader035/viewer/2022081515/56814dc7550346895dbb1af6/html5/thumbnails/24.jpg)
ConclusionConclusionA prototypeRecording system calls and replay
themNeed a fully transparent, analysis
system for further examination
24