let your csirt do malware analysis, recruit-csirt has … · let your csirt do malware analysis,...

Tatsuya Ichida

Recruit Technologies, Co. Ltd.

Let your CSIRT do malware analysis, Recruit-CSIRT has done it!

FIRST TC Amsterdam 2017, 25th April

(C) Recruit Technologies Co.,Ltd. All rights reserved.

Agenda

Self-introduction

Background and Motivation

Malware Analysis System for Recruit-CSIRT

Advantages and Disadvantages

Conclusion

2


Self-introduction

Recruit-CSIRT since 2015

Security Engineer for developing useful tools

Incident handler at Recruit-CSIRT

Loves Malware Analysis

Splunk Log Analyst

Tokyo Denki University CySec speaker

In the past, Security Operation Center, Malware Analysis Leader

CISSP, GCIH, GPEN

3

Tatsuya Ichida (age 29)


Recruit-CSIRT

4

SOC QMG

IRG

CSIRT

IRGIncident Response Group

QMGQuality Management Group

SOCSecurity Operation Center

・Incident Handling・Internal and external coordination・Emergency Forensic

Containment

PreventionDetection

・Security Log Monitoring・Artifacts Analysis・Forensic Investigation

・Vulnerability Assessmentand Management

・Literacy Education

No Incidents -> Can develop tools!

Background and Motivation


Background 1

6

500

Malware explosion in the wild

million

Ref：https://www.av-test.org/en/statistics/malware/


Background 2

7

Ransomeware explosion in our env

100

200


Background 3

8

We discovered a malware file over 100MB in size!

Can it be done Reverse Engineering rapidly? No way!

100MB


Background 4

9

2015.Q1 2015.Q2 2015.Q3 2015.Q4 2016.Q1 2016.Q2 2016.Q3

Work time

Reducing

Work Time


Motivation - Try

10

FIRST Step

Using Commercial Malware Analysis Products.

Sandbox Product A -> Advantage of Anti-Sandbox.

Sandbox Product B -> Advantage of Mal-Signature.

But .. When We got malwaremalware 「α」 was not analyzed by A because of the Browser Version.

malware 「β」 was not analyzed by B because of Anti-sandbox technique

malware 「γ」 was not analyzed by both because of the size !

A B

βα

γ

We paid

a lot of

money!


Motivation - Challenge

Purpose

– Reducing cost

– Reducing user work time

– Stored knowledge internally

11

Let’s create our own malware analysis system

It’s impossible to create a malware analysis system

that can handle all samples perfectly.

government of the people, by the people, for the people

by Lincoln

Analysis of our CSIRT, by our CSIRT, for our CSIRT

by Recruit

Our system’s target is “our malware”

Malware Analysis System for Recruit-CSIRT


Malware Analysis System Overview

13

Manage Server

Analysis Guest VMWindows Server（AD, FileServ）

Internet

Commands

LogsMalware Submit

Receive Malware

Web

Report View

Connections

cuckooDB

Real timeDB

C&C traffic

IntelligenceDB


Analysis Scheme

Auto-Collect Malware

Auto-Optimize Analysis env

Real-time View of Behavior

Changes

■From Introduced Malware Detection Sensor

‐ Targeted Malware

■From Internet Malware DB

- Newest Malwarebefore being

targeted

■Auto-time Sync

■ Over 100MB huge malware analysis

■Auto-Defense againstexternal attacks

■Auto-log collector- Pcap excluding normal

■Real-time Visualization- mark behavior changes- Intelligence Table- Process Behavior Table- Packet Traffic Table

■Anti-Virus management

System

Malware Analysis Scheme

Post Intelligence

InternetOur Sensor

WebProxy/

FW

http://malicious.jp

■Auto-C&C Server Analysis

■ Block The C&C traffic- C&C’s IP- C&C’s FQDN

basedbased

14

Image Confidential

In speaking only

you can see



16

Advantages Disadvantages

Optimized guest image env

Capable of analyzing huge

malware samples

Anti-Virus detection control

Auto-C&C analysis

Real-time visualization

Cannot handlea lot of malware

No accelerated sleep bypass

Weak to virtual env evasion


Advantages -「Optimized guest Image env」

17

Recruit Standard env Image

Replica

Accounts (Ground bait)

Secret

‘Odoriba’ env

・ the Same Image

- Same Middleware, Same Applications, Same Versions

・ Some Ground bait, Mouse Control and Real Date

It help us to focus only on malware infecting our env

Image Confidential

In speaking only

you can see


Advantages -「Capable of analyzing huge malware 」

• Cuckoo Sandbox 2.0 rc1– Agent Default

• /cuckoo/agent/agent.py

– XMLRPC based connection to host

» Huge malware samples cause memory exceptions

» Because of oversize XMLRPC’s memory…

– Manager Default

• /cuckoo/lib/cuckoo/core/guest.py– Has two managers

– OldGuestManager Class(default) and GuestManager Class(for new agent)

18

• We enhanced Cuckoo Sandbox 2.0 rc1– New Agent

• https://github.com/jbremer/agent/blob/master/agent.py

– HTTP based Connection: Agent launches SimpleHTTPServer

– No limit on Chunk Data to submit

– Manager uses “GuestManager Class” in guest.py

We added functions to the agent: time-sync, etc.


Advantages - 「Anti-Virus Detection Control」

19

AntiVirus

Realtime Scan

Submit

’file’ is

deleted

continue

Clean

finish

‘file’ = submitted malware

Almost Sandbox system -> Antivirus disabled

Usually prevents analysis

Our system permits Antivirus to delete the sample.We observe while the malware and its child processes exist in our env.

Image Confidential

In speaking only

you can see


Advantages - 「Auto-C&C Analysis」

20

IntelligenceDB

Monitor TrafficProcess behavior (Based on Win API calls)

Pcap (Catch on host)

Certainly malware emits traffic

But it may not include C&C traffic

Rootkit traffic cannot be caught

Include all traffic even from rootkits

Include much normal traffic

Huge volume

Except

Windows and Normal

App traffic

Store IntelligenceDB

(IPv4 or FQDN)

Monthly Update to

“tcpdump exception rule”

Analysis target

Multi

Thread

Check whether is on hosting server via HTTP

Check its whois info from cymru

Check virustotal reputation

(downloaded malware from)

Result Update DB


Advantages -「Real-Time Visualization」1

21

Default cuckoo report cannot be watched until analysis finished.

Created ‘running page’ to see the real-time behavior

1min F5 update

Real-time view tells us the behavior changes ASAP

-> Rapid Block Action & Rapid Re-Analysis

Analysis Finish Button


Advantages -「Real-Time Visualization」2

22

IntelligenceDBFQDN,IP,WhoisOwner,CC,WebResponse,WebTitleVT MalDownloadUrls,VT MalDownloadFile

Real-

timeDB

bsonbson

bson

pcap page1.json

page2.json

page3.json

・・・

SpentSecond

Thread id

Called API name

Points

Category

Is Success?

Time

SpentSecond

DstIP

DstPort

SrcIP

SrcPort

Protocol

Dump Top 128 byte

Time

page1.json

page2.json

page3.json

・・・


Disadvantages -「Cannot Handle a lot of Malware」

23

System

InternetOur Sensor

Our System handles in “Single Thread”

Malware can make us wait…

Generally, preprocessing seems to be important for this system. ■ reducing the input sample

- (auto) duplicate hash

- (auto) untargeted extension and file-type

- (manual) ‘targeted’ or ‘common’ by analyst

■ reducing during analysis

- Handle Anti-Virus detection

Discovered by forensic

When we catch APT malware through forensic,

We analyze long-term to observe the changes

But we

C&C’s domain, IP

spawn files,

Attacker’s visit etc…

FILE or URL


Disadvantages -「No accelerated sleep bypass」

Malware often calls ‘Sleep’ to wait for some time

Some Sandbox products have functions…Accelerated sleep bypass

In order to analyze the sample efficiently

However malware is evolving…

Have Anti-sandbox techniques for this

Ex. CPU Clock difference using GetTickCount etc..

24

New Anti-sandbox tech

New Anti-anti-sandbox tech

Our human resource is limited.

We don’t take this into account.

i.e. Raw Analysis

Maybe

Endless?


Disadvantages -「Weak to virtual env evasion」

Malware often checks whether it runs on a virtual machine or not, halts its execution in analysis envs.

There may be also endless Anti-Sandbox techniques employed.

25

Recruit changed Office PCs to VDI Thin Client.

Virtual env = Our env

Some Signature should be removed, but not all.

It is important to imitate VDI’s Virtual env.

Conclusion


Conclusion

Effective for our malware which is affected

Can be used flexiblyTheoretically no limit, since it is developed by

ourselves

Our System is not perfect to analyze all malware.

27

ShieldsImitation the env Real-Time

Analyzing View

Do you want this ?


Thank

Thanks to FIRST and OSSs.

28

[email protected]

let your csirt do malware analysis, recruit-csirt has … · let your csirt do malware analysis,...

Documents