mr201407 an example of antivirus detection rates and similarity of undetected malware

13
1 Monthly Research An Example of Antivirus Detection Rates and Similarity of Undetected Malware FFRI,Inc http://www.ffri.jp Ver 2.00.01

Upload: ffri-inc

Post on 20-May-2015

106 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: MR201407 An example of antivirus detection rates and similarity of undetected malware

FFRI,Inc.

1

Monthly Research

An Example of Antivirus Detection Rates and Similarity of Undetected Malware

FFRI,Inc http://www.ffri.jp

Ver 2.00.01

Page 2: MR201407 An example of antivirus detection rates and similarity of undetected malware

FFRI,Inc.

Agenda

• Background and purpose

• Surveyed Malware

• About FFRI Dataset

• An example of antivirus detection rates

• Considerations of the detection rates

• Similarity of undetected malware by fuzzy hashing

• Summary

2

Page 3: MR201407 An example of antivirus detection rates and similarity of undetected malware

FFRI,Inc.

Background and purpose

• The executive of antivirus vendor said "antivirus software is dead" in May 2014. It became a conversation topic.

• In this slides, we show an example of antivirus detection rates.

• We investigated similarity of undetected malware by fuzzy hashing to improve detection rate.

3

Page 4: MR201407 An example of antivirus detection rates and similarity of undetected malware

FFRI,Inc.

Surveyed Malware

• Property

– Known malware

– Those are not targeted malware which is hard to detect in general.

• Collection period

– From January 2014 to April 2014

• Number of malware

– 3,000

• Investigation period of detection rates

– April 24, 2014 and July 7, 2014

4

Page 5: MR201407 An example of antivirus detection rates and similarity of undetected malware

FFRI,Inc.

About FFRI Dataset

• We have provided the dataset for anti-malware research.

– It is available in the MWS (Anti-Malware Workshop). • http://www.iwsec.org/mws/2014/

• Overview of the FFRI Dataset 2014

– Dynamic analysis log (Malware behavior information) of the foregoing malware.

– It was generated by Cuckoo Sandbox and FFR yarai analyzer Professional.

– See below for more information. • http://www.iwsec.org/mws/2014/files/FFRI_Dataset_2014.pdf

5

Page 6: MR201407 An example of antivirus detection rates and similarity of undetected malware

FFRI,Inc.

An example of antivirus detection rates

6

93.2% 91.4%

88.7% 88.0%

78.8%

58.2%

45.9% 43.8%

38.8%

15.5%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

A B C D E F G H I J

Detection rates for 3,000 malware of FFRI Dataset 2014

Page 7: MR201407 An example of antivirus detection rates and similarity of undetected malware

FFRI,Inc.

Considerations of the detection rates

• There were unexpected differences in detection rates.

• The detection rates did not change almost, after 2 months.

• Generic detection (pattern matching) was better than

reputation-based detection.

• Free products was low performance.

• Attention points

– The result is only by static detecting.

– Therefore, it is not comprehensive evaluation.

7

Page 8: MR201407 An example of antivirus detection rates and similarity of undetected malware

FFRI,Inc.

Similarity of malware by fuzzy hashing

• Fuzzy hashing is a kind of technology to identify similar files.

– Refer to Monthly Research in March 2014 for more information.

• We investigated following items by fuzzy hashing.

– How many similar pairs are there in undetected malware group?

– How many undetected malware which are similar to detected malware are there?

• We used the sdhash is a fuzzy hashing tool.

– http://sdhash.org

– Definition of similar pair is sdhash score over 21.

8

Page 9: MR201407 An example of antivirus detection rates and similarity of undetected malware

FFRI,Inc.

Result: Similarity of Undetected Malware

• Similarly avg. - Average number of similar malware when sampling one undetected malware.

9

Vendor Undetected Similarly avg. Similar to detected

A 197 2.89 44

B 253 8.11 75

C 340 2.29 71

D 360 3.72 130

E 628 4.33 124

F 1250 6.51 309

G 1620 16.01 175

H 1670 11.30 320

I 1835 15.95 85

J 2580 14.24 106

Page 10: MR201407 An example of antivirus detection rates and similarity of undetected malware

FFRI,Inc.

Percentage of undetected malware which are similar to detected malware

10

93.2% 91.4% 88.7% 88.0%

78.8%

58.2%

45.9% 43.8% 38.8%

15.5%

1.5% 2.5% 2.4% 4.3%

4.2%

10.3%

5.8% 10.8%

2.8%

4.1%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

A B C D E F G H I J

up

base

Page 11: MR201407 An example of antivirus detection rates and similarity of undetected malware

FFRI,Inc.

Considerations

• Most vendors possibly require 100 to 200 additional patterns to detect undetected malware.

• There are not many undetected malware which are similar to detected malware.

– It is going to improve detection rate of 10.8% in the best, if detecting the undetected malware which are similar to detected malware.

– However, these are improvements of less than 5% for most vendors.

11

Page 12: MR201407 An example of antivirus detection rates and similarity of undetected malware

FFRI,Inc.

Summary

• There are great differences between static detection rates of antivirus in the FFRI Dataset 2014.

• Most vendors possibly require 100 to 200 additional patterns to detect undetected malware.

• There are not many undetected malware which are similar to detected malware.

• Behavior detection is required because static detection is reaching the limit.

12