featuresmith - usenix · • featuresmith discovered new features – getsimoperatorname –...

FeatureSmithLearningtoDetectMalwarebyMiningtheSecurityLiterature

Benign

Malicious

SecurityandMachineLearning

T.Dumitraș::FeatureSmith

Usedfordetec+ngspam,phishing,malware,networka7acks,maliciousdomains,vulnerabilityexploitsinthewild,compromisedWebsites,…

Whatdoesitmeanfortwosamplestobesimilar?

2

FeaturesinMachineLearningModels

•  Howshouldwecomparesamples?–  Spam: keywords,featuresfrom

emailheader,…

–  AIbots:vocabulary,sentence structure,…


Ittakesonetoknowone?Featureengineering

3

RunningExample:AndroidMalwareDetecEon

•  Howshouldwecomparesamples?–  Permissions

•  Protectsensi+vedataandfunc+onality•  Doesnotworkforprivilegeescala+on

–  APImethodcalls•  Revealmalwarebehaviors

•  Featureengineering–  Usedomainknowledgetoiden+fyusefulfeatures– MustconsiderthreatsemanEcs

T.Dumitraș::FeatureSmith 4

TheSecurityBodyofKnowledge

•  Growingvolumeofpapers,industryreports,blogs,…


Difficulttoassimilateallrelevantknowledge

5

Dilemma


VS.

Growingbodyofknowledge Needforgoodfeatures

CanweengineerfeaturesautomaEcally,byminingsecuritypapers?

6

CanwecreateanarEficialintelligencethathelpsusbuild

otherintelligentsystems?

SecurityThreatsinNaturalLanguage

•  “TheZsonemalwareisdesignedtosendSMSmessagestocertainpremiumnumbers”*


*Zhouetal.‘Hey,you,getoffofmymarket:Detec+ngmaliciousappsinofficialandalterna+veandroidmarkets,’NDSS2012.

SMSfraud

8

SecurityThreatsinNaturalLanguage

•  “GingerMaster[…]iso>enbundledwithbenignapplica?onsandtriestogainrootaccess”*


Evasion,privilegeescala+on

*Arpetal.‘Drebin:Effec+veandExplainableDetec+onofAndroidMalwareinYourPocket,’NDSS2014.

9

Plato’sAllegoryoftheCave


Illustra+onbyJohnD’Alembert

10

DomainKnowledge


Challenge#1

UnderstandingthesemanEcmeaning

–  Basedoncommonsense,knowledgeofsecuritydomain


Challenge#2

A7ackerbehaviorskeepevolving

–  Securityarmsrace

– Mustdiscoveropen-endedbehaviors


IEEESecurityandPrivacySymposium

13

IntuiEonforAutomaEcFeatureEngineering


Accesssensi+vedata

Communicateovernetwork

Executeexternalcommands

getDeviceId

getSubscriberId

execH7pRequest

setWifiEnabled

Run+me.exec

Features(suspiciousAPIcalls)Malwarebehaviors*

*Arpetal.NDSS’14

14

BehaviorExtracEon

•  Behavior–  Descrip+onofmalwareac+vity–  Shortphrase

•  <subject?,verb,object?>•  Parsegramma+calstructureofsentences

“TheZsonemalwareis

designedtosendSMS

messagestocertain

premiumnumbers”*


•  ZsonemalwaresendSMSmessages

•  designedZsonemalware

•  Zsonemalwaresendtocertainpremiumnumbers

*Zhouetal.NDSS’12

15

BehaviorUnderstanding

•  Linkbehaviorstoconcretefeatures


“APIcallsforaccessingsensi?vedata,suchasgetDeviceId()”*

accessingsensi+vedata

getDeviceId()

*Arpetal.NDSS’14

16

BehaviorUnderstanding

•  Linkbehaviorstomalware


ZsonemalwareisdesignedtosendSMSmessagestopremiumnumbers

Zsone sendSMSmessages

*Zhouetal.NDSS’12

17

SemanEcNetwork

•  Nodes:securityconcepts– Malwarefamilies: nameden++es–  Concretefeatures: nameden++es–  Behaviors: openended

•  Edges:seman+callyrelatedconcepts– Weightsbasedondistanceandco-occurrence


SemanEcNetworkExample


Zsone

Zitmo

SendSMSmessage

Iden+fyexecu+onpath

Extractsenderphonenumber

Openmanifestfile

SEND_SMS

sendTextMessage

Thread.start

createFromPdu

openXmlResourceParser

Malware Behavior Feature

1

1

0.25

0.75

19

HowWellDoesThisWork?

AutomaEcfeatureengineering•  FeatureSmith

–  Analyzed1,068securitypapers

–  AutomaEcallyengineered195featuresrelevanttoAndroidmalware•  Outof383foundinthepapers

Manualfeatureengineering•  Drebin*

–  State-of-the-artAndroidmalwaredetector

–  Uses545,334features•  Including315suspiciousAPIcalls,manuallycurated


*Arpetal.NDSS’14

20

Autovs.Manual:Experiment

AutomaEc•  Featuresengineeredby

FeatureSmith

Manual•  FeaturesusedinDrebin


•  Sameclassifica+onalgorithm•  Samecorpusofbenignandmaliciousapps•  Samefeaturetypes•  Experiment:Comparethetwofeaturesets

21

Autovs.Manual:Features

•  FeatureSmithdiscoverednewfeatures–  getSimOperatorName–  getNetworkOperatorName–  getCountry

•  Onenusedbymalware–  HelpdetectGappusinfamily (notdetectedbyDrebin)


Missingfrommanuallyengineeredset

HumandatascienEstscannotassimilateallrelevantknowledge

22

Autovs.Manual:DetecEonPerformance

0.00 0.02 0.04 0.06 0.08 0.10FDlse PRsitive 5Dte

0.80

0.85

0.90

0.95

1.00

Tru

e P

Rsi

tive 5

Dte




0.80

0.85

0.90

0.95

1.00

Tru

e P

Rsi

tive 5

Dte

Drebin



0.80

0.85

0.90

0.95

1.00

Tru

e P

Rsi

tive 5

Dte

FeDture6Pith

Drebin



Paritywithmanualfeaturesat1%falseposiEves

25

KnowledgeEvoluEon


0.00 0.02 0.08 0.100.04 0.06 False Positive Rate

0.80

0.85

0.90

0.95

1.00Tr

ue P

ositi

ve R

ate

Feature sets2012 (24 features) 2013 (32 features) 2014 (40 features)2015 (46 features)

Effec+venessoffeaturesdiscoveredindifferentyears

26

AlternaEves

•  Featureselec+on– Mustenumerateallpossiblefeaturesinadvance(e.g.allAndroidpermissions)

•  Representa+onlearning–  Discoversusefulfeatures(representa+ons)fromrawdata(e.g.usinganeuralnetwork)

•  Disadvantages–  Data-driven:mayreflectbiasesinthegroundtruth–  Noautoma+cdiscoveryofthreatsemanEcs


InANutshell•  Automa+cfeatureengineering

–  DiscoversemanEcallymeaningfulfeatures•  Somemissingfrommanuallycuratedset

–  Performanceonparwithstate-of-the-artmalwaredetector–  Manypoten+alapplica+ons

•  Security: AIbots,threatintelligence,intrusiondetec+on,…•  Otherfields: biomedicalresearch,IBM’sWatsonQ&Asystem

•  Complementshuman-drivenfeatureengineering–  Humandatascien+stshaveintuiEon–  FeatureSmithcanreasonoverenErebodyofknowledge

•  Paperanddata:h7p://featuresmith.org


AutomatedsystemscanunderstandthesemanEcsofsecurityconcepts

ThisisapowerfultoolforcreaEngabacksanddefenses

Thankyou!


TudorDumitraș@tudor_dumitras

http://featuresmith.org

Acknowledgments:•  WorkwithZiyunZhu•  RobotcartoonsbyKatyTresedder

30

featuresmith - usenix · • featuresmith discovered new features – getsimoperatorname –...

Documents