making a scalable automated hacking system by artem dinaburg

MakingaScalableAutomatedHackingSystem

ShakaConVIIIArtemDinaburg

artem@trailo=its.com

•  PrincipalSecurityEngineeratTrailofBits– artem@trailo=its.com/artem@dinaburg.org– hFp://blog.trailo=its.com

AboutMe

AboutThisTalk•  Automatedvulnerabilitydiscoveryisaverybroadtopic.

AboutThisTalk•  Automatedvulnerabilitydiscoveryisaverybroadtopic.

•  Thistalkisgoingtobetechnical,butaccessible.

AboutThisTalk•  Automatedvulnerabilitydiscoveryisaverybroadtopic.

•  Thistalkisgoingtobetechnical,butaccessible.– WewilltalkaboutbuildingapracLcalsystem

Outline•  Part1:Learnhowourautomatedbugfindingsystemworks&howtomakeyourown

Outline•  Part1:Learnhowourautomatedbugfindingsystemworks&howtomakeyourown

•  Part2:Understandourapproachtoautomatedbinarypatching

Outline•  Part1:Learnhowourautomatedbugfindingsystemworks&howtomakeyourown

•  Part2:Understandourapproachtoautomatedbinarypatching

•  What’sNext:thefutureofautomatedvulnerabilitydiscovery

IntroducLon•  TrailofBitswasacompeLtorinDARPA’sCyberGrandChallenge

IntroducLon•  TrailofBitswasacompeLtorinDARPA’sCyberGrandChallenge

•  WebuiltCyberdyne,anautomatedbugfindingandpatchingsystem

IntroducLon•  TrailofBitswasacompeLtorinDARPA’sCyberGrandChallenge

•  WebuiltCyberdyne,anautomatedbugfindingandpatchingsystem

•  ItworkedpreFywell…

2ndinBugsFound

77

65

57

57

44

39

23

12

12

9

0 10 20 30 40 50 60 70 80

TeamACyberdyne

TeamBTeamCTeamDTeamETeamFTeamGTeamHTeamI

ConfirmedBugsFound

5thinPatchingEffecLveness

IntroducLon•  TrailofBitswasacompeLtorinDARPA’sCyberGrandChallenge

•  WebuiltCyberdyne,anautomatedbugfindingandpatchingsystem

•  ItworkedpreFywell…•  Butdidn’tqualify 😢

LastinPerformance

AutomatedBugFinding

©flickruserHenriSivonen

VulnerabilityDiscoveryTheory•  Notoolwillfindallthebugs.•  Provablyimpossible.

VulnerabilityDiscoveryTheory•  OverApproximateAnalyses– PointsTo– AliasAnalysis

VulnerabilityDiscoveryTheory•  UnderApproximateAnalyses– Fuzzing,SymbolicExecuLon

Under-ApproximateAnalyses:Roadblocks

HardForFuzzing,EasyforSymbolicExecuLon

if(input[0]==0xBADFOOD)

Under-ApproximateAnalyses:Roadblocks

HardforSymbolicExecuLon,EasyforFuzzing

if(hash(input[0])==hash(input[1]))

Under-ApproximateAnalyses:Theory

•  Alltoolsoperateoverthesamedomain

•  Alldiscoveriesareequallytrue

•  Whatiftoolscouldsharediscoveries?

©flickruserJean-PierreDalbéra

AnalysisBoosLng•  Sharingdiscoveriesacrosstoolscreatesavirtuouscyclethatremovesroadblocks

if(input[0]==0xBADFOOD)if(hash(input[0])==hash(input[1]))BUG();

AnalysisBoosLng•  Sharingdiscoveriesacrosstoolscreatesavirtuouscyclethatremovesroadblocks

if(input[0]==0xBADFOOD)if(hash(input[0])==hash(input[1]))BUG();

AnalysisBoosLng•  Sharingdiscoveriesacrosstoolscreatesavirtuouscyclethatremovesroadblocks

if(input[0]==0xBADFOOD)if(hash(input[0])==hash(input[1]))BUG();

AnalysisBoosLng

• Howdoyoucombineexis1nganalysistools?

AnalysisBoosLng

• Howdoyoucombineexis1nganalysistools?•  ‘Universal’Knowledge:Inputs!– Kindoflame,butitworks!

AnalysisBoosLng•  Inputsgeneratedbyonetoolfeedintoallothers

Inputs

AnalysisBoosLng•  Inputsgeneratedbyonetoolfeedintoallothers

Inputs

Analyses

AnalysisBoosLng•  Inputsgeneratedbyonetoolfeedintoallothers

Inputs

Analyses New Inputs

AnalysisBoosLng•  Inputsgeneratedbyonetoolfeedintoallothers

Inputs

Analyses New Inputs

Merge Knowledge

AnalysisBoosLng:ImplementaLon•  TwosymbolicexecuLonengines– KLEE(viaanx86toLLVMtranslator)– PySymEmu

AnalysisBoosLng:ImplementaLon•  TwosymbolicexecuLonengines– KLEE(viaanx86toLLVMtranslator)– PySymEmu

•  Grr:areallyawesomeDBT-basedfuzzer

AnalysisBoosLng:ImplementaLon•  TwosymbolicexecuLonengines– KLEE(viaanx86toLLVMtranslator)– PySymEmu

•  Grr:areallyawesomeDBT-basedfuzzer•  MergeknowledgeviaMinSet– Minimalsetofmaximalcoverageinputs

PracLcalConsideraLons:MinSet•  MinimizingtheinputsetisessenLalforanalysisboosLngtobepracLcal.

PracLcalConsideraLons:MinSet•  MinimizingtheinputsetisessenLalforanalysisboosLngtobepracLcal.

•  MinSetneedsafastmethodtomeasureandcomparecoverage.

PracLcalConsideraLons:MinSet•  MinimizingtheinputsetisessenLalforanalysisboosLngtobepracLcal.

•  MinSetneedsafastmethodtomeasureandcomparecoverage.

•  Doesn’thavetobeperfect.

AutomatedBugFinding:BuildingIt

©flickruserJuliaManzerova

BuildingYourOwn

A NB

•  Step1:Findthetoolstocombine

BuildingYourOwn•  Step2:Wrapyourtoolswithamanagerscript

A NB

BuildingYourOwn•  Step3•  WriteTests

• WriteTests

• WriteTests

BuildingYourOwn•  Step4:SharedResources

A NB

MessageQueueDistributed Storage

BuildingYourOwn•  Step5:MinSet

A NB

MessageQueueDistributed Storage

BuildingYourOwn•  Step6•  WriteTests

• WriteTests

• WriteTests

BuildingYourOwn•  Step7:MonitoringandStaLsLcs

A NB

MessageQueueDistributed Storage

BuildingYourOwn•  Step8:ControlandProvision

MessageQueueDistributed Storage

MessageQueueDistributed Storage

MessageQueueDistributed Storage

MessageQueueDistributed Storage

MessageQueueDistributed Storage

MessageQueueDistributed Storage

MessageQueueDistributed Storage

MessageQueueDistributed Storage

Control & Provision

ObligatoryAFLComparison

78

69

68

65

61

16

14

0 10 20 30 40 50 60 70 80

Boosted(current)

Boosted(CQE)

AFL(DrillerPaper)

Union(KLEE,PSE,Grr)

Grr(Fuzzing)

PSE(Symbolic)

KLEE(Symbolic)

BugsFoundinCQEBinariesByMethod

(approximate)

Patching

©USArmy

Patching•  Makethebinarynotcrash

Patching•  Makethebinarynotcrash•  PatchingMethod– Whatdoyou“fix”?

Patching•  Makethebinarynotcrash•  PatchingMethod– Whatdoyou“fix”?

•  PatchingMechanics– Howdoyouapplythe“fix”?

Patching•  Makethebinarynotcrash•  PatchingMethod– Whatdoyou“fix”?

•  PatchingMechanics– Howdoyouapplythe“fix”?

•  PatchLocaLon– Wheredoyoufix,ifyouaren’tsureofthebug?

Patching•  PatchingMethod– Preventaccesstoinvalidmemory

Patching•  PatchingMethod– Preventaccesstoinvalidmemory

•  PatchingMechanics– TranslatebinarytoLLVM,patchthebitcode

PatchingviaLLVM

Bin LLVMPatchBin

PatchLLVM

Patching•  PatchingMethod– Preventaccesstoinvalidmemory

•  PatchingMechanics– TranslatebinarytoLLVM,patchthebitcode

•  PatchLocaLon– Specificvs.Generic

PatchLocaLon:SpecificPatching•  So,wehavethisthingthatfindsbugs…– Letsjustpatchthose

•  Insertcheckformemoryvalidity•  Problems– Wrongbug– MulLplecodepaths

PatchLocaLon:GenericPatching•  Fixeverybugofacertainclass•  Examples:– StackCookies– ASLR– CFI

•  Problem:slow

PatchingProblems•  Slow.Bloated.

Patching:EntertheDragon

PatchingMontage•  Cross-BlockDeadStoreEliminaLon

PatchingMontage•  Cross-BlockDeadStoreEliminaLon•  Unusedregisteranalysis

PatchingMontage•  Cross-BlockDeadStoreEliminaLon•  Unusedregisteranalysis•  DominatorTreeTraversal

PatchingMontage•  Cross-BlockDeadStoreEliminaLon•  Unusedregisteranalysis•  DominatorTreeTraversal•  DataFlowfrominputs

PatchingMontage•  Cross-BlockDeadStoreEliminaLon•  Unusedregisteranalysis•  DominatorTreeTraversal•  DataFlowfrominputs•  CombinaLonsoftheabove

Reminder:LastinPerformance

What’sNext?•  CyberGrandChallengeconLnues!•  FinalsareAugust4th,co-locatedwithDEFCON– Freetoenter

•  WinnerwillplaythewinninghumanCTFteamatDEFCON

WhatsNext?•  FindLinuxbugs!

©lewing@isc.tamu.eduandTheGIMP

What’sNext?•  Integratemoretoolsandknowledge!– AFL(viaNCC’sprojecttriforce)– Grammar-basedfuzzers–  InformaLonfromcompiler/sourcecode

•  Broaderlibrarysupport

What’sNext?•  Letsautomate(orsemi-automate)boringassessmentwork.

•  WearedoingapilotauditofzlibfortheMozillaFoundaLon.

•  EventuallywewouldliketoautomaLcallyauditopensourcesosware.

QuesLons?ContactInformaLon:

artem@trailo=its.comhFp://blog.trailo=its.com

FurtherReading:hFp://blog.trailo=its.com/2015/07/15/how-we-fared-in-the-cyber-grand-challenge/hFp://blog.trailo=its.com/2016/01/13/hacking-for-charity-automated-bug-finding-in-libotr/hFps://github.com/trailo=its/mcsemahFps://www.usenix.org/conference/usenixsecurity15/technical-sessions/presentaLon/walker