mostly sunny with a chance of cyber - nist sunny with a chance of cyber david flater...

Mostlysunnywithachanceofcyber1DavidFlater,NIST,2016-05-09

Countingknownvulnerabilitiesandcorrelatingdifferentfactorswiththevulnerabilitytrackrecordsofsoftwareproductsafterthefactisobviouslyfeasible.Theharderchallengeistoproduce“evidencetotellhowvulnerableapieceofsoftwareis”withrespecttovulnerabilitiesandattackvectorsthatarecurrentlyunknown.Thismeansforecastingtheseverityandtherateatwhichcurrentlyunknownvulnerabilitieswillbediscoveredorexploitedinthefuture,givenacandidatesystemanditsenvironment.Meteorologistscanobservethepresentstateofaweathersystemandassumethatthefuturestatemustevolvefromitthroughtheapplicationofknownphysics.Smallfeaturesthatarebelowtheresolutionoftheradararecorrespondinglylimitedintheirimpact,sotheuncertaintycanbebounded.Butforcomputersystemvulnerabilities,therearenoanalogouslimits.High-impactexploitsoftiny,obscurequirksthatwerenotonanyone’s“radar”appearwithregularity.Althoughtheresolutionofthat“radar”iscontinuouslyimproved,thecomplexityofsystemsisincreasingfaster,sotherelevantdetailsareinexorablyrecedingintothebackground.Undertheseconditions,ourbestavailablepredictorsoffuturevulnerabilitiesinsystemsthatwereresponsiblydesignedandimplementedmaybenothingmorethanmetricsofsize,complexity,andtransparency.Unexcitingasitmaybe,thereisrationalitytothisapproach.Todevelopamarketforsmaller,simpler,moreverifiablesystemswouldnotbetoomodestagoalforalargegovernmentefforttoattempt.1Disclaimer:Thisstatementreflectsonlytheviewsoftheauthoronthetopicsdiscussed,anddoesnotnecessarilyreflecttheofficialpositionthatNISTmayhaveaboutthosetopics.

Iaddedthesenotesaftertheworkshoptoincludeimportantpointsthatdon'tappearinthetextoftheslides.

Mostlysunnywithachanceofcyber

DavidFlaterdflater@nist.gov

2016-07-06withnotesadded2016-07-14

1. Thispresentationreflectsonlytheviewsoftheauthoronthetopicsdiscussed,anddoesnotnecessarilyreflecttheofficialpositionthatNISTmayhaveaboutthosetopics.

2. IdentificationofcommercialproductsandentitiesisnotintendedtoimplyrecommendationorendorsementbyNIST,norisitintendedtoimplythattheproductsorentitiesarenecessarilythebestavailableforthepurpose.

Thesis

• Thenatureofthechallengeisnotmeasurement,butprediction• Conditionsareunfavorableformakingarationalprediction• Measuringwhatismeasurableandapplyingempiricismwillmoveusforward• Measuringcost revealsacomplication

Themetrologyperspectiveisthatmeasurementisaboutquantities.Aquantitylike5kghasmeaningbecauseitisdefinedas5timesastandardreference,theunit.InmostcasesitwouldbenonsensetosaythatSoftwareAis5timesasvulnerableasSoftwareB.Vulnerabilityisaquality,notaquantity.Atbestwemaymeasuresomequantitythathelpsustocharacterizeitbetter.

Thecountofknownvulnerabilitiesisunsuitableasasurrogatemeasureofvulnerability.Thefuturequestionisthemostinterestingone.

NISTWorkshoponSoftwareMeasuresandMetrics toReduceSecurityVulnerabilities

Challenge:produce“evidencetotellhowvulnerableapieceofsoftwareis”

SoftwareartifactsDevelopmentandmaintenanceprocesses

Otherartifacts

Somemeasurement process Some[surrogate]measureofvulnerability expressedasamagnitudewithmeaningfulunitsandaconfidenceinterval

Measurementvs.forecasting• Past:correlatedifferentfactorswiththevulnerabilitytrackrecordsofsoftwareproducts• Present:countknownvulnerabilities

• Abuseofscale:count=2doesnotmeantwiceasvulnerableascount=1;anycount>0meansgofixyourstuff

• Future:forecasttheseverityandtherateatwhichcurrentlyunknownvulnerabilitieswillbediscoveredorexploited• Nolongerdeterminingfactsbasedonobservations• Notcausal:intheory,today'sCVEcouldbethelast• Predictionmodelscanbebetterorworse

Ineveryrespectbutone(controllability),cyberemergenciesarelesspredictablethanweatheremergencies.Iwillfocusonthedifferentimpactofunseendetails.

Wecanobtainanadequatepredictionofimpendingweatheremergencieseventhoughtheradarmissesmanysmalldetails.Thebutterflyeffectsdonotmatteraslongaswecanseethehurricaneonitswaywithampletimetoreact.Butforcyberemergenciesitisexactlytheopposite;itistheunseendetailsthataremostlikelytocreateanemergencywithnowarningatall.

PredictionmodelsAttribute Weatheremergencies Cyber emergenciesPreconditions Known Unknown,randomConditions Taketimetoevolve Alreadyin placeSet ofvariables Fixed Ever-expandingUnseendetails Not important CriticallyimportantGuidance Unguided Precision-guidedUncertainty Frequentist EpistemicDegrees ofcontrol Prepare,mitigate Preventable,inprinciple?

Thethreatmodelisoffinitesize.Theunknownuniverseofpotentialattacksmaybeinfinitelylarge.Atleastitislargerthanourimagination,asweareconsistentlycaughtbysurprise.

Theideathatfullyaddressingthetop10ortop25attackvectorswouldcausetheretobefewersuccessfulattacksisanuntestedhypothesis.Pastexperiencesuggeststhatthereisalargereserveofattackvectorsthatdonotappearinthethreatmodel.Perhapsattackerswillsimplymovefartherdownthelistandneverrunoutofattacks.

Differentperspectives,differentmetrics:thesecurityindustryseesprogressinincreasingthecomplexityofattacks,butthetargetseesnoprogressunlessthefrequencyofattacksactuallygoesdown.

Unseendetails=blindsideattackvectors

• Electricalengineers• Memoryintegrityquietlydeclined,enablingrowhammer.js

• Implementationquirk,documentedbutoverlooked• Intelimplementedanx86_64instructioninaslightlydifferentwaythanAMDhad,enablingVMescapeandescalatetohypervisor(XSA-7)

• Unforeseenconsequenceofnewfeature• Memorydeduplicationbecameathing,enablingamuchbiggersidechannelthanwasanticipated(Bosmanetal.2016)

• Forgotaboutthatlegacyfeature• EveryoneforgotaboutAPICregisterrelocationorfailedtoseeitsusefulness,enablinganotherescalationtoSMM(Domas 2015)

• Accidentallyintroducedfault• ArandomCPUerratumwasdiscovered,enablingaremoteexploitthatlookslikeharmlesscode(Kaspersky&Chang2008)

Wheredotheycomefrom?Everywhere.

Evenifyouhadcompletevisibilityintothesystemasitstands,thereistheproblemoffuture-proofingtheassurancecase.Weareforcedtoupgradeinordertoclosethebarndooronknownvulnerabilities.Eachupgradecomeswithanexpandedattacksurface,whichleadsdirectlytonewvulnerabilities.

Ariskmodelcannotdojusticetounknownunknowns.Wecannotpossiblyestimatetheprobabilityofsomethingthat,bydefinition,weknowabsolutelynothingabout.Suchanumberisnothingbutanarbitrarilychosensafetymargin.

Thefuturewillnotbemitigated• Anassurancecaseisafixed,closed-formexpression upagainstanevolving,openworld• Theunseenattacksurfaceisvastandgrowing• Noopt-out

Riskmodelsvs.unknownunknowns

• "Risks"• Validtoestimatebasedonhistoricaldata

• "Structuraluncertainties"• Followfromeventsthatarerareornonexistentinthehistoricalrecord

• Frequentistreasoningbreaksdown• "Unknowables"• Followfrominconceivableevents• Bayesianreasoningbreaksdown

Kees vanderHeijden. Scenarios:TheArtofStrategicConversation.JohnWiley&Sons,2ndedition,2005.

Securitymaygrowovertimeintightly-controlledsystems,butthemoretypicaltreadmillofvulnerabilitiesandmitigationssuggeststhatitdoesnotgrowovertimeingeneral.(Takingthetarget'sperspectivethatthedifficultyofexploitsisirrelevantiftheyjustkeeponhappening.)

Inventingametricisonlythebeginning.Hypothesesmustbetested.Measurementsmustbevalidated.

Growthmodels• Noevidencethatsecuritygrows/vulnerabilitydecreasesovertime(?)• "Trivialforecasthassome predictiveaccuracy"(TimmGrams,"ReliabilityGrowthModelsCriticized")• Applicabletothefrequencyofvulnerabilitydiscovery

Whatismeasurable?• Knownquantities• Trackrecordoffixedvulnerabilities• Knownunfixedvulnerabilities• Measurablehardnessofcertainkindsofdefenses

• Hypothesizedindicatorsofunknownvulnerabilities• Measuresofdiligence

• Test/analysiscoverage• Hardeningmeasures

• Size&complexity• Areaofattacksurface• "Codesmells"(operationalized)• Transparency(includingamenabilitytoanalysisofwhateverkind)

Thisargumentisnotvalidforproductswhoseprimarycustomeristhegovernment,forregulatedindustries,orforlong-lifecyclesoftware.Itappliesonlytothemassmarket.

Wearefamiliarwithstudiesshowingthatthecostofcorrectingdefectsislessiftheyaredetectedandcorrectedearlierintheprocess.Butaslongasthemarkettoleratesfaultysoftware,theproducer'scostcanbeloweredfurtherbyjustnevercorrectingthedefects.Alotofsoftwareisbeingproducedasaconsumable(oraspartofaconsumable)ratherthanadurablegood.Maintenanceisminimized,andafteradatecertaintheproductissimplyabandonedandthenextproductisrolledout.

Withinthemassmarket,thecostofpoorsecuritymayevengonegative:amoresecureproductmaybetoodifficulttoconfigure,resultinginacompetitivedisadvantage.Evenifthecostofbuildingsecurityinisreducedtomarginalasthestrategicplanenvisions,thebusinesscasemayremainbroken.

Thiseconomicproblemmayoverwhelmandobviatethemeasurementproblem.

Onmeasuringcost,andtheproblemthatthisreveals• "Priceofnonconformance"(PhilipCrosby)orCostOfPoorQuality(ASQ)• Post-releasepatchingismuchlesscostlythananautorecall• TheconsequentialcostsofvulnerabilitiesinCOTSsoftwarearealmostentirelypaidbyconsumers,not producers

• "Qualityisfree"—nottrue• "Youcan'taffordnot totest/buildsecurityin"—alsonottrue• Brokeneconomy• Consequence:theremaybenosecurityto'measure'

Empiricismisausefulstrategywhenweareoverwhelmedbyunknowns,butitmustbeusedwithgreatcaution.Correlationisnotcausation.Agoodfittopastdatadoesnotensureagoodprediction.Hypothesesmustbetested.Measurementsmustbevalidated.Applyscience.

Notaddressed:wealsoneedsoftwaretobesufficientlyfunctionalrunningatleastprivilegethattrickingusersintograntingexcesspermissionstotrojanswillnolongerwork.

Conclusions

• Thereisvalueincorrelatingdifferentfactorswiththevulnerabilitytrackrecordsofsoftwareproductsafterthefact• Hypothesizedindicators• Programminglanguages• Developmenttechniques• Qualityprocesses• Formalmethods….

• Engineeringwasn'tinvented;itevolved

• Dowhat[apparently]works,butverifyandtrackprogress

• Goal:reliablepredictors,bestpractices

• However,therealsoneedstobeabusinesscase

• Redistributingriskmaybenecessaryto"significantlycurtailsoftwarevulnerabilities"intheCOTSmarket

"Measurewhatismeasurable,andstopyer lyin'abouttherest"

(MisquotingGalileo)

SoftwareMetrologyDavidFlater

dflater@nist.gov