![Page 1: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/1.jpg)
BeyondMatching:ApplyingdatasciencetechniquestoIOC-baseddetection
(#BeyondMatching)
AlexPinto- ChiefDataScientist– Niddel@alexcpsec@NiddelCorp
![Page 2: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/2.jpg)
• SecurityDataScientist• CapybaraEnthusiast• Co-FounderandChiefDataScientistatNiddel(@NiddelCorp)
• LeadofMLSec Project(@MLSecProject)
WhoamI?
• WhatisaNiddel?• NiddelprovidesaSaaS-basedAutonomousThreatHuntingSystem• ResearchfromthistalkwasperformedusinganonymizedNiddeldataandusesconceptsimplementedonitsproducts.• Notavendor-centrictalk,focusonlearningandy’all toreproducethis.
![Page 3: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/3.jpg)
• ThePromiseofIOCs• 7 HabitsofHighlyEffectiveAnalysts(ok,only3)
• Nation-StateAPTDetectionDeluxeRecipe• DataSciencetoAssistonPivoting• MaliciousnessRatio• MaliciousnessRating
• RevisitingTIQ-TEST– TelemetryTest
Agenda
![Page 4: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/4.jpg)
ThePromiseofIOCs
Ifyouhaven’timplementedThreatIntelligencefeedsonyourorganization
Iwillrevealtheendingofyourupcominggruelingjourney
Apologiesinadvance
![Page 5: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/5.jpg)
Promise- SomeDefinitionsFirst• IOCs:Indicatorsofcompromise• CTI:CyberThreatIntelligence
• Willbeusingtheminterchangeablyduringthispresentation
• IOCs->technicaldatathatallowsfor”tactical”discoveryofapotentialcompromiseonasystem
• WewillbefocusingonnetworkIOCsonthistalk
LittleBobbyComicsby@RobertMLee andJeffHaas
![Page 6: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/6.jpg)
Promise– SoundsGreat!Signmeup!• Notsofast,myfriend• MainchallengeswithIOCsconsumption:• QualityandCuration
• Vettingandqualitycontrol• OpenfeedsvsPaidfeeds• ManualvsAutomated
• VelocityandVolume• Howtooperationalize?• AddtoSIEM?• BlockinFirewall/WebProxy?
![Page 7: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/7.jpg)
Promise– QualityandVelocityatOdds• AIS– ThreatIntelsharinginitiativefrom
USDepartmentofHomelandSecurity
• Ifullysupportsharing(seepreviousintelsharingdecksfrom2015)
• Butifweareresignedtothislevelofquality,”itiswhatitis”,howcanCTI/IOCsbeshapedintoausefultoolatscale?
![Page 8: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/8.jpg)
Promise– CurrentImplementationStrategies1. AlertingbasedonmatchingwithIOCdata:• Bybeingcareful,onlymatchingonmore”precise”indicators(URLs>>IPs),
youcanreducenumberofFalsePositives,butstillchallenging
2. UsingIOCdatatobuildcontextforexistingalerts:• Saferbet,butyouarenotaddinganydetectionpowertoexistingcontrols
SPOILER ALERT: Everyone starts with (1) because ”the FPs can’t be that bad”, and then begrudgingly moves to (2) because there is not enough time in the world to go through all the
noise that (1) generates.
![Page 9: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/9.jpg)
SadIntermission
DISCLAIMER:Could not find a picture of a sad capybara. Not sure there is one.
![Page 10: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/10.jpg)
Whatmakesanalystseffective?• Theylearnfromtheexamples!!
• Theydon’tlookatIOCsasa”finishedproduct”,butasawaytolearnfromtheattackerinfrastructure.
• Afterunderstandingandresearchonsamplesofdata,theycanextrapolatetheTTPs(Tactics,TechniquesandProcedures)oftheattackerstobuilddefenses.
PyramidofPainfrom@DavidJBianco
![Page 11: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/11.jpg)
InternetInfrastructure101
Actually, ”everything” is connected
![Page 12: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/12.jpg)
Nation-StateAPTDetectionDeluxeRecipeWhenyour”favoriteIRcompany”blamesFROSTYPENGUINforanattack:1. Findapieceofmalwareoncompromisedorganization2. Extract”non-benign”placestheyconnectto(realworkhere,BTW)3. PivotonInternetInfrastructuretofindrelatedIPs/Domains/URLs4. Searchfortheseonorg,findmoremalware(Hunting,FTW!)5. RepeatSteps1-4untilnomorenewmalware6. Remediateorganization(hopefully!)7. Publishreportorblogposttogreatfanfare8. PROFIT(oratleastmediaattentionandsalesleads)
![Page 13: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/13.jpg)
DataSciencetoAssistonPivoting• Doingitourselves:- Beginwithdatacollection1. GetIOCsfromyourfavorite/availableproviders– thereareafewoptions
thatarefairlygood.Pleasedoselectaccordingtocollectioncriteria.2. ”Enrich”thedatatogatherthe”pivotpoints”andfindtheconnections.
Combine (https://github.com/mlsecproject/combine) can help with IOC gathering and enrichment for ASN data and pDNS (if you have a Farsight pDNS key)
• IPAddresses:• ASnumber• BGPprefix• Country• pDNS relationshiptodomains
• Domainnames:• pDNS relationshiptoIPs• WHOISRegistrations• SOA• NSServers
![Page 14: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/14.jpg)
DataCollection– ExampleWithRIGEKWHOISregistrante-mailonasmallsampleofRIGEKdomainsonOct2016:
![Page 15: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/15.jpg)
![Page 16: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/16.jpg)
DataCollection– ExampleWithRIGEKThisoneisNOTDomainShadowing– activeactorregisteringe-mails:
![Page 17: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/17.jpg)
DataCollection– ExampleWithRIGEKAutonomousSystem/CountryofIPsarelocated,RIGEKsample– Oct2016:
![Page 18: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/18.jpg)
DataCollection– ExampleWithRIGEKAutonomousSystemwhereIPsarelocated,RIGEKsample– Oct2016:
![Page 19: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/19.jpg)
DataAggregation– RigEKExample
In summary: let’s create different graphs for each one of the pivoting points and measure the cardinality of the node connectedness
AS48096- ITGRAD
AS16276– OVHSASL
AS14576– HostingSolutionLtd(actuallyking-servers.com)
![Page 20: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/20.jpg)
DataAggregation– ContextMatters
• Whatifmyfavoritewebsitesareactuallyhostedatthosepivotingpoints?• Imean,thereareafew”ok”thingson.comand.org
![Page 21: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/21.jpg)
MaliciousnessRatioLet’sbuildsimilaraggregationmetricsfor”goodplaces”yourorganizations
Weproposearatiothatcomparesthecardinalityofthenodeconnectedness:• Bpp – countof”badentities”connectedtoaspecificpivotingpoint• Gpp – countof”goodentities”connectedtoaspecificpivotingpoint
𝑀𝑅## = &''
('')&''
![Page 22: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/22.jpg)
Holdon!!GoodPlacesontheInternet?• CreatingandmaintainingwhitelistsisMUCHHARDERthanblacklists
• Sometips:• Useyourowntelemetry- giventhebaseratefallacy,placesthat”everyone”
goestoaremorelikelytobebenign• Raritydoesnotmeanbad(shutup,UEBApeople),buthighvisitationalmost
alwaysmeangood• Harvestdatafromyourownsecuritytools,likewebfilters(ifyoutrustthem)• VeryshallowscoopsofAlexaTopSites.Very.Shallow.
![Page 23: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/23.jpg)
MaliciousnessRatio– Examples• TelemetryfromanpoolofNiddelcustomers:
• AS48096– ITGRAD 87.5%• CountryRU 5.2%• .orgTLD 2.9%
• Lookingatthebaserate:• ASNBaseRate 0.6%• CountryBaseRate 0.58%• TLDBaseRate 1.9%
• SevereoutliersbelowbaseratemayindicatethattheIOCisinvalid
![Page 24: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/24.jpg)
MaliciousnessRating• Aratiofrom0to1canbecoolformathpeople,buthowriskyarethose
thingsanyway?• Weneedtocompareittothebaseratetohaveagoodmeasure• Weproposeamaliciousnessratingwhichexpresshowmuchmorelikelyto
bebadtheconnectionwithaspecificpivotingpointthananaveragepivotingpointofthatkindontheInternet.
𝑀𝑅𝑇## =𝑀𝑅##
∑ 𝑀𝑅##(-)/-01 𝑛3
![Page 25: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/25.jpg)
MaliciousnessRating– SampleDistributions
![Page 26: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/26.jpg)
ChallengeswiththeApproach• Howcanwebestdefinethecuttingscoresonallthosepotential
maliciousnessratings?• Howtocombineandweightthemultivariatecompositionofthesepivoting
points?
• Solutionisprobablyuniquepercompany,includingunderstandingtelemetrypatterns,riskappetiteforFPs/FNsanddecisionpointsonwhentoblockandwhentoalertonsomething.
![Page 27: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/27.jpg)
Whatifthechallengeshadbeensolved?
![Page 28: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/28.jpg)
AMoreInvolvedExample(1)
![Page 29: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/29.jpg)
AMoreInvolvedExample(2)
Buildthecampaignbasedontherelationships- theyallsharethesamesupportinfrastructureontheIPAddressandNameServers.
![Page 30: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/30.jpg)
ShiaLeBeouf Approves
![Page 31: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/31.jpg)
Onemorething…
![Page 32: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/32.jpg)
GoingbacktoTIQ-Test• BiggestcriticismofTIQ-Test(mostlyself-inflicted)isthatiswasalwaysrelative,notabsolute.
• Howcanonedefinewhatita”good”feed?• Doesthatevenmakesense?• Itiseasytotellifafeedisbad(lotsofFPs,lowcuration)
• Mythoughtprocess:• Maybe withtelemetry,youcanidentifyan”applicable”feed• Or”actionable”ifyoulikeyourCybersecuritywithextracamo
![Page 33: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/33.jpg)
ActualalertIOC
accounting
Percentageofthematchesofanspecificfeedthatwereactualalertsorincidentsatanorganization
![Page 34: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/34.jpg)
ActualalertUNIQUEIOCaccounting
PercentageofUNIQUE(onlycontributedbythefeed)matchesofanspecificfeedthatwereactualalertsorincidentsatanorganization
![Page 35: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/35.jpg)
ChallengeswiththeApproach(2)• Howdoesonedefineavalidalertorincident?• NotmanywaysbuttoimproveunderstandingandgrowthofIRpractice:• Yourownincidenthistory(forthe1%-ers)• YourownCTI/IOCcreationprocesses(forthe0.01%-ers)
• The”TelemetryTest”hasbeenINVALUABLEforNiddelonpartnershipandfeedselection
• ”MyThreatIntelligenceCanBeatUpYourThreatIntelligence”(h/tRickHolland)
• Howmuchvaluesdoesafeedaddanyway?Lookforuniquecontributions.
![Page 36: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/36.jpg)
Nomagicthistime– ImproveyourIRprocesses
![Page 37: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/37.jpg)
Takeaways• Lotsofideastoimplement,gogogo!!• IOCs(andCTIingeneralforthatmatter)arenotacompletewasteoftime.It’sjustrawdata,andneedstoberefinedinordertobeusedproperly
• Bringingautomation(andsimplicityofuse)tothreatintelligenceandthreathuntingisparamounttobringitsusabilityfromthe1%oforgstoamorebroadaudienceatscale
![Page 38: Beyond Matching: Applying Data Science Techniques to IOC-based Detection](https://reader031.vdocuments.us/reader031/viewer/2022030214/5899a02c1a28ab30688b4cb3/html5/thumbnails/38.jpg)
Thanks!• Share,like,subscribe,EDMoutro• Q&AandFeedbackplease!
AlexPinto– [email protected]@alexcpsec@NiddelCorp
LittleBobbyComicsby@RobertMLee andJeffHaas