threat hunting workshop
Post on 13-Apr-2017
22 Views
Preview:
TRANSCRIPT
ThreatHuntingwithSplunkPresenter:LeeImreySplunk,SecurityMarketSpecialist
Prework fortoday
● SetupSplunk EnterpriseSecuritySandbox● InstallfreeSplunk onlaptop● InstallMLToolkitapp
https://splunkbase.splunk.com/app/2890/
Agenda• ThreatHuntingBasics
• ThreatHuntingDataSources
• Sysmon EndpointData
• CyberKillChain
• WalkthroughofAttackScenarioUsingCoreSplunk (handson)
• AdvancedThreatHuntingTechniques&SecurityEssentials
• EnterpriseSecurityWalkthrough
• ApplyingMachineLearningandDataSciencetoSecurity
LogInCredentials
January,February&March https://54.144.69.125April,May&June https://52.55.68.96July and August https://54.164.82.160SeptemberandOctober https://52.23.227.212NovemberandDecember https://52.202.90.207
User:hunterPass:pr3dat0r
BirthMonth
Thesewon’twork…
AmIintherightplace?
Somefamiliaritywith…
● CSIRT/SOCOperations
● GeneralunderstandingofThreatIntelligence
● GeneralunderstandingofDNS,Proxy,andEndpointtypesofdata
6
Whatisthreathunting,whydoyouneedit?TheWhat?
• Threathunting- theactofaggressively
intercepting,trackingand
eliminatingcyberadversariesasearlyaspossible intheCyberKillChain 2
7
TheWhy?
• Threatsarehuman.Focusedandfundedadversarieswillnotbecounteredbysecurityboxesonthenetwork
alone.Threathuntersareactivelysearchingforthreatstopreventor
minimizedamage[beforeithappens] 1
2 CyberThreatHunting- SamuelAlonsoblog,Jan20161 TheWho,What,Where,When,WhyandHowof EffectiveThreatHunting,SANSFeb2016
“ThreatHuntingisnotnew,it’sjustevolving!”
ThreatHuntingwithSplunk
9
Vs.
Search&Visualisation
Enrichment
Data
Automation
10
HumanThreatHunter
KeyBuildingBlockstoDriveThreatHuntingMaturity
Ref:TheheWho,What,Where,When,WhyandHowof EffectiveThreatHunting,SANSFeb2016
Objectives> Hypotheses> Expertise
“Agoodintelligenceofficercultivatesanawarenessofwhatheorshedoesnotknow.Youneedadoseofmodestytoacknowledgeyourownignorance- evenmore,toseekoutyourignorance.Thentheharderpartcomes,tryingtodosomethingaboutit.Thisoftenrequiresanimmodestdetermination”HenryA.CrumptonTheArtofIntelligence:LessonsFromAlifeIntheCIA’sClandestineService
11
SANSThreatHuntingMaturity
12
AdHocSearch
StatisticalAnalysis
VisualizationTechniques
Aggregation MachineLearning/DataScience
85%55%50%48%32%
Source:SANSIR&ThreatHuntingSummit2016
Search&Visualisation
Enrichment
Data
Automation
HumanThreatHunter
HowSplunkhelpsYouDriveThreatHuntingMaturity
ThreatHuntingAutomationIntegrated&outoftheboxautomationtoolingfromartifactquery,contextual“swim-laneanalysis”,anomaly×eriesanalysistoadvanceddatascienceleveragingmachinelearning
ThreatHuntingDataEnrichmentEnrichdatawithcontextandthreat-intelacrossthestackortime
todiscerndeeperpatternsorrelationships
Search&VisualiseRelationshipsforFasterHuntingSearchandcorrelatedatawhilevisuallyfusingresultsforfaster
context,analysisandinsight
Ingest&OnboardAnyThreatHuntingMachineDataSourceEnablefastingestionofanymachinedatathroughefficient
indexing,abigdatarealtimearchitectureand‘schemaontheread’technology
Hypotheses
AutomatedAnalytics
DataScience&MachineLearning
Data&IntelligenceEnrichment
DataSearch
Visualisation
Maturity
HuntingTools:InternalData
14
• IPAddresses:threatintelligence,blacklist,whitelist,reputationmonitoringTools:Firewalls,proxies,Splunk Stream,Bro,IDS
• NetworkArtifactsandPatterns:networkflow,packetcapture,activenetworkconnections,historicnetworkconnections,portsandservicesTools:Splunk Stream,BroIDS,FPC,Netflow
• DNS:activity,queriesandresponses,zonetransferactivityTools:Splunk Stream,BroIDS,OpenDNS
• Endpoint– HostArtifactsandPatterns:users,processes,services,drivers,files,registry,hardware,memory,diskactivity,filemonitoring:hashvalues,integritycheckingandalerts,creationordeletionTools:Windows/Linux,CarbonBlack,Tanium,Tripwire,ActiveDirectory
• VulnerabilityManagementDataTools:TripwireIP360,Qualys,Nessus
• UserBehaviorAnalytics:TTPs,usermonitoring,timeofdaylocation,HRwatchlistSplunk UBA,(Alloftheabove)
Persist,Repeat
ThreatIntelligence
Access/Identity
Endpoint
Network
Attacker,knowrelay/C2sites,infectedsites,IOC,attack/campaignintentandattribution
Wheretheywentto,whotalkedtowhom,attacktransmitted,abnormaltraffic,malwaredownload
Whatprocessisrunning(malicious,abnormal,etc.)Processowner,registrymods,attack/malwareartifacts,patchinglevel,attacksusceptibility
Accesslevel,privilegedusers,likelihoodofinfection,wheretheymightbeinkillchain
• Third-partythreatintel• Open-sourceblacklist• Internalthreatintelligence
• Firewall,IDS,IPS• DNS• Email
• Endpoint(AV/IPS/FW)• Malwaredetection• PCLM
• DHCP• OSlogs• Patching
• ActiveDirectory• LDAP• CMDB
• Operatingsystem• Database• VPN,AAA,SSO
TypicalDataSources
• Webproxy• NetFlow• Network
Endpoint:MicrosoftSysmonPrimer
16
● TAAvailableontheAppStore
● GreatBlogPosttogetyoustarted
● IncreasesthefidelityofMicrosoftLogging
BlogPost:http://blogs.splunk.com/2014/11/24/monitoring-network-traffic-with-sysmon-and-splunk/
LogInCredentials
January,February&March https://54.144.69.125April,May&June https://52.55.68.96July and August https://54.164.82.160SeptemberandOctober https://52.23.227.212NovemberandDecember https://52.202.90.207
User:hunterPass:pr3dat0r
BirthMonth
SysmonEventTags
18
MapsNetworkCommtoprocess_id
Process_idcreationandmappingtoparentprocess_id
sourcetype=X*|searchtag=communicate
19
sourcetype=X*|deduptag|searchtag=process
20
DataSourceMapping
DemoStory- KillChainFrameworkSuccessfulbruteforce– downloadsensitivepdfdocument
WeaponizethepdffilewithZeusMalware
Convincingemailsentwithweaponizedpdf
Vulnerablepdfreaderexploitedbymalware.Droppercreatedonmachine
Dropperretrievesandinstallsthemalware
Persistenceviaregularoutboundcomm
DataExfiltration
Source:LockheedMartin
Servers
Storage
DesktopsEmail Web
TransactionRecords
NetworkFlows
DHCP/DNS
HypervisorCustomApps
This image cannot currently be displayed.
PhysicalAccess
Badges
ThreatIntelligence
Mobile
CMDB
This image cannot currently be displayed.
IntrusionDetection
Firewall
DataLossPrevention
Anti-Malware
VulnerabilityScans
Traditional
Authentication
StreamInvestigations– chooseyourdatawisely
23
APTTransactionFlowAcrossDataSources
24
http(proxy)sessiontocommand&controlserver
RemotecontrolStealdataPersistincompanyRentasbotnet
Proxy
ConductBusiness
Createadditionalenvironment
GainAccesstosystemTransaction
ThreatIntelligence
Endpoint
NetworkEmail,Proxy,DNS,andWeb
DataSources
.pdf executes&unpacksmalwareoverwritingandrunning“allowed”programs
Svchost.exe(malware)
Calc.exe(dropper)
AttackerhackswebsiteSteals.pdf files
WebPortal.pdf
Attackercreatesmalware,embed in.pdf,
emailstothetarget
Reademail,openattachment
OurInvestigationbeginsbydetectinghighriskcommunicationsthroughtheproxy,attheendpoint,andevenaDNScall.
index=zeus_demo3
25
insearch:
Tobeginourinvestigation,wewillstartwithaquicksearchtofamiliarizeourselveswiththedatasources.
Inthisdemoenvironment,wehaveavarietyofsecurityrelevantdataincluding…
WebDNSProxyFirewallEndpointEmail
Takealookattheendpointdatasource.WeareusingtheMicrosoftSysmon TA.
Wehaveendpointvisibilityintoallnetworkcommunicationandcanmapeachconnectionbacktoaprocess.
}Wealsohavedetailedinfooneachprocessandcanmapitbacktotheuserandparentprocess.}
Letsgetourdaystartedbylookingusingthreatintel toprioritizeoureffortsandfocusoncommunicationwithknown highriskentities.
WehavemultiplesourceIPscommunicatingtohighriskentitiesidentifiedbythese2threatsources.
Weareseeinghighriskcommunicationfrommultipledatasources.
Weseemultiplethreatintel relatedeventsacrossmultiplesourcetypesassociatedwiththeIPAddressofChrisGilbert.Let’stakecloserlookattheIPAddress.
Wecannowseetheownerofthesystem(ChrisGilbert)andthatitisn’taPIIorPCIrelatedasset,sotherearenoimmediatebusinessimplicationsthatwouldrequireinformingagenciesorexternalcustomerswithinacertaintimeframe.
Thisdashboardisbasedoneventdatathatcontainsathreatintelbasedindicatormatch(IPAddress,domain,etc.).ThedataisfurtherenrichedwithCMDBbasedAsset/identityinformation.
Wearenowlookingatonlythreatintel relatedactivityfortheIPAddressassociatedwithChrisGilbertandseeactivityspanningendpoint,proxy,andDNSdatasources.
Thesetrendlinestellaveryinterestingvisualstory.ItappearsthattheassetmakesaDNSqueryinvolvingathreatintel relateddomainorIPAddress.
ScrollDo
wn
Scrolldownthedashboardtoexaminethesethreatintel eventsassociatedwiththeIPAddress.
Wethenseethreatintel relatedendpointandproxyeventsoccurringperiodicallyandlikelycommunicatingwithaknownZeusbotnetbasedonthethreatintelsource(zeus_c2s).
It’sworthmentioningthatatthispointyoucouldcreateatickettohavesomeonere-imagethemachinetopreventfurtherdamageaswecontinueourinvestigationwithinSplunk.
Withinthesamedashboard,wehaveaccesstoveryhighfidelityendpointdatathatallowsananalysttocontinuetheinvestigationinaveryefficientmanner.Itisimportanttonotethatnearreal-timeaccesstothistypeofendpointdataisnotnotcommonwithinthetraditionalSOC.
Theinitialgoaloftheinvestigationistodeterminewhetherthiscommunicationismaliciousorapotentialfalsepositive.Expandtheendpointeventtocontinuetheinvestigation.
Proxyrelatedthreatintel matchesareimportantforhelpingustoprioritizeoureffortstowardinitiatinganinvestigation.Furtherinvestigationintotheendpointisoftenverytimeconsumingandofteninvolvesmultipleinternalhand-offstootherteamsorneedingtoaccessadditionalsystems.Thisencryptedproxytrafficisconcerningbecauseofthelargeamountofdata(~1.5MB)beingtransferredwhichiscommonwhendataisbeingexfiltrated.
Exfiltrationofdataisaseriousconcernandoutboundcommunicationtoexternalentitythathasaknownthreatintelindicator,especiallywhenitisencryptedasinthiscase.
Letscontinuetheinvestigation.
Anotherclue.Wealsoseethatsvchost.exe shouldbelocatedinaWindowssystemdirectorybutthisisbeingrunintheuserspace.Notgood.
Weimmediatelyseetheoutboundcommunicationwith115.29.46.99viahttpsisassociatedwiththesvchost.exeprocessonthewindowsendpoint.Theprocessidis4768.ThereisagreatdealmoreinformationfromtheendpointasyouscrolldownsuchastheuserIDthatstartedtheprocessandtheassociatedCMDBenrichmentinformation.
WehaveaworkflowactionthatwilllinkustoaProcessExplorerdashboardandpopulateitwiththeprocessidextractedfromtheevent(4768).
ThisisastandardWindowsapp,butnotinitsusualdirectory,tellingusthatthemalwarehasagainspoofedacommonfilename.
Wealsocanseethattheparentprocessthatcreatedthissuspicuous svchost.exe processiscalledcalc.exe.
ThishasbroughtustotheProcessExplorerdashboardwhichletsusviewWindowsSysmon endpointdata.
SuspectedMalware
Letscontinuetheinvestigationbyexaminingtheparentprocessasthisisalmostcertainlyagenuinethreatandwearenowworkingtowardarootcause.
ThisisveryconsistentwithZeusbehavior.TheinitialexploitationgenerallycreatesadownloaderordropperthatwillthendownloadtheZeusmalware.Itseemslikecalc.exemaybethatdownloader/dropper.
SuspectedDownloader/Dropper
Thisprocesscallsitself“svchost.exe,”acommonWindowsprocess,butthepathisnotthenormalpathforsvchost.exe.
…whichisacommontraitofmalwareattemptingtoevadedetection.WealsoseeitmakingaDNSquery(port53)thencommunicatingviaport443.
TheParentProcessofoursuspecteddownloader/dropperisthelegitimatePDFReaderprogram.Thiswilllikelyturnouttobethevulnerableappthatwasexploitedinthisattack.
SuspectedDownloader/Dropper
SuspectedVulnerableAppWehaveveryquicklymovedfromthreatintel relatednetworkandendpointactivitytothelikelyexploitationofavulnerableapp.Clickontheparentprocesstokeepinvestigating.
WecanseethatthePDFReaderprocesshasnoidentifiedparentandistherootoftheinfection.
ScrollDo
wn
ScrolldownthedashboardtoexamineactivityrelatedtothePDFreaderprocess.
Chrisopened2nd_qtr_2014_report.pdfwhichwasanattachmenttoanemail!
Wehaveourrootcause!Chrisopenedaweaponized .pdf filewhichcontainedtheZeusmalware.Itappearstohavebeendeliveredviaemailandwehaveaccesstoouremaillogsasoneofourimportantdatasources.Letscopythefilename2nd_qtr_2014_report.pdfandsearchabitfurthertodeterminethescopeofthiscompromise.
Letsdigalittlefurtherinto2nd_qtr_2014_report.pdftodeterminethescopeofthiscompromise.
index=zeus_demo32nd_qtr_2014_report.pdf
38
insearch:
Letssearchthoughmultipledatasourcestoquicklygetasenseforwhoelsemayhavehavebeenexposedtothisfile.
Wewillcomebacktothewebactivitythatcontainsreferencetothepdf filebutletsfirstlookattheemaileventtodeterminethescopeofthisapparentphishingattack.
Wehaveaccesstotheemailbodyandcanseewhythiswassuchaconvincingattack.Thesenderapparentlyhadaccesstosensitiveinsiderknowledgeandhintedatquarterlyresults.
Thereisourattachment.
HoldOn!That’snotourDomainName!Thespellingisclosebutit’smissinga“t”.TheattackerlikelyregisteredadomainnamethatisveryclosetothecompanydomainhopingChriswouldnotnotice.
Thislookstobeaverytargetedspearphishingattackasitwassenttoonlyoneemployee(Chris).
RootCauseRecap
41
DataSources
.pdf executes&unpacksmalwareoverwritingandrunning“allowed”programs
http(proxy)sessiontocommand&controlserver
RemotecontrolStealdataPersistincompanyRentasbotnet
Proxy
ConductBusiness
Createadditionalenvironment
GainAccesstosystemTransaction
ThreatIntelligence
Endpoint
NetworkEmail,Proxy,DNS,andWeb
.pdfSvchost.exe(malware)
Calc.exe(dropper)
AttackerhackswebsiteSteals.pdf files
WebPortal.pdf
Attackercreatesmalware,embed in.pdf,
emailstothetarget
Reademail,openattachment
Weutilizedthreatintel todetectcommunicationwithknownhighriskindicatorsandkickoffourinvestigationthenworkedbackwardthroughthekillchaintowardarootcause.
Keytothisinvestigativeprocessistheabilitytoassociatenetworkcommunicationswithendpointprocessdata.
ThishighvalueandveryrelevantabilitytoworkamalwarerelatedinvestigationthroughtorootcausetranslatesintoaverystreamlinedinvestigativeprocesscomparedtothelegacySIEMbasedapproach.
42
Letsrevisitthesearchforadditionalinformationonthe2nd_qtr_2014-_report.pdffile.
Weunderstandthatthefilewasdeliveredviaemailandopenedattheendpoint.Whydoweseeareferencetothefileintheaccess_combined (webserver)logs?
Selecttheaccess_combinedsourcetype toinvestigatefurther.
43
Theresultsshow54.211.114.134hasaccessedthisfilefromthewebportalofbuttergames.com.
ThereisalsoaknownthreatintelassociationwiththesourceIPAddressdownloading(HTTPGET)thefile.
44
SelecttheIPAddress,left-click,thenselect“Newsearch”.WewouldliketounderstandwhatelsethisIPAddresshasaccessedintheenvironment.
45
That’sanabnormallylargenumberofrequestssourcedfromasingleIPAddressina~90minutewindow.
Thislookslikeascriptedactiongiventheconstanthighrateofrequestsoverthebelowwindow.
ScrollDo
wn
Scrolldownthedashboardtoexamineotherinterestingfieldstofurtherinvestigate.
NoticetheGooglebotuseragent string whichisanotherattempttoavoidraisingattention..
46
Therequestsfrom52.211.114.134aredominatedbyrequeststotheloginpage(wp-login.php).It’sclearlynotpossibletoattemptaloginthismanytimesinashortperiodoftime– thisisclearlyascriptedbruteforceattack.
Aftersuccessfullygainingaccesstoourwebsite,theattackerdownloadedthepdf file,weaponized itwiththezeusmalware,thendeliveredittoChrisGilbertasaphishingemail.
Theattackerisalsoaccessingadminpageswhichmaybeanattempttoestablishpersistenceviaabackdoorintothewebsite.
KillChainAnalysisAcrossDataSources
47
http(proxy)sessiontocommand&controlserver
RemotecontrolStealdataPersistincompanyRentasbotnet
Proxy
ConductBusiness
Createadditionalenvironment
GainAccesstosystemTransaction
ThreatIntelligence
Endpoint
NetworkEmail,Proxy,DNS,andWeb
DataSources
.pdf executes&unpacksmalwareoverwritingandrunning“allowed”programs
Svchost.exe(malware)
Calc.exe(dropper)
AttackerhackswebsiteSteals.pdf files
WebPortal.pdf
Attackercreatesmalware,embed in.pdf,
emailstothetarget
Reademail,openattachment
Wecontinuedtheinvestigationbypivotingintotheendpointdatasourceandusedaworkflowactiontodeterminewhichprocessontheendpointwasresponsiblefortheoutboundcommunication.
WeBeganbyreviewingthreatintel relatedeventsforaparticularIPaddressandobservedDNS,Proxy,andEndpointeventsforauserinSales.
Investigationcomplete!LetsgetthisturnedovertoIncidentReponse team.
Wetracedthesvchost.exeZeusmalwarebacktoit’sparentprocessIDwhichwasthecalc.exedownloader/dropper.
Onceourrootcauseanalysiswascomplete,weshiftedoutfocusintotheweblogstodeterminethatthesensitivepdffilewasobtainedviaabruteforceattackagainstthecompanywebsite.
Wewereabletoseewhichfilewasopenedbythevulnerableappanddeterminedthatthemaliciousfilewasdeliveredtotheuserviaemail.
Aquicksearchintothemaillogsrevealedthedetailsbehindthephishingattackandrevealedthatthescopeofthecompromisewaslimitedtojusttheoneuser.
Wetracedcalc.exe backtothevulnerableapplicationPDFReader.
10minBreak!
Appendix- SQLi- DNSExfilatration- Splunk SecurityEssentials
SQLi
SQLInjection● SQLinjection● Codeinjection● OScommanding● LDAPinjection● XMLinjection● XPathinjection● SSIinjection● IMAP/SMTPinjection● Bufferoverflow
ImpervaWebAttacksReport,2015
TheanatomyofaSQLinjectionattack
SELECT * FROM users WHERE email='xxx@xxx.com' OR 1 = 1 -- ' AND password='xxx';
xxx@xxx.xxx' OR 1 = 1 -- '
xxx
admin@admin.sys
1234
Anattackermightsupply:
…andsofarthisyear…39
index=web_vuln passwordselect
Whathavewehere?Ourlearningenvironmentconsistsof:
• Abunchofpublically-accessiblesingleSplunk servers
• Eachwith~5.5Mevents,fromrealenvironmentsbutmassaged:
• WindowsSecurityevents• Apachewebaccesslogs• BroDNS&HTTP• PaloAltotrafficlogs• Someothervariousbits
https://splunkbase.splunk.com/app/1528/
SearchforpossibleSQLinjectioninyourevents:ü looksforpatternsinURIqueryfieldtoseeif
anyonehasinjectedthemwithSQLstatements
ü usestandarddeviationsthatare2.5timesgreaterthantheaveragelengthofyourURIqueryfield
Macrosused• sqlinjection_pattern(sourcetype,uri queryfield)• sqlinjection_stats(sourcetype,uri queryfield)
RegularExpressionFTWsqlinjection_rex isasearchmacro.Itcontains:
(?<injection>(?i)select.*?from|union.*?select|\'$|delete.*?from|update.*?set|alter.*?table|([\%27|\'](%20)*=(%20)*[\%27|\'])|\w*[%27|\']or)
Whichmeans:Inthestringwearegiven,lookforANY ofthefollowingmatchesandputthatintothe“injection”field.
• AnythingcontainingSELECTfollowedbyFROM• AnythingcontainingUNIONfollowedbySELECT• Anythingwitha‘attheend• AnythingcontainingDELETEfollowedbyFROM• AnythingcontainingUPDATEfollowedbySET• AnythingcontainingALTERfollowedbyTABLE• A%27ORa‘andthena%20andanyamountofcharactersthena%20andthena%27ORa‘
• Note:%27isencoded“’”and%20isencoded<space>• Anyamountofwordcharactersfollowedbya%27ORa‘andthen“or”
Bonus:TryouttheSQLInjectionapp!
Summary:Webattacks/SQLinjection● SQLinjectionprovideattackerswitheasyaccesstodata
● DetectingadvancedSQLinjectionishard– useanapp!
● UnderstandwhereSQLi ishappeningonyournetworkandputastoptoit.
● AugmentyourWAFwithenterprise-wideSplunk searches.
DNSExfiltration
domain=corp;user=dave;password=12345
encrypt
DNSQuery:ZG9tYWluPWNvcnA7dXNlcj1kYXZlO3Bhc3N3b3JkPTEyMzQ1DQoNCg==.attack.com
ZG9tYWluPWNvcnA7dXNlcj1kYXZlO3Bhc3N3b3JkPTEyMzQ1DQoNCg==
DNSexfil tendstobeoverlookedwithinanoceanofDNSdata.
Let’sfixthat!
DNSexfiltration
FrameworkPOS:acard-stealingprogramthatexfiltrates datafromthetarget’snetworkbytransmittingitasdomainnamesystem(DNS)traffic
Butthebigdifferenceisthewayhowstolendataisexfiltrated:themalwareusedDNSrequests!https://blog.gdatasoftware.com/2014/10/23942-new-frameworkpos-
variant-exfiltrates-data-via-dns-requests
“”
…feworganizationsactuallykeepdetailedlogsorrecordsof theDNStraffictraversingtheirnetworks— makingitanidealwaytosiphondatafromahackednetwork.
http://krebsonsecurity.com/2015/05/deconstructing-the-2014-sally-beauty-breach/#more-30872
“”
DNSexfiltration
https://splunkbase.splunk.com/app/2734/
DNSexfil detection– tricksofthetradeü parseURLs&complicatedTLDs(TopLevelDomain)ü calculateShannonEntropy
Listofprovidedlookups• ut_parse_simple(url)• ut_parse(url,list)orut_parse_extended(url,list)• ut_shannon(word)• ut_countset(word,set)• ut_suites(word,sets)• ut_meaning(word)• ut_bayesian(word)• ut_levenshtein(word1,word2)
Examples• Thedomainaaaaa.com hasaShannonEntropyscoreof1.8 (verylow)• Thedomaingoogle.com hasaShannonEntropyscoreof2.6 (ratherlow)• A00wlkj—(-a.aslkn-C.a.2.sk.esasdfasf1111)-890209uC.4.com hasaShannon
Entropyscoreof3 (ratherhigh)
Layman’sdefinition:ascorereflectingtherandomness ormeasureofuncertainty ofastring
ShannonEntropy
DetectingDataExfiltration
index=brosourcetype=bro_dns|`ut_parse(query)`|`ut_shannon(ut_subdomain)`|eval sublen =length(ut_subdomain)|tableut_domain ut_subdomainut_shannon sublen
TIPSq LeverageourBroDNSdataq CalculateShannonEntropyscoresq Calculatesubdomainlengthq DisplayDetails
DetectingDataExfiltration
…|statscountavg(ut_shannon)asavg_shaavg(sublen)asavg_sublenstdev(sublen)asstdev_sublenbyut_domain|searchavg_sha>3avg_sublen>20stdev_sublen<2
TIPSq LeverageourBroDNSdataq CalculateShannonEntropyscoresq Calculatesubdomainlengthq Displaycount,scores,lengths,
deviations
DetectingDataExfiltrationRESULTS• Exfiltrating datarequiresmanyDNSrequests– lookforhighcounts• DNSexfiltrationtomooo.com and chickenkiller.com
Summary:DNSexfiltration● ExfiltrationbyDNSandICMPisaverycommontechnique● ManyorganizationsdonotanalyzeDNSactivity– donotbelikethem!● NoDNSlogs?NoSplunkStream?LookatFWbytecounts
Splunk SecurityEssentials
https://splunkbase.splunk.com/app/3435/
Identifybadguysinyourenvironment:ü 45+usecasescommoninUEBAproducts,all
freeonSplunkEnterpriseü Targetexternalattackersandinsiderthreatü Scalesfromsmalltomassivecompaniesü Savefromtheapp,sendresultstoES/UBA
ThemostwidelydeployedUEBAvendorinthemarketisSplunkEnterprise,butnooneknowsit.
Solveusecasesyoucantodayforfree,thenuseSplunkUBAforadvancedMLdetection.
SplunkSecurityEssentials
TimeSeriesAnalysiswithStandardDeviation
FirstTimeSeenpoweredbystats
GeneralSecurityAnalyticsSearches
TypesofUseCases
SplunkSecurityEssentialsDataSources
ElectronicMedicalRecords
SourceCodeRepository
● Howdoestheappwork?– Leveragesprimarily| stats forUEBA– AlsoimplementsseveraladvancedSplunksearches(URLToolbox,etc.)
● WhycallitUEBA?– TheseusecasesareofteninUEBAtools– 2/3ofusecasebuildonabaseline,whichisahallmarkofUEBA– 1/3areadvancedanalyticsthatothervendorsshowcaseintheirUEBA
● Howdoesitscale?– Appautomatestheutilizationofhighscaletechniques– SummaryindexingforTimeSeries,cachinginlookupforFirstTime
Splunk EnterpriseSecurity
78
SplunkEnterprise
- BigDataAnalyticsPlatform-
SplunkEnterpriseSecurity
- SecurityAnalyticsPlatform-
ThreatHuntingwithSplunk
Hypotheses
AutomatedAnalytics
DataScience&MachineLearning
Data&IntelligenceEnrichment
DataSearch
Visualisation
Maturity
ThreatHuntingDataEnrichment
ThreatHuntingAutomation
Ingest&OnboardAnyThreatHunting
MachineDataSource
Search&VisualiseRelationshipsforFasterHunting
OtherItemsToNote
ItemstoNote
Navigation- HowtoGetHere
Descriptionofwhattoclickon
Click
KeySecurityIndicators(buildyourown!)
Sparklines
Editable
Variouswaystofilterdata
Malware-SpecificKSIsandReports
SecurityDomains->Endpoint->MalwareCenter
Filterable
KSIsspecifictoRisk
Riskassignedtosystem,userorother
UnderAdvancedThreat,selectRiskAnalysis
(ScrollDown)
RecentRiskActivity
UnderAdvancedThreat,selectRiskAnalysis
Filterable,downtoIoC
KSIsspecifictoThreat
Mostactivethreatsource
Scrolldown… Scroll
UnderAdvancedThreat,selectThreatActivity
Specificsaboutrecentthreatmatches
UnderAdvancedThreat,selectThreatActivity
Toaddthreatintelgoto:Configure->DataEnrichment->ThreatIntelligenceDownloads
Click
Click“ThreatArtifacts”Under“AdvancedThreat”
Click
ArtifactCategories–clickdifferenttabs…
STIXfeed
Customfeed
UnderAdvancedThreat,selectThreatArtifacts
ReviewtheAdvancedThreatcontent
Click
DatafromassetframeworkConfigurableSwimlanes
Darker=moreevents
AllhappenedaroundsametimeChangeto“Today”ifneeded
AssetInvestigator,enter“192.168.56.102”
DataScience&MachineLearningInSecurity
91
Disclaimer:Iamnotadatascientist
TypesofMachineLearningSupervised Learning:generalizingfromlabeled data
SupervisedMachineLearning
94
DomainName TotalCnt RiskFactor AGD SessionTime RefEntropy NullUa Outcome
yyfaimjmocdu.com 144 6.05 1 1 0 0 Maliciousjjeyd2u37an30.com 6192 5.05 0 1 0 0 Maliciouscdn4s.steelhousemedia.com 107 3 0 0 0 0 Benignlog.tagcade.com 111 2 0 1 0 0 Benigngo.vidprocess.com 170 2 0 0 0 0 Benignstatse.webtrendslive.com 310 2 0 1 0 0 Benigncdn4s.steelhousemedia.com 107 1 0 0 0 0 Benignlog.tagcade.com 111 1 0 1 0 0 Benign
Unsupervised Learning:generalizingfromunlabeled data
UnsupervisedMachineLearning
• Notuning
• Programmaticallyfindstrends
• UBAisprimarilyunsupervised
• Rigorouslytestedforfit
96
AlgorithmRawSecurityData AutomatedClustering
97
MLToolkit&Showcase• SplunkSupportedframeworkforbuildingMLApps
– Getitforfree:http://tiny.cc/splunkmlapp
• LeveragesPythonforScientificComputing (PSC)add-on:– Open-sourcePythondatascienceecosystem– NumPy,SciPy,scitkit-learn,pandas,statsmodels
• Showcaseusecases:PredictHardDriveFailure,ServerPowerConsumption,ApplicationUsage,CustomerChurn&more
• Standardalgorithms outofthebox:– Supervised:LogisticRegression,SVM,LinearRegression,RandomForest,etc.– Unsupervised: KMeans,DBSCAN,SpectralClustering,PCA,KernelPCA,etc.
• Implementoneof300+algorithmsbyeditingPythonscripts
MachineLearningToolkitDemo
99
Splunk UBA
102
SplunkEnterprise
- BigDataAnalyticsPlatform-
SplunkEnterpriseSecurity
- SecurityAnalyticsPlatform-
ThreatHuntingwithSplunk
ThreatHuntingDataEnrichment
ThreatHuntingAutomation
Ingest&OnboardAnyThreatHunting
MachineDataSource
Search&VisualiseRelationshipsforFasterHunting
Hypotheses
AutomatedAnalytics
DataScience&MachineLearning
Data&IntelligenceEnrichment
DataSearch
Visualisation
Maturity
UserBehaviorAnalytics
- SecurityDataSciencePlatform-
103
MachineLearningSecurityUseCasesMachine
LearningUseCases
PolymorphicAttackAnalysis
BehavioralPeerGroupAnalysis
User&EntityBehaviorBaseline
Entropy/RareEventDetection
CyberAttack/ExternalThreatDetection
Reconnaissance,BotnetandC&CAnalysis
LateralMovementAnalysis
StatisticalAnalysis
DataExfiltrationModels
IPReputationAnalysis
InsiderThreatDetection
User/DeviceDynamicFingerprinting
Splunk UBAUseCases
ACCOUNTTAKEOVER• Privilegedaccountcompromise• Dataexfiltration
LATERALMOVEMENT• Pass-the-hashkillchain• Privilegeescalation
SUSPICIOUSACTIVITY• Misuseofcredentials• Geo-locationanomalies
MALWAREATTACKS• Hiddenmalwareactivity
BOTNET,COMMAND&CONTROL• Malwarebeaconing• Dataleakage
USER&ENTITYBEHAVIORANALYTICS• Suspiciousbehaviorbyaccountsor
devices
EXTERNALTHREATSINSIDERTHREATS
SplunkUserBehaviorAnalytics(UBA)• ~100%ofbreachesinvolvevalidcredentials(Mandiant Report)• Needtounderstandnormal&anomalousbehaviorsforALLusers• UBAdetectsAdvancedCyberattacks andMaliciousInsiderThreats• LotsofMLunderthehood:
– BehaviorBaselining&Modeling– AnomalyDetection(30+models)– AdvancedThreatDetection
• E.g.,DataExfil Threat:– “Sawthisstrangelogin&datatransferfor userkwestin
at3aminChina…”– SurfacethreattoSOCAnalysts
RAW SECURITY EVENTS
ANOMALIES ANOMALY CHAINS
(THREATS)
MACHINELEARNING
GRAPHMINING
THREAT MODELS
Lateral MovementBeaconingLand-Speed Violation
HCI
Anomalies graphEntity relationship graph
Kill chain sequenceForensic artifactsThreat/Risk scoring
FEEDBACK
OverallArchitecture
107
Real-TimeInfra(Storm-based)
Filte
rEvents
Drop
Events
Mod
elExecutio
n&
OnlineTraining
Runtim
eTo
pologies
ThreatandAnomalyReview
Hadoop/HDFS
DataReceivers
(flume,REST,etc.)
Real-Tim
eUpd
ates/N
otifications
App/SaaSConnectors
Core+ES
NetworkData
Push/PullModel
PersistenceLayer
DataDistributedKafka
ETL
IRModelParsers Filters
Attribution
ControlP
ath–Re
source/H
ealth
Mon
itorin
g
HBase/HDFSDirectAccessFaçade
GraphDB
SQL AccessLayer
Node.js
Socket.ioserver
SQLStore(Threats/Anomalies)
Time-SeriesDBModelRegistry
ModelStore HBase
ModelNData
Model1
ModelN
Model1
ModelN
Neo4J(Graph
visualizations)
RulesEngine
Anomalies+Threats
AnalyticsStore
SyslogandOtherData
DataFlowandSystemRequirements
APICONNECTOR
SYSLOG
FORWARDER
Explore Visualize ShareAnalyze Dashboards
RESULTS
THREAT&ANOMALYDATA
QUERYUBA
REQUESTFORADDITIONALDETAILS
THREATS
RESULTS
QUERY
NOTABLEEVENTS
RISKSCORINGFRAMEWORK
WORKFLOWMANAGEMENT
VM
Searchhead
StandardRTQuery
VMspecs:- Ubuntu/RHEL- 16cores- 64GBRAM- Localandnetworkdisks- GigEconnectivity
Performance/scale:- UBAv2.3- E.g.,5-nodes
- 25KEPS- Addnodesfornear-linearscale
SplunkEnterprise:- RTsearchcapability- 8-10concurrent
searches- RESTAPIport(8089)- SA-LDAPSEARCH
Sharednetworkstorage
Splunk UBADemo
109
SecurityWorkshops
● SecurityReadinessWorkshop● DataScienceWorkshop● EnterpriseSecurityBenchmarkAssessment
SecurityWorkshopSurvey
https://www.surveymonkey.com/r/3T6T9TH
kwestin@splunk.comTwitter:@kwestinlinkedin.com/in/kwestin
top related