honeycon2014: mining iocs from honeypot data feeds

71
Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF Mining compromise indicators from Honeypot Systems Vladimir Kropotov, Vitaly Chetvertakov, Fyodor Yarochkin HoneyCON 2014 Affilations: Academia Sinica, o0o.nu, chroot.org Jul 07, 2014, Taipei Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Upload: fyodor-

Post on 11-Aug-2014

268 views

Category:

Data & Analytics


5 download

DESCRIPTION

This Honeynet/Taiwan chapter talk

TRANSCRIPT

Page 1: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Mining compromise indicators from Honeypot Systems

Vladimir Kropotov, Vitaly Chetvertakov, Fyodor YarochkinHoneyCON 2014

Affilations: Academia Sinica, o0o.nu, chroot.org

Jul 07, 2014, Taipei

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 2: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Outline

Introduction

IOC Standards

V:IOCs

mining IOCs

Applying IOCs

EOF

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 3: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

WHOAMI

Affilations: Academia Sinica, chroot, and a few others Mainly independentresearch (not vendor affilated ;-))

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 4: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

WHOAMI:2

Our data sources:I Academia SinicaI Not to be named networks in Russian Federation

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 5: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Good things to know

Main Assumption: All networks are compromisedThe difference between a good security team and a bad security team is thatwith a bad security team you will never know that you’ve been compromised.Running Honeypots in the parts network gives a team visibility on emergingthreats that your network might face.

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 6: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

HP landscape

I HP platforms typically would have very low false/positive ratio. If

your HP is hit, it is most likely a suspicious event.I HP typically should replicate your typical enviroment. We focus

on simulation of both end-user machines and servers/services.

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 7: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Statistic on end-user compromises

I about 40,000,000 internet users in RussiaI for every 10,000 server hosts 500 hosts trigger redirects to malicious

content per weekI about 20-50 user machines (full AV installed, NAT, FW) get ..affectedI many infect .ru IP addresses only (source matters)

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 8: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Campaigns

r*.ru News ~ 790 000ne*.com news ~ 590 000ga*.ru news ~ 490 000a*f.ru news ~ 330 000m*.ru news ~ 315 000v*.ru news ~ 170 000li*.ru news ~ 170 000top*s.ru news ~ 140 000

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 9: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Introduction:terminology

Indicators of CompromiseIndicator of compromise (IOC) in computer forensics is an artifact observed onnetwork or in operating system that with high confidence indicates a computerintrusion.http://en.wikipedia.org/wiki/Indicator_of_compromise

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 10: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Why Indicators of compromise

Indicators of Compromise help us to answer questions like:

I is this document/file/hash malicious?I is there any past history for this IP/domain?I what are the other similar/related domains/hashes/..?I who is the actor?I am I an APT target?!!;-)

They shorten initial-detection -*to*- detection-automation cycle.

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 11: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

IoCs: old dog - new tricks

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 12: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

A Network compromise case study:

I Attackers broke via a web vuln.I Attackers gained local admin accessI Attackers created a local userI Attackers started probing other machines for default user idsI Attackers launched tunneling tools – connecting back to C2I Attackers installed RATs to maintain access

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 13: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

IoC Indicators

So what are the compromise indicators here?

I Where did attackers come from? (IP)I What vulnerability was exploited? (pattern)I What web backdoor was used? (pattern, hash)I What tools were uploaded? (hashes)I What users were created locally? (username)I What usernames were probed on other machinesI Detailed IoCs (unsual port to serve exploit kit, URI pattern,

mime-content, user agent)

Warning: Blind use of IoCs may lead to disaster. (some IoCs are more suitablefor statistical studies)

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 14: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Where to look for IOCs internally

I Outbound Network TrafficI User Activities/Failed LoginsI User profile foldersI Administrative AccessI Access from unsual IP addressesI Database IO: excessive READsI Size of responses of web pagesI Unusual access to particular files within Web Application (backdoor)I Unusual port/protocol connectionsI DNS and HTTP traffic requestsI Suspicious Scripts, Executables and Data Files

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 15: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

IoCs (good and bad)

Why we need IOCs? because it makes it easier to systematically describeknowledge about breaches.

I Identifying intrusions is hardI Unfair game:

I defender should protect all the assetsI attacker only needs to ’poop’ one system.

I Identifying targeted, organized intrusions is even harderI Minor anomalous events are important when put togetherI Seeing global picture is a mastI Details matterI Attribution is hard

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 16: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

What’s wrong with IoCs

I IoCs expire (IP addresses get discovered, cleaned)I Domain names expireI Hash collisionsI Benign binaries might be malicious (depending on context)

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 17: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Good or Bad?F i l e Name : RasTls . exeF i l e S i z e : 105 kBF i l e Mod i f i c a t i o n Date/Time : 2009 :02 :09 19 :42 :05+08:00F i l e Type : Win32 EXEMIME Type : a p p l i c a t i o n / octe t−st reamMachine Type : I n t e l 386 or l a t e r , and compa t i b l e sTime Stamp : 2009 :02 :02 13 :38 :37+08:00PE Type : PE32L i n k e r Ve r s i on : 8 . 0Code S i z e : 49152I n i t i a l i z e d Data S i z e : 57344U n i n i t i a l i z e d Data S i z e : 0Entry Po in t : 0 x3d76OS Ve r s i on : 4 . 0Image Ve r s i on : 0 . 0Subsystem Ve r s i on : 4 . 0Subsystem : Windows GUIF i l e Ve r s i on Number : 1 1 . 0 . 4 0 10 . 7Product Ve r s i on Number : 1 1 . 0 . 4 0 10 . 7F i l e OS : Windows NT 32− b i tObject F i l e Type : Execu tab l e a p p l i c a t i o nLanguage Code : Eng l i s h (U. S . )Cha r a c t e r Set : Windows , La t i n1Company Name : Symantec Co rpo r a t i o nF i l e D e s c r i p t i o n : Symantec 802 .1 x Supp l i c a n tF i l e Ve r s i on : 1 1 . 0 . 4 0 10 . 7I n t e r n a l Name : do t 1 x t r a y

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 18: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

It really depends on contextRasTls . DLLRasTls . DLL . mscRasTls . exe

http://msdn.microsoft.com/en-us/library/ms682586(v=VS.85).aspxDynamic-Link Library Search Order

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 19: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

IOC representations

Multiple standards have been created to facilitate IOC exchanges.I Madiant: OpenIOCI Mitre: STIX (Structured Threat Information Expression), CyBOX

(CyberObservable Expression)I Mitre: CAPEC, TAXIII IODEF (Incident Object Description Format)

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 20: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Standards: OpenIOCOpenIOC - Mandiant-backed effort for unform representation of IOC (nowFireEye) http://www.openioc.org/

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 21: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

OpenIOCsD i g i t a l Append ices /Appendix G ( D i g i t a l ) − IOCs$ l s0 c7c902c−67f8−479c−9f44−4d985106365a . i o c 6bd24113−2922−4d25−b490−f 727 f47ba948 . i o cad521068−6f18−4ab1−899c−11007 a18ec73 . i o c12 a40bf7−4834−49b0−a419−6abb5fe2b291 . i o c 70 b5be0c−8a94−44b4−97a4−1e95b09498a8 . i o ca f 5 f 6 5 f c−e1ca−45db−88b1−6ccb7191ee6a . i o c2106 f0d2−a260−4277−90ab−edd3455e31fa . i o c 7c739d52−c669−4d51−ac15−8ae66305e232 . i o cAppendix G IOCs README. pdf26213db6−9d3b−4a39−abeb−73656 acb913e . i o c 7 d2eaadf−a5 f f −4199−996e−af6258874dad . i o cc32b8af3−28d0−47d3−801 f−a2c2b0129650 . i o c2 b f f 2 23 f −9e46−47a7−ac35−d35f8138a4c7 . i o c 7 f9a6986−f00a−4071−99d3−484c9158beba . i o cc71b3305−85e5−4d51−b07c−f f 2 27181 fb5a . i o c2 fc55747 −6822−41d2−bcc1−387 fc1b2e67b . i o c 806 be f f 3 −7395−492e−be63−99a6b4a550b8 . i o cc7fa2ea5 −36d5−4a52−a6cf−ddc2257cb6f9 . i o c32b168e6−dbd6−4d56−ba2f −734553239 e f e . i o c 84 f04d f2 −25cd−4f59−a920−448d8843b6fc . i o cd14d5f09−9050−4769−b00d−30 fce9e6eb85 . i o c3433dad8−879e−40d9−98b3−92ddc75f0dcd . i o c 8695bb5e−29cd−41b9−b8b1−a0d20a6b960d . i o cd1c65316−cddd−4d9c−8e fe−c539aa5965c0 . i o c3e01b786−fe3a −4228−95 fa−c3986e2353d6 . i o c 86 e9b8ec−7413−453b−a932−b5fb95a8dba6 . i o cd4f103f8−c372−49d1−b9f4−e127d61d0639 . i o c4 a2c5f60−f4c0−4844−ba1f−a14dac9 fa36c . i o c 86 f988b7−fa02−46df−8e19−e50c e37 f 0 f ed . i o cd5e49501−c30d−41ae−b381−c3c473040c39 . i o c4 d1ced5f−fe47−4ba4−be0e−81d547f3aa8a . i o c 8900 aa6b−883d−48d3−a07d−d49b0429dd2b . i o cd8240090−a f f d −466e−a39c−64add5b98813 . i o c5477b392−e565−45c5−9cb4−f561d6daeddc . i o c 8dd23e0a−a659−45b4−a168−67e4b00944fb . i o ce928aac0−9f71−4adf −9978−4177345 ec610 . i o c547 e4128−9d f f −45d9−b90f−081ce3966dee . i o c 9 c9368cd−3a1f−4200−b093−adb97d5f1 f5d . i o ceb91abad−afe0−4bd6−80f2−850d14a99308 . i o c56468547−6 cf5−4c66−af56 −2543d4271482 . i o c a1f02cbe−7d37−4f f 8−bad7−c5 f 9 f 7 ea63a3 . i o cece1846e−98d3−4ddc−a520−0dcda4866989 . i o c6091 c4ce−6d73−4202−a7a8−b52406fa4d77 . i o c a461f381−8612−4ce1−a0dc−68bcaca028d0 . i o cfabdf553−b3ed−4bc9−9ac6−13d6bd174dad . i o c61695156−298c−4d77−ad7f−48 f eb562 fb75 . i o c a486d837−9f05−4360−908e−b4244c24723d . i o cfd fb2c22−d0c4−4bf0−8ea4−27d8d51f98ea . i o c

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 22: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Standards: Mitre

Mitre CybOX: http://cybox.mitre.org/https://github.com/CybOXProject/Toolshttps://github.com/CybOXProject/openioc-to-cybox Mitre CAPEC:http://capec.mitre.org/ Mitre STIX: http://stix.mitre.org/ MitreTAXII http://taxii.mitre.org/

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 23: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Mature: stix

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 24: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Indicators of Compromise

I Complex IOCs covering all steps of attackI Dynamic creation of IOCs on the flyI Auto-reload of IOCs, TTLsI Dealing with different standards/import export

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 25: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Exploit pack trace

url ip mime type refhttp://cuba.eanuncios.net/1/zf3z9lr6ac8di6r4kw2r0hu3ee8ad.html 93.189.46.222 text/html http://www.smeysyatut.ru/ 118162 413 200

http://cuba.eanuncios.net/2909620968/1/1399422480.htm 93.189.46.222 text/html http://cuba.eanuncios.net/1/zf3z9lr6ac8di6r4kw2r0hu3ee8ad.html 37432 441 200

http://cuba.eanuncios.net/2909620968/1/1399422480.jar 93.189.46.222 application/java-archive - 18451 323 200http://cuba.eanuncios.net/2909620968/1/1399422480.jar 93.189.46.222 application/java-archive - 18451 280 200http://cuba.eanuncios.net/f/1/1399422480/2909620968/2 93.189.46.222 - - 115020 244 200http://cuba.eanuncios.net/f/1/1399422480/2909620968/2/2 93.189.46.222 - - 327 246 200

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 26: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Nuclearsploit pack{ ’ Nu c l e a r s p l o i t p a c k ’ : {’ s tep1 ’ : {’ f i l e s ’ : [ ’ wz3u6s i8e5 lh7k2tk5ox4ne6d8g . html ’ , ’ t 3 f 5 y9a2bb3d l 7 z8gc4o6 f . html ’ , ’ z f 3 z 9 l r 6 a c8d i 6 r 4 kw2 r 0hu3ee8ad . html ’ , ’ r x 3 v b 9 q g 6 l q 8 l l 6 i j 4 u 2 s a 0 x x 3 l n 8 l e . html ’ , ’ k2qx3dv0ey7 l o3 rp8q6ce4 lw0 fp0z . html ’ , ’ k z6 tp7k4cx3h4 j 8k r3za5a . html ’ , ’ wq6 ln7o4z j3d4 fu8zc3a5sw . html ’ , ’ z2c8mg6h0df2n2ss8kd2e6k7y . html ’ ] ,’ domains ’ : [ ’ f a t h e r . f e r r em o v i l . com ’ , ’ t h a i . a l o h a t r a n s l l c . com ’ , ’ cuba . e anunc i o s . net ’ , ’ duncan . d i s e n o c o r p o r a t i v o . com . ar ’ , ’ homany . c o l l e c t i v e i t . com . au ’ , ’ p r i v a c y . t e r a p i a . o rg . ar ’ ] ,

’ arguments ’ : [ ] ,’ d i r e c t o r i e s ’ : [ ’ 1 ’ ] ,

’ ip ’ : [ ’ 9 3 . 1 8 9 . 4 6 . 2 0 1 ’ , ’ 9 3 . 1 8 9 . 4 6 . 2 0 3 ’ , ’ 9 3 . 1 8 9 . 4 6 . 2 2 2 ’ , ’ 9 3 . 1 8 9 . 4 6 . 2 2 4 ’ , ’ 9 3 . 1 8 9 . 4 6 . 2 3 3 ’ ] } ,’ s tep2 ’ : {’ f i l e s ’ : [ ’ 1399422480 . htm ’ , ’1399704720 . htm ’ , ’1399513440 . htm ’ , ’1399514040 . htm ’ ,’1399773300 . htm ’ ] ,’ domains ’ : [ ’ cuba . e anunc i o s . net ’ , ’ duncan . d i s e n o c o r p o r a t i v o . com . ar ’ , ’ homany . c o l l e c t i v e i t . com . au ’ , ’ p r i v a c y . t e r a p i a . o rg . ar ’ ] ,’ arguments ’ : [ ] ,’ d i r e c t o r i e s ’ : [ ’ 2909620968 ’ , ’ 1 ’ , ’507640988 ’ , ’940276731 ’ , ’3957283574 ’ , ’ 952211704 ’ ] ,’ ip ’ : [ ’ 9 3 . 1 8 9 . 4 6 . 2 2 2 ’ , ’ 9 3 . 1 8 9 . 4 6 . 2 2 4 ’ , ’ 9 3 . 1 8 9 . 4 6 . 2 3 3 ’ ] } ,’ s tep3 ’ : {’ f i l e s ’ : [ ’ 1399422480 . j a r ’ , ’1399513440 . j a r ’ ] ,’ domains ’ : [ ’ cuba . e anunc i o s . net ’ , ’ homany . c o l l e c t i v e i t . com . au ’ ] ,’ arguments ’ : [ ] ,’ d i r e c t o r i e s ’ : [ ’ 2909620968 ’ , ’ 1 ’ , ’ 940276731 ’ ] ,’ ip ’ : [ ’ 9 3 . 1 8 9 . 4 6 . 2 2 2 ’ , ’ 9 3 . 1 8 9 . 4 6 . 2 2 4 ’ ] } ,’ s tep4 ’ : {’ f i l e s ’ : [ ’ 2 ’ ] ,’ domains ’ : [ ’ cuba . e anunc i o s . net ’ ] ,’ arguments ’ : [ ] ,’ d i r e c t o r i e s ’ : [ ’ f ’ , ’ 1 ’ , ’1399422480 ’ , ’2909620968 ’ , ’ 2 ’ ] ,’ ip ’ : [ ’ 9 3 . 1 8 9 . 4 6 . 2 2 2 ’ ] }}}

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 27: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Redirect (example)

http://mysimuran.ru/forum/kZsjOiDMFb/ 89.111.178.33 http://agency.accordinga.pw/remain/unknown.html?mods=8&id=26,text/htmlhttp://mysimuran.ru/forum/kZsjOiDMFb/js.js?4231 89.111.178.33 http://mysimuran.ru/forum/kZsjOiDMFb/,text/plainhttp://c.hit.ua/hit?i=59278&g=0&x=2 89.184.81.35 http://mysimuran.ru/forum/kZsjOiDMFb/,image/gifhttp://f-wake.browser-checks.info:28001/d1x/3/87475b26a521024ce78d7ea73164140a/http%3A%2F%2Fagency.accordinga.pw%2Fremain%2Funknown.html%3Fmods%3D8%26id%3D26 46.254.16.209 http://mysimuran.ru/forum/kZsjOiDMFb/,text/html

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 28: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Redirect Example{ ’28001 ’ : {’ s tep1 ’ : {

’ d i r e c t o r i e s ’ : [ ’ forum ’ , ’ kZsjOiDMFb ’ , ’ epygFrFsoU ’ ] ,’ arguments ’ : [ ] ,’ f i l e s ’ : [ ’ ’ ] ,’ ip ’ : [ ’ 8 9 . 1 1 1 . 1 7 8 . 3 3 ’ ] ,’ domains ’ : [ ’ mysimuran . ru ’ ] } ,’ s tep2 ’ : {

’ d i r e c t o r i e s ’ : [ ’ forum ’ , ’ kZsjOiDMFb ’ , ’ epygFrFsoU ’ , ’kJXshWOMNC’ ] ,’ arguments ’ : [ ’ 4231 ’ , ’7697 ’ , ’9741 ’ ] ,’ f i l e s ’ : [ ’ j s . j s ’ , ’ cnt . html ’ ] ,’ ip ’ : [ ’ 8 9 . 1 1 1 . 1 7 8 . 3 3 ’ ] ,’ domains ’ : [ ’ mysimuran . ru ’ ] } ,’ s tep3 ’ : {’ d i r e c t o r i e s ’ : [ ] ,’ arguments ’ : [ ’ i ’ , ’ g ’ , ’ x ’ ] ,’ f i l e s ’ : [ ’ h i t ’ ] ,’ ip ’ : [ ’ 8 9 . 1 8 4 . 8 1 . 3 5 ’ ] ,’ domains ’ : [ ’ c . h i t . ua ’ ] } ,’ s tep4 ’ : {’ d i r e c t o r i e s ’ : [ ’ d1x ’ , ’ 3 ’ , ’87475 b26a521024ce78d7ea73164140a ’ , ’ d36eb1fc80ebe9df515d043be1557f57 ’ ] ,’ arguments ’ : [ ] ,’ f i l e s ’ : [ ’ h t tp%3A%2F%2Fagency . a c c o r d i n g a . pw%2Fremain%2Funknown . html%3Fmods%3D8%26i d%3D26 ’ , ’ h t tp%3A%2F%2Fs t ruck . l ookeda . pw%2Fcongre s s%2Fp r e s i d e n t . html%3F lo s e%3D21%26amid%3D463 ’ ] ,’ ip ’ : [ ’ 4 6 . 2 5 4 . 1 6 . 2 0 9 ’ ] ,’ domains ’ : [ ’ f−wake . browser−checks . i n f o ’ , ’ a−oprzay . browser−checks . pw ’ ] }

}}

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 29: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

IOCs

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 30: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

IOCs3

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 31: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

IOCs viz

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 32: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

IOCs viz(02)

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 33: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

IOCs viz(3)

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 34: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

IOCs viz(4)

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 35: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

IOCs viz(5)

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 36: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Sourcing External IOCsI CIF - https:

//code.google.com/p/collective-intelligence-framework/I feeds (with scrappers):

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 37: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Sourcing External IOCsI feed your scrappers:

https://zeustracker.abuse.ch/blocklist.php?download=badipshttp://malc0de.com/database/https://reputation.alienvault.com/reputation.data . . .

I VT intelligence

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 38: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Sourcing IOCs Internally

I honeypot feedsI log analysisI traffic analysis

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 39: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Extracting IoCs from HTTP traffic caps

01/14/13 06:57 PM 178.238.141.19 (url1) application/x-java-archive01/14/13 06:57 PM 178.238.141.19 (url2) application/x-java-archive01/14/13 06:57 PM 178.238.141.19 (url3) application/octet-stream

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 40: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Use honeypots

I Running honeypots gives enormous advantage in detecting emerging

threatsI Stategically placing honeypots is extemely important

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 41: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

HPfeeds, Hpfriends and more

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 42: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

HPFeeds Architecture

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 43: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

HPFeeds API in nutshell:

import pygeo ipimport hp f e ed simport j s o n

HOST=’ b rok e r ’PORT = 20000CHANNELS= [ ’ g eo l o c . e v en t s ’ ]IDENT=’ i d e n t ’SECRET=’ s e c r e t ’g i = pygeo ip . GeoIP ( ’ GeoL i t eC i t y . dat ’ )hpc = hp f e ed s . new (HOST, PORT, IDENT , SECRET)msg = { ’ l a t i t u d e ’ : g i . record_by_addr ( i p ) [ ’ l a t i t u d e ’ ] ,

’ l o n g i t u d e ’ : g i . record_by_addr ( i p ) [ ’ l o n g i t u d e ’ ] ,’ t ype ’ : ’ honeypot ␣ h i t ’ }

hpc . p u b l i s h (CHANNELS, j s o n . dumps (msg ) )

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 44: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

hpfeeds integrationI HPFEEDS works with glastopf out of the boxI Kippo (module provided http://github.com/disaster/kippo/)

I ntp/smb - custom modules

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 45: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

NTP probe collector

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 46: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

HPFeeds and honeymap

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 47: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

HPFeeds indexingHPFeed custom broker: writes indicators into ElasticSearch.Could be automatically reused by other security tools

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 48: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

HPfeeds and post processingAside from analyzing HP events post-processing can mine interesting things:

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 49: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Content analysis:Hosting domains: over 300~ unique domain names:

a c v i l l a . ucoz . com . bengosa c v i l l a . ucoz . com . g r ad i n aa c v i l l a . ucoz . com . sa d e l i n u . ucoz . ro . boa d i s o r . webs . com . bnc2adryanb . i . was . i nandyakamus ic . a l t e r v i s t a . org . wpa n g e l f i r e . com . komales88 . gosha n g e l f i r e . com . komales88 . psybnca n g e l f i r e . com . l u k y l u s . rha n g e l l o v e . ucoz . net .apropo . ucoz . net . 2apropo . ucoz . net . compapropo . ucoz . net . psyapropo . ucoz . net . psycompapropo . ucoz . net . s s lapropo . ucoz . net . s s l la u s t r y an .110mb. com . bbarac . c o f f e e c up . com . s b i nbeaumult . go . ro . butblowme . a l t e r v i s t a . org . 1blowme . a l t e r v i s t a . org . a r h i v eblowme . a l t e r v i s t a . org . s n i f fblowme . a l t e r v i s t a . org . s n i f f e rboaka . go . ro . b u t z ibogghy . a l t e r v i s t a . org . popscanborac . 3 owl . com . aws

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 50: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Content analysis:

Tools, Scanners

20120211233926_http___www_freewebs_com_westcoste_php_zip20120211234012_http___www_freewebs_com_westcoste_php_tar_gz20120213081741_http___www_freewebs_com_westcoste_php_zip20121217032335_http___r_o_o_t_hi2_ro_scanner_php_jpg20130306173911_http___botiphp_go_ro_rdp_tgz20111006193700_http___system_arhive_do_am_scanner_web_jpg20120204145752_http___www_click4me_home_ro_scanbun_zip20120407032809_http___XxLx2010_hi2_ro_XxLxScan_zip20120424100124_http___pragu_webs_com_Scanner_History_tgz20120424104136_http___qiss_ucoz_de_scanptvasy_jpg20120701095229_http___haXers_Webs_Com_Scanner_gosh_tgz20121006034334_http___system_comule_com_scanner_gosh_jpg20121212214201_http___procesed_do_am_NGS_scan_CScan_tgz

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 51: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Content analysis:Exploits:

20121122231601_http___system_comule_com_exploit_e_jpg20121122231632_http___system_comule_com_exploit_e_tgz20140510104703_http___treeball_tripod_com_ex_tgz20140527103805_http___treeball_tripod_com_ex_tgz. . .−rwxr−xr−x danam1/danam1 2275 2012−04−03 05 :38 x/do . c−rwxr−xr−x danam1/danam1 6910 2012−04−03 05 :42 x/me . c−rwxr−xr−x danam1/danam1 6554 2012−04−03 00 :29 x/ab . c−rwxr−xr−x danam1/danam1 4709 2012−04−03 00 :08 x/new . c−rwxr−xr−x danam1/danam1 10300 2012−04−03 00 :53 x/newdrwxr−xr−x danam1/danam1 0 2012−03−29 22 :58 x/x86/−rwxr−xr−x danam1/danam1 5538 2012−03−29 22 :16 x/x86/newx86 . c−rwxr−xr−x danam1/danam1 11302 2012−03−29 22 :16 x/x86/newx86drwxr−xr−x danam1/danam1 0 2012−03−29 22 :45 x /2011/

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 52: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Tools for Dynamic Detection of IOC

I Snort (everyone knows, SourceFire is just outside ;-))I Yara + yara-enabled toolsI MolochI Splunk/Log search (they are also here :p)I roll-your-own:p

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 53: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Applying IOCs to your detection process

moloch moloch moloch :)

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 54: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Moloch

Moloch is awesome:

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 55: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Open-source tools

OpenIOC manipulationhttps://github.com/STIXProject/openioc-to-stixhttps://github.com/tklane/openiocscriptsMantis Threat Intelligence Frameworkhttps://github.com/siemens/django-mantis.git Mantis supportsSTIX/CybOX/IODEF/OpenIOC etc via importers:https://github.com/siemens/django-mantis-openioc-importerSearch splunk data for IOC indicators:https://github.com/technoskald/splunk-searchOur framework: http://github.com/fygrave/iocmap/

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 56: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

iocmap

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 57: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

MISP

I http://www.secure.edu.pl/pdf/2013/D2_1530_A_Socha.pdfI https://github.com/MISP

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 58: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Tools for Dynamic Detection

I MolochI Moloch supports Yara (IOCs can be directly applied)I Moloch has awesome tagger plugin:

# tagge r . so# p r o v i d e s a b i l i t y to impor t t e x t f i l e s w i th IP and/ or hostnames# i n t o a s e n s o r t ha t would cause au to t agg i ng o f a l l matching s e s s i o n sp l u g i n s=tagge r . sot a g g e r I p F i l e s=b l a c k l i s t , tag , tag , tag . . .t a gge rDoma inF i l e s=doma i n b a s e db l a c k l i s t s , tag , tag , tag

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 59: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Moloch pluginsMoloch is easily extendable with your own plugins

I https://github.com/fygrave/moloch_zmq - makes it easy tointegrate other things with moloch via zmq queue pub/sub or push/pull model

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 60: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Moloch ZMQ example

CEP-based analysis of network-traffic (using ESPER):https://github.com/fygrave/clj-esptool/

( esp : add " c r e a t e ␣ con t e x t ␣SegmentedBySrc␣ p a r t i t i o n ␣by␣ s r c ␣ fromWebDataEvent" )( esp : add " con t e x t ␣SegmentedBySrc␣ s e l e c t ␣ s r c , ␣ r a t e (30) ␣ as ␣ ra t e ,avg ( r a t e ( 30 ) ) ␣ as ␣ avgRate ␣ from␣WebDataEvent . win : t ime (30) ␣ hav ingr a t e (30) ␣<␣avg ( r a t e ( 30 ) ) ␣∗␣ 0 .75 ␣ output ␣ snapshot ␣ e v e r y ␣60␣ sec " )( f u t u r e−c a l l s t a r t−coun t i ng )

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 61: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Detecting DGA botnets (moloch)Easy with our plugin. ;-)

I we want to label any IP addresses as ’suspicious’if they are generating more than X DNS packets per minute with rcode != 0

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 62: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Other Sources of IOCs

I ioc bucket:

http://iocbucket.com

I Public blacklists/trackers could also be used as source:

https://zeustracker.abuse.ch/blocklist.php?download=ipblocklisthttps://zeustracker.abuse.ch/blocklist.php?download=domainblocklist

I Eset IOC repository

https://github.com/eset/malware-iocmore coming?

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 63: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Tools: IoC lookup service

show me all the entries similar to this IOCWe implemented a whois service for IOC look-ups

whois −h ioc−ap i . ho s t . com a t t r i b u t e : v a l u e+a t t r i b u t e : v a l u e

We can return results in various formats: Snort, Yara, OpenIOC (ask for yourfavourite)

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 64: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Tools: Use YARA

r u l e susp_params_in_ur l_k ind_of_f i l e l e ss_bot_dr ive_by{

meta :date = " oct ␣2013"d e s c r i p t i o n = "Landing ␣hxxp :// j d a t a s t o r e l ame . i n f o / indexm . html ␣␣ 04 .10 . 2013 ␣ 13 :14 ␣␣ 108 . 62 . 112 . 84 ␣␣"d e s c r i p t i o n 1 = "␣Java ␣ S p l o i t ␣ hxxp :// j d a t a s t o r e l ame . i n f o /054RIwj ␣␣␣␣␣"

s t r i n g s :$ s t r i n g 0 = " ht tp "$ s t r i n g 1 = " indexm . html "$ s t r i n g 2 = "054RI"

c o n d i t i o n :a l l o f them

}

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 65: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Use snort to catch suspicious traffic:

# many plugX dep loyments connect to goog l e DNS when not i n usea l e r t tcp !$DNS_SERVERS any −> 8 . 8 . 8 . 8 53 (msg : "APT␣ p o s s i b l e ␣PlugX␣Google ␣DNS␣TCPpo r t ␣53␣ connec t i o n ␣ attempt " ; c l a s s t y p e : misc−a c t i v i t y ; s i d : 500000112 ;r e v : 1 ; )

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 66: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

GRR: Google Rapid Response:Other nice application of IoCs:http://code.google.com/p/grr/Hunting IOC artifacts with GRR

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 67: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

GRR: Creating rules

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 68: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

GRR: hunt in progress

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 69: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Conclusion

I Most of the tools shown here are opensource.

Either developed, contributed by me or by other good guys.I HP nodes are good source of compromise indicatorsI IoCs should be used with great-care. You need to know what you are

doing. ;-)I IoCs are getting easier to integrate with off-shelf security products

(no product advertisements here ;-))

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 70: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

Things to shareI We are very interested in data-sharingI Academia Sinica: we run anonymized IoC feed services (openioc XML

format)I Academia Sinica: we have custom HPFeeds brokers to facilitate data

sharingI Academia Sinica: we run our own passive DNSI We are very interested in new data sources and can help you to run

analysis platforms: (big data, time series analysis of network flows, DNStraffic, HTTP, IoC based pattern match, APK analysis).

Everything is free and open-source. Talk to us :)

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org

Page 71: Honeycon2014: Mining IoCs from Honeypot data feeds

Introduction IOC Standards V:IOCs mining IOCs Applying IOCs EOF

QuestionsI Questions?I Comments?

@fygrave ([email protected])

Mining compromise indicators from Honeypot Systems Affilations: Academia Sinica, o0o.nu, chroot.org