meetup - @jsendor jakub (kuba) sendor yelp's malware ...files.meetup.com/16943162/yelp's...

Yelp's Malware Incident ResponseMarch 2016 Bay Area Cyber Security Meetup

Jakub (Kuba) Sendor@jsendor

@jsendor

whoami● Joined Yelp security team in July 2014.

● Mostly involved in malware incident response.

● Also working on automating our security processes.

● Previously worked at SAP in Sophia Antipolis (France) in the Security &

Trust research group.

● Before that: MSc from AGH University of Science and Technology in

Kraków (Poland) and Telecom ParisTech/Institut Eurecom (France).

@jsendor

Yelp’s Mission:Connecting people with great

local businesses.

@jsendor

Yelp Stats:As of Q4 2015

86M 3270%95M

@jsendor

Yelp Stats:As of Q4 2015

> 300> 3000

@jsendor

Malware response process at a glance

Detection Analysis Remediation

@jsendor

DetectionVarious alert sources:● endpoint monitoring

○ antivirus○ osquery

● network traffic monitoring● SIEM (Security Incident and Event Management)● email (phishing, adware, popups, etc.)

@jsendor

AIR: Automated Incident ResponseAV

Filter out potential false positives

Email HelpDesk Cut ticket

Match employee office

@jsendor

{

"UserName": "YELP-KUBA\\kuba",

"ThreatType": "Viruses",

"@timestamp": "2016-02-28T15:06:20.868Z",

"ScannerType": "On demand",

"InsertedAt_UTC": "2016-02-28 15:11:27",

"Status": "Cleanable",

"ComputerDomain": "AD",

"StatusID": "300",

"FullFilePath": "/Users/kuba/Downloads/4akAhdUB.exe.part",

"ComputerName": "YELP-1234",

"EventTime_UTC": "2016-02-28 15:11:18",

..

}

Antivirus alert

@jsendor

osquery● kernel extensions

● user logins

● config file hashes

● browser extensions

● startup items

● launchd

@jsendor

Alerting pipeline

report collect indexalert

visualize

@jsendor

The Men Who Stare at Goats Graphs

@jsendor

ElastAlertAlerting out of data in Elasticsearch indexes.

https://github.com/Yelp/elastalert

@jsendor

ElastAlerthttp://engineeringblog.yelp.com

@jsendor

{

"@ingestionTime": "2016-02-28T15:05:33Z",

"_id": "AVLwlmFxKVkRUjUGMJlD",

"_index": "logstash-osquery-osx-weekly-2016.09",

"_type": "osquery",

"columns": {

"name": "Window Resizer",

"path": "/Users/kuba/Library/Application Support/Google/Chrome/Profile 1/Extensions/kkelicaakdanhinjdeammmilcgefonfh/1.9.1.2_0/"

},

"filter_result": "blacklisted",

"hostIdentifier": "A43F47D0-A921-5895-8A59-AB49EB616A5D",

"kibana_link": "https://..."

}

osquery + ElastAlert

@jsendor

ElastAlert rules● frequency

● spikes

● flatline

● timeframes

@jsendor

Spikes in DNS block

This machine had one day with more than 20 blocked DNS lookups, and at least three subsequent days with

more than 2 blocked DNS lookups. It should be examined.

('2016-01-09', 21, Counter({'standout[.]tv[.]': 21}))

('2016-01-10', 6, Counter({'ads2[.]contentabc[.]com[.]': 6}))

('2016-01-11', 5, Counter({'bttrack[.]com[.]': 2, 'cdn[.]bttrack[.]com[.]': 2,

'94982c5b634975e50103ce96082d2827[.]adsk2[.]co[.]': 1}))

('2016-01-12', 20, Counter({'ads2[.]contentabc[.]com[.]': 8, 'loadm[.]exelator[.]com[.]': 5, 'standout[.]

tv[.]': 3, 'loadus[.]exelator[.]com[.]': 2, 'secure-au[.]imrworldwide[.]com[.]': 1, '1049theeagle[.]com

[.]': 1}))

('2016-01-13', 47, Counter({'ads2[.]contentabc[.]com[.]': 22, 'www[.]4chan[.]org[.]': 14, 'sys[.]4chan[.]

org[.]': 8, '4chan[.]org[.]': 2, 'cdn[.]directrev[.]com[.]': 1}))


@jsendor

Analysis● False positive?

● Wrong OS?

● Who is it?

● How did that malware get there?

● Is the machine really infected?

@jsendor

Spikes in DNS block

This machine had one day with more than 20 blocked DNS lookups, and at least three subsequent days with

more than 2 blocked DNS lookups. It should be examined.

('2016-01-09', 21, Counter({'standout[.]tv[.]': 21}))


('2016-01-11', 5, Counter({'bttrack[.]com[.]': 2, 'cdn[.]bttrack[.]com[.]': 2,

'94982c5b634975e50103ce96082d2827[.]adsk2[.]co[.]': 1}))

('2016-01-12', 20, Counter({'ads2[.]contentabc[.]com[.]': 8, 'loadm[.]exelator[.]com[.]': 5, 'standout[.]

tv[.]': 3, 'loadus[.]exelator[.]com[.]': 2, 'secure-au[.]imrworldwide[.]com[.]': 1, '1049theeagle[.]com

[.]': 1}))

('2016-01-13', 47, Counter({'ads2[.]contentabc[.]com[.]': 22, 'www[.]4chan[.]org[.]': 14, 'sys[.]4chan[.]

org[.]': 8, '4chan[.]org[.]': 2, 'cdn[.]directrev[.]com[.]': 1}))


@jsendor

Requesting osquery data on the host

Found 660 launch daemons for victim machine

Checking incidence of launch daemons in general population

........................................................................................................

00001 launch daemons named /Users/joel/Library/LaunchAgents/com.apple.macbuddy.icloudsetup.user.plist

found

00001 launch daemons named /Library/LaunchDaemons/com.avid.bsd.DigiShoeTool.plist found

00001 launch daemons named /Users/joel/Library/LaunchAgents/com.adobe.AAM.Updater-1.0.plist found

00002 launch daemons named /Users/joel/Library/LaunchAgents/com.jdibackup.ZipCloud.notify.plist found

00002 launch daemons named /Users/joel/Library/LaunchAgents/com.spotify.webhelper.plist found

00002 launch daemons named /Users/joel/Library/LaunchAgents/com.jdibackup.ZipCloud.autostart.plist found

00002 launch daemons named /Users/joel/Library/LaunchAgents/com.facebook.videochat.joel.plist found

00002 launch daemons named /Users/joel/Library/LaunchAgents/com.nero.HSMMonitor.plist found

00002 launch daemons named /Users/joel/Library/LaunchAgents/com.adobe.ARM.

925793fb327152fd34795896fa1fb9ffa268b2

@jsendor

@jsendor

$ sudo osxcollector.py --id BlossomingLotusWrote 35394 lines.Output in BlossomingLotus-2016_02_28-15_08_38.tar.gz$

1 Python file0 dependencies

@jsendor

OS System Info Applications Web Browser Info

Kernel Extensions Quarantines Email Info

Downloads Startup Items Groups & Accounts

@jsendor

{ "file_path": "/System/Library/Extensions/Apple_iSight.kext/Contents/MacOS/Apple_iSight", "sha2": "19b7b85eaedb17d9565dce872f0d1ea8fc0761f508f28bedcc8606b828cbf614", "sha1": "99005b68295c202fd359b46cd1411acea96b2469", "md5": "b8cc164b6546e4b13768d8353820b216", "ctime": "2014-12-05 16:50:39", "mtime": "2014-09-19 00:16:50", "osxcollector_section": "kext", "osxcollector_incident_id": "BlossomingLotus-2016_02_28-15_12_46", "osxcollector_plist_path": "/System/Library/Extensions/Apple_iSight.kext/Contents/Info.plist", "osxcollector_bundle_id": "com.apple.driver.Apple_iSight", "signature_chain": [ "Software Signing", "Apple Code Signing Certification Authority", "Apple Root CA" ]}

@jsendor

Shadowserver API

OpenDNS Investigate API

Internal blacklists

VirusTotal API

Browser history filter

JSONin

JSONout

@jsendor

We put stuff on a blacklist for a reason. Mostly so you don't do this.

- applications applications ctime: "2015-04-13 10:15:32" file_path: "/Applications/MacKeeper.app/Contents/Resources/ZBRemoteSupport.app/Contents/MacOS/ZBRemoteSupport" md5: "50be328745e25afc875842ed578cd3fa" mtime: "2013-01-29 07:03:51" sha1: "f22e7953d0d360956fd43cb79788676e1af60700" sha2: "03ed9cb6e46221d219127b07e1d139132c05509f90636ee1da76c9610a67ae3f" blacklist-hashes: ["50be328745e25afc875842ed578cd3fa"] related-files: ["mackeeper.app"]

- chrome history id: 627 name: "http://stream2watch.me/" url_id: 291987 blacklist-domains: ["stream2watch.me"]

Analysis summary

@jsendor

https://github.com/Yelp/osxcollector

@jsendor

Threat Intel API

https://github.com/Yelp/threat_intel

@jsendor

Phishing

@jsendor

● employee education

● email alias for reporting phishing attempts

● reward positive behavior

● automated email scanning

Phishing

@jsendor

Analyzing phishing emails● analyze message headers

● detonate attachments

● past user interaction

● who else received it?

● https://www.phishtank.com/

@jsendor

Remediation

courtesy of @sroberts https://github.com/Yelp/osxcollector/pull/70

@jsendor

Remediation, more seriously● DNS/firewall blocking

● update IoCs (Indicators of Compromise)

● block/quarantine email senders

● whitelisting

● communication

@jsendor

Recap

Detect Analyze Remediate● endpoint protection● network monitoring● SIEM● employees

● collect forensics● correlate

information● automated analysis

● wipe :(● block at

DNS/firewall● blacklist/whitelist● educate

@jsendor

Improving the response process

faster response

better tools education

reduce the number offalse positives

@jsendor

Thanks for tuning in!

meetup - @jsendor jakub (kuba) sendor yelp's malware ...files.meetup.com/16943162/yelp's...

Documents