fix what matters
DESCRIPTION
A deep look inside real-world vulnerability, remediation and breach stats.TRANSCRIPT
![Page 1: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/1.jpg)
Fix What MattersEd Bellis & Michael Roytman
![Page 2: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/2.jpg)
Nice To Meet You
• CoFounder Risk I/O
About Us
Risk I/O
• Former CISO Orbitz• Contributing Author: Beautiful Security• CSO Magazine/Online Writer
• Data-Driven Vulnerability Intelligence Platform• DataWeek 2012 Top Security Innovator• 3 Startups to Watch - Information Week
• InfoSec Island Blogger
• 16 Hot Startups - eWeek
Ed Bellis
• Naive Grad Student• Still Plays With Legos• Barely Passed Regression Analysis
• Once Jailbroke His iPhone 3G• Has Coolest Job In InfoSec
Michael Roytman
![Page 3: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/3.jpg)
Starting From Scratch
“It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories,
instead of theories to suit facts.”
-Sir Arthur Conan Doyle, 1887
![Page 4: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/4.jpg)
Starting From Scratch
![Page 5: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/5.jpg)
Starting From Scratch
Academia!• GScholar!• JSTOR!• IEEE!• ProQuest!
InfoSec Blogs!• CSIOs!• Pen Testers!• Threat Reports!• SOTI/DBIR!!
Twitter!• Thought Leaders (you
know who you are)!• BlackHats!• Vuln Researchers!
Primary Sources!• MITRE!• OSVDB!• NIST CVSS
Committee(s)!• Internal Message
Boards for ^!Text
CISOs
![Page 6: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/6.jpg)
Data Fundamentalism
Don’t Ignore What a Vulnerability Is: Creation Bias
(http://blog.risk.io/2013/04/data-fundamentalism/)
Jerico/Sushidude @ BlackHat
(https://www.blackhat.com/us-13/briefings.html#Martin)
Luca Allodi - CVSS DDOS
(http://disi.unitn.it/~allodi/allodi-12-badgers.pdf):
![Page 7: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/7.jpg)
Data Fundamentalism - What’s The Big Deal?
”Since 2006 Vulnerabilities have declined by 26 percent.” (http://csrc.nist.gov/groups/SNS/rbac/documents/vulnerability-trends10.pdf)
“The total number of vulnerabilities in 2013 is up 16 percent so far when compared to what we saw in the same time period in 2012. ”
(http://www.symantec.com/content/en/us/enterprise/other_resources/b-intelligence_report_06-2013.en-us.pdf)
![Page 8: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/8.jpg)
What’s Good?
Bad For Vulnerability Statistics:
NVD, OSVDB, ExploitDB, CVSS, Patches, Microsoft Reports, etc, et al, and so on.
Good For Vulnerability Statistics:
Vulnerabilities.
![Page 9: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/9.jpg)
What’s Good?
![Page 10: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/10.jpg)
What’s Good?
![Page 11: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/11.jpg)
What’s Good?
![Page 12: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/12.jpg)
What’s Good?
![Page 13: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/13.jpg)
What’s Good?
![Page 14: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/14.jpg)
What’s Good?
![Page 15: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/15.jpg)
Counterterrorism
Known Groups
Surveillance
Threat Intel, Analysts
Targets, Layouts
Past Incidents, Close Calls
![Page 16: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/16.jpg)
What’s Good?
![Page 17: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/17.jpg)
Uh, Sports?
Opposing Teams, Specific Players
Gameplay
Scouting Reports, Gametape
Roster, Player Skills
Learning from Losing
![Page 18: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/18.jpg)
InfoSec?
![Page 19: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/19.jpg)
Defend Like You’ve Done It Before
Groups, Motivations
Exploits
Vulnerability Definitions
Asset Topology, Actual Vulns on System
Learning from Breaches
![Page 20: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/20.jpg)
Work With What You’ve Got:
Akamai, Safenet
ExploitDB, Metasploit
NVD, MITRE
![Page 21: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/21.jpg)
Add Some Spice
![Page 22: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/22.jpg)
Show Me The Money
23,000,000 Vulnerabilities!
Across 1,000,000 Assets!
Representing 9,500 Companies!
Using 22 Unique Scanners!
![Page 23: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/23.jpg)
Whatchu Know About Dat?(a)
Duplication
Vulnerability Density
Remediation
![Page 24: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/24.jpg)
Duplication
0
225,000
450,000
675,000
900,000
1,125,000
1,350,000
1,575,000
1,800,000
2,025,000
2,250,000
2 or more scanners 3 or more 4 or more 5 or more 6 or more
![Page 25: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/25.jpg)
Duplication - Lessons From a CISO
We Have: F(Number of Scanners) => Number of Duplicate Vulnerabilities
We Want: F(Number of Scanners) => Vulnerability Coverage
Make Decisions At The Margins!
<---------Good Luck!
0
25.0
50.0
75.0
100.0
0 1 2 3 4 5 6
![Page 26: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/26.jpg)
Density
Type of Asset ~Count
Hostname 20,000
Netbios 1000
IP Address 200,000
File 10,000
Url 5,000
Hostname
Netbios
IP
File
Url
0 22.5 45.0 67.5 90.0
![Page 27: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/27.jpg)
CVSS And Remediation Metrics
0
375.0
750.0
1125.0
1500.0
1 2 3 4 5 6 7 8 9 10
Average Time To Close By Severity Oldest Vulnerability By Severity
![Page 28: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/28.jpg)
CVSS And Remediation - Lessons From A CISO
1 2 3 4 5 6 7 8 9 10
Remediation/Lack Thereof, by CVSS
1 2 3 4 5 6 7 8 9 10
NVD Distribution by CVSS
![Page 29: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/29.jpg)
The Kicker - Live Breach Data
1,500,000 !Vulnerabilities Related to Live Breaches Recorded!
June, July 2013 !
![Page 30: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/30.jpg)
CVSS And Remediation - Nope
0
1750.0
3500.0
5250.0
7000.0
1 2 3 4 5 6 7 8 9 10
Oldest Breached Vulnerability By Severity
![Page 31: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/31.jpg)
CVSS - A VERY General Guide For Remediation - Yep
0
37500.0
75000.0
112500.0
150000.0
1 2 3 4 5 6 7 8 9 10
Open Vulns With Breaches Occuring By Severity
![Page 32: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/32.jpg)
The One Billion Dollar Question
Probability(You Will Be Breached On A Particular Open Vulnerability)?
1.98%=(Open Vulnerabilities | Breaches Occurred On Their CVE)/(Total Open Vulnerabilities)
![Page 33: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/33.jpg)
I Love It When You Call Me Big Data
RANDOM VULN
CVSS 10
CVSS 9
CVSS 8
CVSS 6
CVSS 7
CVSS 5
CVSS 4
Has Patch
0 0.01000 0.02000 0.03000 0.04000
Probability A Vulnerability Having Property X Has Observed Breaches
![Page 34: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/34.jpg)
Enter The Security Mendoza Line
Wouldn’t it be nice if we had something that helped us divide who we considered “Amateur” and who we considered “Professional”?
http://riskmanagementinsight.com/riskanalysis/?p=294
Josh Corman expandsthe Security Mendoza Line
“Compute power grows at the rate of doubling about every 2
years”
“Casual attacker power grows at the rate of Metasploit”
http://blog.cognitivedissidents.com/2011/11/01/intro-to-hdmoores-law/
Alex Hutton comes up with Security Mendoza Line
![Page 35: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/35.jpg)
I Love It When You Call Me Big Data
Random Vuln
CVSS 10
Exploit DB
Metasploit
MSP+EDB
0 0.1 0.2 0.2 0.3
Probability A Vulnerability Having Property X Has Observed Breaches
![Page 36: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/36.jpg)
Be Better Than The Gap
![Page 37: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/37.jpg)
I Love It When You Call Me Big Data
Spray and Pray => 2%
CVSS 10 => 4%
Metasploit + ExploitDB => 30%
![Page 38: Fix What Matters](https://reader034.vdocuments.us/reader034/viewer/2022052321/554beb87b4c9055a368b4e80/html5/thumbnails/38.jpg)
Thank You
Follow UsBlog: http://blog.risk.ioTwitter: @mroytman
@ebellis@riskio
We’re Hiring! http://www.risk.io/jobs