Download - Fix What Matters: BSidesDetroit 2014
![Page 1: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/1.jpg)
Fix What Matters: !
Why CVSS Sucks And How To
Do Better
![Page 2: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/2.jpg)
Once Jailbroke an Iphone 3G
Michael Roytman
Proud Owner of Remote Controlled AirplaneRecently a Naive Grad Student
Data Scientist, Risk I/ODoes Not Wake Up Before 11 CST
qualifications:
![Page 3: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/3.jpg)
15x better than CVSS
![Page 4: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/4.jpg)
Probability A Vuln Having Property X Has Observed Breaches
Random Vuln
CVSS 10
Exploit DB
Metasploit
MSP+EDB
0.0 0.1 0.2 0.2 0.3
![Page 5: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/5.jpg)
PART 1: !
YOU SUCK AT YOUR JOB
!
(and don’t even know it yet)
![Page 6: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/6.jpg)
Why Are We Here?
Empirical Failures of CVSSProper Remediation Frameworks
CVSS SUCKS
Analytical Failures of CVSS
(+Data Driven Alternatives)
![Page 7: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/7.jpg)
Remove the Threat
RemediationAccept the Risk
Repair the Vulnerability
![Page 8: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/8.jpg)
C(ommon) V(ulnerability) S(coring) S(ystem)
“CVSS is designed to rank information system vulnerabilities”
Exploitability/Temporal (Likelihood)
Impact/Environmental (Severity)
The Good: Open, Standardized Scores
![Page 9: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/9.jpg)
“It is a capital mistake to theorize before one has data.
!
!
!
Insensibly, one begins to twist facts to suit theories, instead of
theories to suit facts.”
![Page 10: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/10.jpg)
FAIL 1: A Priori Modeling“Following up my previous email, I have tweaked my equation to try to achieve better separation between adjacent scores and to have CCC have a perfect (storm) 10 score...There is probably a way to optimize the problem numerically, but doing trial and error gives one plausible set of parameters...except that the scores of 9.21 and 9.54 are still too close together. I can adjust x.3 and x.7 to get a better separation . . .”
![Page 11: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/11.jpg)
F2: Data FundamentalismDon’t Ignore What a Vulnerability Is: Creation Bias http://blog.risk.io/2013/04/data-fundamentalism/ !
Jerico/Sushidude @ BlackHat https://www.blackhat.com/us-13/briefings.html#Martin
!
Luca Allodi - CVSS DDOS http://disi.unitn.it/~allodi/allodi-12-badgers.pdf
![Page 12: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/12.jpg)
F2: Data FundamentalismSince 2006 Vulnerabilities have declined by 26 percent.” http://csrc.nist.gov/groups/SNS/rbac/documents/vulnerability-trends10.pdf !
!
The total number of vulnerabilities in 2013 is up 16 percent so far when compared to what we saw in the same time period in 2012. ” http://www.symantec.com/content/en/us/enterprise/other_resources/b-intelligence_report_06-2013.en-us.pdf
![Page 13: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/13.jpg)
F3: Stochastic Ignorance
Attackers Change Tactics Daily
![Page 14: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/14.jpg)
F3: Stochastic Ignorance
![Page 15: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/15.jpg)
Empirical Failures of CVSSObjective: Remediate the riskiest vulnerabilities
Constraint: Can’t measure impact/priority
Need:
MOAR DATA!!!
![Page 16: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/16.jpg)
Repair the Vulnerability
![Page 17: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/17.jpg)
I Love It When You Call Me Big Data50,000,000 Live Vulnerabilities
1,500,000 Assets
2,000 Organizations
![Page 18: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/18.jpg)
I Love It When You Call Me Big Data
3,000,000 Breaches
![Page 19: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/19.jpg)
Baseline AllthethingsProbability (You Will Be Breached On A Particular Open Vulnerability)?
=(Open Vulnerabilities | Breaches Occurred On Their CVE) /(Total Open Vulnerabilities)
2%
![Page 20: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/20.jpg)
Probability A Vuln Having Property X Has Observed Breaches
RANDOM VULN
CVSS 10
CVSS 9
CVSS 8
CVSS 6
CVSS 7
CVSS 5
CVSS 4
Has Patch
0.000 0.010 0.020 0.030 0.040
![Page 21: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/21.jpg)
PART 2: !
FIX WHAT MATTERS
![Page 22: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/22.jpg)
Empirical Failures of CVSSObjective: Remediate the riskiest vulnerabilities
Constraint: Can’t measure impact/priority
Need:
MOAR DATA!!!
![Page 23: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/23.jpg)
Proper Framework
Know which vulnerabilities put you most at risk.
![Page 24: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/24.jpg)
![Page 25: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/25.jpg)
![Page 26: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/26.jpg)
![Page 27: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/27.jpg)
![Page 28: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/28.jpg)
![Page 29: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/29.jpg)
![Page 30: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/30.jpg)
![Page 31: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/31.jpg)
Uh, Sports?
Opposing Teams, Specific Players
Gameplay
Scouting Reports, Gametape
Roster, Player Skills
Learning from Losing
![Page 32: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/32.jpg)
InfoSec?
![Page 33: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/33.jpg)
Defend Like You’ve Done It Before
Groups, Motivations
Exploits
Vulnerability Definitions
Asset Topology, Actual Vulns on System
Learning from Breaches
![Page 34: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/34.jpg)
Work With What You’ve Got:
Akamai, Safenet
ExploitDB, Metasploit
NVD, MITRE
![Page 35: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/35.jpg)
Alternatives
![Page 36: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/36.jpg)
Probability A Vuln Having Property X Has Observed Breaches
Random Vuln
CVSS 10
Exploit DB
Metasploit
MSP+EDB
0.0 0.1 0.2 0.2 0.3
![Page 37: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/37.jpg)
Be Better Than The Gap
![Page 38: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/38.jpg)
I Love It When You Call Me Big Data
!
Spray and Pray => 2% !
CVSS 10 => 4% !
Metasploit + ExploitDB => 30%
![Page 39: Fix What Matters: BSidesDetroit 2014](https://reader033.vdocuments.us/reader033/viewer/2022052822/554beb77b4c9055a368b4e79/html5/thumbnails/39.jpg)
Holler!www.risk.io@mroytman