risk in cyber systems - amazon web services · 2018. 5. 14. · significant data exists in...
TRANSCRIPT
Risk in Cyber Systems
2/11/2016 1
Marshall KuypersPhD Candidate, Department of Management Science and Engineering
Stanford University
Dr. Elisabeth Paté-CornellProfessor, Department of Management Science and Engineering
Stanford [email protected]
Copyright Stanford, 2015
Notes meant for voice track are in blue bubbles Presented at the Society of Risk Analysis Annual Meeting
Arlington, Virginia. December 7-9, 2015.
significant uncertainty surrounds cyber security investments
Stanford 3 2/11/2016
Our research is motivated by the idea that…
4 2/11/2016
Subscription for threat intelTwo-factor authenticationData loss prevention
An organization considering three investments currently does not have a rigorous way to assess the value of different safeguards, or to quantify cyber risk.
Current methods are limiting
6 2/11/2016
PSATHand Waving
The cybersecurity framework in action: an Intel use case
Organizations use ‘people sitting around a table’ to make decisions, or rely on hand-wavy explanations from security vendors.
If a method exists, it is likely to be qualitative: Intel published an example, but the analysis may not have been data driven
rigorous, quantitative methods now exist
Stanford 8 2/11/2016
Quantitative approaches lead to more insights
Stanford 9 2/11/2016
ModelingData Analysis Risk Analysis
1
375
Malicious Insider
Website Compromises
Significant data exists in organizations!
Website Attacks
Malicious Email Laptop
Theft
Data Spillage
Insider
Use dollars
Use distributions, not averages
Our method is data driven, uses dollars, and uses distributions. Overall, we model the frequency and impact of different cyber attack categories and quantify risk
incident databases are treasure troves of intel
Stanford 11 2/11/2016
Our work has been successful in part because we’ve gotten access to security incident data. These…
Shellshock attacks
Stanford 13 2/11/2016
Shellshock publically announced on September 24th
Within 5 hours, a shellshock attack was detected
Thursday and Friday were the most common days for attacks
Incidents continued to occur for several months
Attacks did not correlate with US workday hours
We can analyze shellshock attacks
frequency and impact of cyber incidents can be quantified
Stanford 15 2/11/2016
We can also do a really good job of quantifying the frequency and impact of cyber security incidents.
Most incidents take less than 100 hours to resolve
Stanford 16 2/11/2016
Here we see 60,000 incidents that occurred at one organization over 6 years. The cost is measured by the time it takes to resolve an incident (investigation and remediation).
Lost devices: constant rate, decreasing impact
Stanford 21 2/11/2016
Large incidents do not occur after FDE is implemented
Rate of lost devices is remarkably consistent over time
Change in rate is due to reporting guidelines such as recording more device types (cellphones, tokens, etc.)
Incidents follow unique patterns that are often consistent over time. Here, we find that lost devices occur at a remarkably constant rate over time.
Malware: Decreasing rate, constant impact
Stanford 22 2/11/2016
Impact is constant (heavy-tailed)
Rate of incidents decreases over time
Large Events are NOT outliers
No ‘average’ or ‘typical’ cyber breach
Largest incident can be more impactful
than all other incidents combined!
Standard deviations and some risk
metrics (value at risk) are not valid
Malware incidents are decreasing in frequency, but have a consistent impact distribution that is heavy-tailed. This turns out to be very important for a number of reasons shown at right.
Investigation is a major cost, and can be quantified
23 2/11/2016
100
101
102
10-4
10-3
10-2
10-1
100
CC
DF
Investigation Time (Hours)
Lost Devices
α =2.77Xmin =9.5
100
101
102
103
10-4
10-3
10-2
10-1
100
CC
DF
Investigation Time (Hours)
α =2.09Xmin =4
100
101
102
10-5
10-4
10-3
10-2
10-1
100
CC
DF
Investigation Time (Hours)
Malware
α =3.76Xmin =20.5
100
101
102
103
10-5
10-4
10-3
10-2
10-1
100
CC
DF
Investigation Time (Hours)
Website
α =1.69Xmin =4
100
101
102
103
10-3
10-2
10-1
100
CC
DF
Investigation Time (Hours)
Spillage
α =2.07Xmin =3.5
100
101
102
103
10-5
10-4
10-3
10-2
10-1
100
CC
DF
Investigation Time (Hours)
Other
α =2.93Xmin =9
3 PII incidents removed1 system downtime incident removed
3 ‘network effect’ incidents removed
1 system downtime incident removed
1 system downtime incident removed
1 PII incident removed
After we assess the rate of incidents, we assess the impact by breaking it into several cost categories. Here, we have data on investigation costs, which are remarkably consistent over time.
Reputation damage uncertainty is modeled
24 2/11/2016
0 20 40 60 80 100 120 140 160 1800
0.002
0.004
0.006
0.008
0.01
0.012
0.014
Reputation Damage
Losses (Millions)
Density
Mars Global SurveyorFailure: 2006Cost:$154M to build, $65 to launch, $20M per year to operateDescription: Software update error causes computer crash and fried batteries
Mars Climate Orbiter Failure: 1999Cost:$193MDescription: metric and standard units conversion crashes the orbiter into mars
Reputation damage has been a hurdle in the past, but we explicitly model the uncertainty of losses (seen at right). For a case study, take chip manufacturer that stocks satellite parts. We can look at failures of satellites (that are cyber attack flavored, not attacks) to estimate costs. Academic research shows that stock prices only fall for 2 days after a breach, and we can look at Target, RSA, or Sony for other case studies.
Direct costs are well understood
25 2/11/2016
Probability Device Average Cost0.34 Cellphone $4000.32 Token $1000.20 Laptop $10000.07 Other $300
0.05 Desktop $10000.02 Tablet $700
0 200 400 600 800 1000 1200 1400 16000
0.005
0.01
0.015
0.02
0.025
Cost Distribution
Cost (Dollars)D
ensity
Cellphone
Token
Laptop
Desktop
Tablet
Other
Equipment Losses Extortion
Direct costs are well understood because we have data on lost devices, ransomware, and other direct costs.
Willingness-to-pay used for intellectual property losses
27 2/11/2016
We use a willingness-to-pay to elicit the cost of IP loss. We also use case studies like Solyndra and Cisco.
rolling this information together, we can obtain excellent risk assessments
Stanford 28 2/11/2016
Alpha 1.22, scale 0.827
Distribution
A case study demonstrates the method
29 2/11/2016
Impact Distributions (Data Spillage)
Investigation
IP Loss
Fines
Reputation
Rate of spillage incidents
𝜆Yearly Rate: 50
2% chance of losing more than $4M
30% chance of losing more than $20K
Case study: take the rate of incidents, model the impacts, and simulate. This graph shows the probability of different losses at an org.
A case study demonstrates the method
30 2/11/2016
Spillage is low risk
Losses from website compromises are usually small…
…but every so often, losses from websites are huge
FDE was cheap, and really reduced the risk curve
DLP is expensive, and still does not reduce tail risk
Co
mp
lem
enta
ry C
um
ula
tive
D
istr
ibu
tio
n F
un
ctio
n (
CC
DF)
Here, we roll all the attack types together, and obtain excellent insights from the quantitative analysis that can save organizations huge amounts of money.
Conclusions
31 2/11/2016
Probabilistic risk analysis methods inform actionabledecisions.
Incident data is priceless.
Safeguards can be compared and prioritized.
Monetary impacts help justify budgets and communicaterisk.