vulnerability assessments: are you really doing them? · • vulnerability assessments are...
TRANSCRIPT
Roger G. Johnston, Ph.D., CPP
Right Brain Sekurity
http://rbsekurity.com
+1-630-551-0740
Vulnerability Assessments:
Are You REALLY Doing Them?
Threat: Who might attack, why, when, how,
with what probability, and with what resources.
(Includes information on goals and attack
modes.)
Threat Assessment (TA): Attempting to
identify threats.
Terminology
Vulnerability: Flaw or weakness that could be exploited to cause
undesirable consequences.
Vulnerability Assessment (VA): Creatively devising & discover-
ing (and perhaps demonstrating) ways to defeat a security
device, system, or program. Should include thinking like the bad
guys, and also suggesting countermeasures and security
improvements.
Terminology
mimics what
the bad guys do!
Threat: Adversaries might try to steal PII
information (SSNs, credit card numbers, etc.)
from our computer systems to commit crimes.
Vulnerability: We don’t keep our anti-malware
software up to date.
Threat vs. Vulnerability
4
•Pass a test
•“Test” security
•Generate metrics
•Justify the status quo
•Praise or accuse anybody
•Check against some standard
•Claim there are no vulnerabilities
•Engender warm & happy feelings
•Determine who gets salary increases
•Rationalize the research & development
•Apply a mindless, bureaucratic stamp of approval
•Endorse a security product/program or
Certify it as “good” or “ready to use”
The purpose of a VA is to improve security &
minimize risk, NOT to:
Purpose
5
A VA is Not… pen testing
“Red Teaming”
feature analysis
security auditing
quality control
threat assessment
reliability testing
efficiency testing
software scanning
compliance testing
acceptance testing
ergonomics testing
performance testing
response time testing6
operational assessment
fault or event tree analysis
(from safety engineering)
Design Basis Threat
a security survey
gap analysis
7
Questions Vulnerability Assessors Ask
And You Should, Too
Are vulnerabilities being confused with threats, assets
needing protection, security or infrastructure features, or
attack scenarios?
Are vulnerabilities being thought of as good news?
(They should be!)
Are VAs being confused with other things like TAs or
security “testing”?
Are they being done continuously, or at least frequently?
Vulnerability Assessments
8
Vulnerability Assessments (con’t)
Are the following kinds of employees (even if not security or
cyber experts) drafted to help examine your security:
trouble-makers, creative types, loophole finders,
questioners of authority, skeptics/cynics, hackers,
narcissists, hands-on enthusiasts, and puzzle solvers.
Resiliency & PR preparation for when
security inevitably fails?
Questions Vulnerability Assessors Ask And You Should, Too
9
Vulnerability Assessments (con’t)
Do your VAs suffer from any of these problems?
- sham rigor
- the Fallacy of Precision
- lack of imagination
- reactive not proactive
- done only be insiders
- shooting-the-messenger
- conflicts of interest
- cognitive dissonance
- focused only on high-tech attacks
- artificial constraints (scope, time, effort,
modules/components/disciplines)
- letting the good guys and the current security
infrastructure/strategy define the vulnerabilities &
attacks
Questions Vulnerability Assessors Ask And You Should, Too
10
Questions Vulnerability Assessors Ask And You Should, Too
Use hardware passwords & device IDs?
Have you changed the default password &
device ID, and security settings?
Devices adhere to emerging security standards?
Do the devices follow Minimalist Principles? range & power
duty cycle
bandwidth
data acquisition
data retention & duration
IoT Devices
11
Questions Vulnerability Assessors Ask And You Should, Too
Trusted manufacturers & vendors?
Is security built in from the start, or just a last
minute afterthought?
Early & iterative VAs on the devices?
Secure chain of custody?
IoT Devices (con’t)
12
Questions Vulnerability Assessors Ask And You Should, Too
Are your devices safe from physical/electronic tampering
(~20 secs), counterfeiting, and backdoor insertion including
• at vendor or factory?
• during shipments?
• on loading dock?
• before installation?
• after installation?
Chain of Custody for Devices*
13
Questions Vulnerability Assessors Ask And You Should, Too
Is there a lot of empty space inside your devices? Are they
frequently opened up and examined for tampering and
alien electronics? Do you know what the insides are
supposed to look like? Can you spot a counterfeit device?
Are you under the mistaken impression that:- “anti-counterfeiting” tags (even if high-tech) are difficult to lift or
counterfeit?
- tamper-indicating seals or packaging (even if high-tech) are difficult
to spoof, and trivial to use?
- sticky labels (even if high tech) provide effective tamper detection?
- a mechanical tamper switch is serious security?
- cargo/shipment supply chains are secure?
- engineers understand security?
Chain of Custody for Devices (con’t)
14
Are your physical access control systems designed by
the sales guy, amateurs, or your cyber security people?
Do your locked doors have hinges on the outside?
Can someone open the door without using the access
control system and without it knowing?
Does your physical access control system know when an
employee has left the control area?
Are you under the mistaken impression that biometric
access control devices can’t be easily defeated? That
biometric signatures can’t be easily counterfeited?
Physical Access Control for Cyber
Questions Vulnerability Assessors Ask And You Should, Too
15
Questions Vulnerability Assessors Ask And You Should, Too
Do you have Role-Based Access Control, so
that access is halted INSTANTLY when
someone is promoted, given a new assignment,
or terminated?
Do you periodically review access control
privileges for all employees?
General Access Control
16
Questions Vulnerability Assessors Ask And You Should, Too
Is HR’s role in security objectively evaluated at least annually?
Does HR harm security instead of helping it?
If HR is indeed evil (likely), do managers, supervisors, & security
managers try to compensate?
Do you rely on the 80% rule (“listen, empathize, validate”) to mitigate
insider threats?
Do narcissists get their ego stroked on a regular basis?
Are there constraints on bully/harassing bosses?
Are retiring and terminated employees treated well? Is there a perp-walk
for terminated employees? Is there considerable HR glee at
firing employees?
HR & Insider Threat Mitigation
17
Questions Vulnerability Assessors Ask And You Should, Too
Are background checks on key personnel done periodically
and thoroughly, including interviewing acquaintances?
Do you do bribery anti-stings?
HR & Insider Threat Mitigation (con’t)
18
Questions Vulnerability Assessors Ask And You Should, Too
Do you exploit psychology research?
Sign a pledge of honesty at the top of documents, not the bottom.
Angry eye posters in critical areas.
Warn well-paid employees of the risk to themselves if they do
something unethical, but warn low-paid employees of the potential
harm to others.
Social influence for better security
Sunk-cost bias
Countermeasures to groupthink & to cognitive dissonance
Research on creativity
If someone has a security concern, including
about a fellow employee, can they
submit it anonymously? Does everybody know how?
Is it safe? Does anybody do it?
What happens when they do?
HR & Insider Threat Mitigation (con’t)
19
Questions Vulnerability Assessors Ask And You Should, Too
Is Security getting confused with
Control
Hassling/Threating Employees
Privacy or Safety
Inventory Management
Compliance & Auditing
Is high-tech confused with high-security?
Is your security awareness & social engineering
training effective? One-size-fits-all?
Security Culture & Management
20
Questions Vulnerability Assessors Ask And You Should, Too
Do you warn employees about what happened elsewhere
after a serious security incident?
Do people affected by security rules have input about them?
Do security rules get reviewed often?
Is there unwarranted faith in “layered security”?
Security Culture & Management (con’t)
21
Questions Vulnerability Assessors Ask And You Should, Too
Are employees told what security attacks look like, or just
given an unmotivated list of “things not to do”?
Are security rules and procedures motivated and justified?
Is security “accountability” mostly through disciplining,
firing, or scapegoating people?
Are awards and recognition given for good security
practices, or is security only about bad news?
Security Culture & Management (con’t)
22
Questions Vulnerability Assessors Ask And You Should, Too
Have a cyber monoculture?
Overlook the security benefits of OpenBSD, Linux, Mac OS
X, and iOS, especially for routine use?
Is your SOC your NOC?
How do regular employees recognize legitimate IT personnel
and instructions?
Use of 2-Factor Authentication?
Cyber Specific Issues
23
Watch Out for Compliance-Based Security:
Compliance Can Harm Security!
Rule of Thumb: About 30% of security rules, standards, and
guidelines in large organizations make security worse!
How
Creates a false sense of security
Wastes security resources, energy, and attention
on bureaucratic busywork/documentation/auditing
Supplants thinking and paying attention in favor of
formalistic mindlessness
Increases insider threat with all the extra auditors,
documenters, and checkers checking the
checkers
24
Watch Out for Compliance-Based Security:
Compliance Can Harm Security!How
Makes auditors the enemy, not adversaries
Engenders cynicism about security when rules are:- outdated
- unmotivated
- one-size-fits-all
- Security Theater
- only followed by the good guys
- ignorant of local conditions, culture, & vulnerabilities
- not given a sanity check by those affected
Makes security the enemy of employees &
productivity
25
Watch Out for Compliance-Based Security:
Compliance Can Harm Security!How
Used as an excuse not to do better when
minimum requirements are met
Following one standard or guideline is used as
an argument that security is good in other,
unrelated areas
Some standards are just bad (drafted by
vendors, special interests, and bureaucrats)
26
Security Metrics
Are things that are important getting measured,
or only things that are easy to measure?
Is only quantity measured but not quality?
Are your metrics mostly about costs, security
management, routine cyber activity,
and/or past incidents, not about security
effectiveness?
27
Some Unconventional Security Metrics
“What If?” exercises (+)
Transparency* (+)
Controversy & thoughtful pushback (+)
Count quality & quantity of:
28
Some Unconventional Security Metrics
Informal contact rate between non-security employees and
security employees. (+)
How frequently the grievance complaint resolution
process is used. (+)
Percent of security personnel for whom security is a career
choice. (+)
How often do terms like ”hackers”, “adversaries”,
“tamperers”, “counterfeiters”, and “bad guys” appear
in oral and written communication? (+)
29
Some Unconventional Security Metrics
Do minor security incidents or errors serve as statistical
precursors to serious incidents?
Employee turnover rates and Security personnel turnover
rates. (-)
Number of security changes
recently introduced. (+)
30
Marginal Analysis1. Continuously try incremental changes (real or theoretic) in your security
to see if it improves & risk decreases.
2. If it does, try more change in that ”direction”. If not, try another direction.
3. Occasionally try large changes to try to escape local minima in the risk
surface.
4. Somewhat counter-intuitively, change multiple parameters at once.
5. You have “pretty good security” if changes do not significantly lower the
risk.
.
31
Marginal Analysis Advantages
It’s easier to judge incremental changes in
security than total absolute security
effectiveness.
The emphasis on change may help encourage
proactive, flexible security, and overcome
security inertia, groupthink, cognitive dissonance,
complacency, and boredom..
In Summary
• Vulnerability Assessments are different & better than pen
testing, “Red Teaming”, security audits, threat assess-
ments, etc. Be sure you are doing VAs frequently (and not
something else that is getting confused with a VA)!
• VAers ask a lot of questions. You should, too.
• There are many possible unconventional security metrics
you could consider—including Marginal Analysis.
• Don’t rely on Compliance-Based Security! Compliance
and good Security are not that well correlated.
32
This presentation (with references)
and additional papers/talks are
available at:
http://rbsekurity.com
(Use “Papers & Talks” Tab)
For More Information...
http://jps.rbsekurity.com
34
Resources“Security Maxims”, https://tinyurl.com/y94wekyn
“Devil’s Dictionary of Security Terms”, http://rbsekurity.com/Papers/devils.pdf
“Compliance Versus Security”, Journal of Physical Security 10(1), 77-81 (2017), http://jps.rbsekurity.com
“Some Unconventional Security Metrics”, Journal of Physical Security 10(1), 82-85 (2017), http://jps.rbsekurity.com
“What Vulnerability Assessors Know That You Should, Too” Asia Pacific Security Magazine 50, 40-42, Aug/Sept 2013.
“Avoiding Shock and Awe”, Journal of Physical Security 9(1), 26-48 (2016) , http://jps.rbsekurity.com
wikiHow, “How to Validate Someone’s Feelings”, https://www.wikihow.com/Validate-Someone%27s-Feelings
”Time Series Analysis”, http://r-statistics.co/Time-Series-Analysis-With-R.html
Wikipedia, “Detrended Fluctuation Analysis”, https://en.wikipedia.org/wiki/Detrended_fluctuation_analysis.
R. Herold, “Do Compliance Requirements Help or Hurt Information Security?”, http://www.realtimepublishers.com/chapters/1699/esitcv1-13.pdf
J. Ross, Discover, “Signing a Form at the Top”, http://blogs.discovermagazine.com/80beats/2012/09/06/liar-liar-bottom-signer-signing-a-form-at-the-top-
leads-to-more-honest-answers/
J. Metcalfe, “Posters of Angry Eyes”, https://www.citylab.com/transportation/2013/04/posters-angry-eyes-actually-scare-bike-thieves/5420/
N Charky, “Eyeballs Have an Interesting Effect on Your Behavior”, https://archive.attn.com/stories/2854/eyeballs-effect-on-crime
M Hutson, “Rich People and Poor People Cheat for Different Reasons”,
https://www.thecut.com/2015/02/rich-and-poor-people-cheat-for-different-reasons.html
SS Wiltermuch, “Cheating More When the Spoils are Split”, Organizational Behavior and Human Decision Processes, 115(2), 157-168 (2001).
A Michel, “Psyber Securit”y, https://www.psychologicalscience.org/observer/psyber-security-thwarting-hackers-with-behavioral-science
R Anderson, “Psychology and Security Resource Page, https://www.cl.cam.ac.uk/~rja14/psysec.html
RG Johnston, “Security Sound Bites: Important Ideas About Security from Smart Ass, Dumb Ass, and Kick Ass Quotations”.
The Journal of Physical Security, http://jps.rbsekurity.com