The Human Risk Survey

Dr. Lance Hayden

Solutions Architect – Cisco Global Security Services

September 10, 2014

Evolution of the SANS Security Awareness Survey

Cisco Public 2 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Introductions

What makes a good survey?

How can we make our security awareness survey better?

Using the Human Risk Survey

Examples of data and analysis

Lessons Learned

Session Talking Points

Cisco Confidential 3 Cisco Confidential 3 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Dr. Lance Hayden

Cisco Global Security Services

www.linkedin.com/in/drhayden

Introductions

Cisco Public 4 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

What Makes a Good (Security) Survey?

Good Survey Results

Demographic Information

Defined Purpose

Appropriate Structure

and Questions

Solid Analysis

xkcd

.co

m

Cisco Public 5 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Demographic Information

Who? Are you?

What? Do you do?

Where? Do you work?

When? Did you start?

Cisco Public 6 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

What is the instrument supposed to measure? What is it you want to know?

Also commonly known (in some circles) as a research question…

Is your survey primarily intended to be descriptive? Probably not.

You probably want to infer things from your data…

Are people making bad or uniformed security decisions?

Are security behaviors changing over time?

How effective are your training and awareness program efforts?

Defined Purpose

Cisco Public 7 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Survey Structure

Length

How many questions?

How long to complete?

Response

Why should people take it?

Can they answer the questions?

Are the answers valid and/or repeatable?

Data

Are results comparable?

Who is the audience for the findings?

Cisco Public 8 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Solid Analysis

What do we have?

• Responses

• Raw counts

What can we prove?

• Patterns

• Trends

What can we infer?

• Causes

• Appropriate Responses

Cisco Public 9 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

How Can We Make Our Security Survey Better?

Question our assumptions

Resist the lure of aggregate “risk scores”

Find the hidden stories in the data

Cisco Public 10 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Weighing security with a microscope…

“jet engine x peanut butter = shiny!”

Apples to Agent Orange…

Hitchens’ Razor

The Hubbard Axiom

Consider the Happiness Index

Why Aggregate Risk Scores are a Problem

Cisco Public 11 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Designing the Survey

The Human Risk Survey is a community project. Feel free to:

• Tweak it

• Add to it

• Rework / Reboot / Mashup

But be aware:

• Cross-organizational comparison requires normalized data

See guidelines and tips in the survey document for more insight.

• Do you have the right sponsorship and support?

• Is your sample of respondents representative?

• How do you keep track of respondents, changes, and results?

Using the Survey

Cisco Confidential 12 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Examples and Use Cases

Cisco Public 13 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

What an odd thing to ask…

Exploring user behavior

Exploring security program effectiveness

Exploring security culture

Examples and Use Cases

Cisco Public 14 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Do you ever feel pressure to “do more with less” in your job, even if that means cutting corners in some areas in order to complete others?

Always

Often, but not always

Sometimes

Not very often

Never

We seek to understand, “why do good people make bad decisions?”

Awareness is visibility into what is non-intuitive, but still important…

An Odd Thing to Ask..?

Cisco Public 15 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Exploring User Behavior

Q: How confident are you that you would recognize the symptoms and signs of a computer security incident?

05

101520253035

Respondents (n=100)

Cisco Public 16 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Exploring User Behavior

Q: How important do you feel it is to keep your computers, mobile devices and programs updated and current?

05

10152025303540

Respondents (n=100)

Cisco Public 17 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Exploring User Behavior

05

10152025303540

Respondents (n=100)

01020304050607080

Respondents (n=100)

Cisco Public 18 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Confidence to recognize an incident by most recent interaction with security team

Exploring Security Program Effectiveness

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

One Week One Month One Year Over a Year Never

Confident

Neutral

Not Confident

Cisco Public 19 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Exploring Security Culture

Q: How often do you feel pressure to do more with less? mapped against Q: Do people share passwords?

ALWAYS

OFTEN

SOMETIMES RARELY

NEVER

Sharing

No Sharing

Cisco Public 20 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Surveys are scientific instruments

Any tool can be misused or used poorly

The human vulnerability scanner metaphor

Avoid the trap of risk scores

Your data will always tell better stories, if you give them a voice

True insight rarely comes cheap or easy

The Human Risk Survey is a community resource. Use it. Improve it. Share it.

Lessons Learned

Cisco Public 21 © 2013-2014 Cisco and/or its affiliates. All rights reserved.

Lessons Learned

Surveys are scientific instruments

• The human scanner metaphor

• Any tool can be misused or used poorly

Avoid the “risk score” trap

• Data tell better stories, given a voice

• Real insight rarely comes easy (or cheap)

Assumptions and Audiences

• Make them explicit

• Answers lead to better questions

A Community Resource

• Use it!

• Improve it!

• Share it!

Human Risk Survey