leveraging human factors for effective security training, for issa 2013 ciso forum, in pittsburgh...

©2

00

9 C

arn

eg

ie M

ello

n U

niv

ers

ity :

1

Leveraging Human Factors for Effective Security Training

ISSA CISO Forum 2013

Jason HongAssociate Professor

Carnegie Mellon University

CTO and Co-FounderWombat Security Technologies

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

2

Interactions Can Be Successful

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

3

Interactions Can Also Fail

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

4

HumanRobot

Interaction

SocialWeb

CognitiveTutors

NewInteractionTechniques

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

5

Human Factors Issues in Cybersecurity

• Studying human factors issues in cybersecurity for 9+ years– Why do people fall for phishing scams?– How can we train people in a manner that

is fun, effective, and measurable?– How can we build better user interfaces

and security warnings?

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

6

InfluencedMSIE

Warnings

WombatSecurity

Technologies

SciAm&

CACM

APWGLanding

Page

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

7

Today’s Talk

• Discuss some of our research findings– Better user interfaces for avoiding attacks– Teaching people effectively

• A model for thinking about cybersecurity awareness and education

• Three cross-cutting strategies for effective cybersecurity training

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

8

• Every browser now has basic anti-phishing detection built in

• Are these user interfaces effective?• Our 2008 study on warnings

• And what does it mean for training?

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

9

Screenshots

Internet Explorer 7 – Passive Warning

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

10

Screenshots

Internet Explorer 7 – Active Block

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

11

Screenshots

Mozilla Firefox – Active Block

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

12

Tested These Four Interfaces

• Shopping study– IE Passive Warning– IE Active Block– FireFox Active Block– Control (no warnings or blocks)

• Overall results– Passive warning completely ineffective– About half of people still fell for IE warning– No one fell for FireFox warning

S. Egelman, L. Cranor, and J. Hong. You've Been Warned: An Empirical Study of the Effectiveness of Web Browser Phishing Warnings. CHI 2008.

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

13

Analyzing the Results

• C-HIP model for real-world warnings– See the warning?– Understand it?– Believe it?– Motivated?– Can and will act?

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

14

Screenshots

• MSIE 7 Active Block• Half still fell for phish

despite the warning (?)• Habituation (similar warnings)• Two pathological cases• Most saw the warning, but

many did not believe it• “Since it gave me the option of

still proceeding to the website, I figured it couldn’t be that bad”

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

15

Two Takeaways

• Better interfaces can dramatically reduce security problems

• Model for warnings also relevant for cybersecurity in general– See the warning?– Understand it?– Believe it?– Motivated?– Can and will act?

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

16

Basis for the Cybersecurity

Training Model

Aware of the security issue?

Knowledge of what actions to take?

Motivated to act?

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

17

Cybersecurity Training ModelExample: Passwords



Motivated to act?

Don’t reuse passwordsCommon security risk

How to changeSecure and memorable

Stories of breachesRequire changes

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

18

Cybersecurity Training ModelExample: Smartphone Security



Motivated to act?

Have a PIN on device(about 50% don’t)

How to do it on deviceAvoiding bad PINs

At end of trainingStart with upper mgt

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

19

Cybersecurity Training Model



Motivated to act?

• Most training starts with awareness

• Unfortunately, most training also stops with awareness

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

20

Most Posters not Effective

http://mindfulsecurity.com/2009/09/19/free-threats-security-awareness-posters/

http://mindfulsecurity.com/2009/09/19/free-threats-security-awareness-posters/

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

21

Cybersecurity Training Model

• Effective training needs to address all these steps

• Strategy #1– Foster better

mental models



Motivated to act?

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

22

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

23

Mental Models

• People inevitably build models of how things work– Ex. me and my car– Ex. children & computers– Ex. maps of New York

and Boston

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

24

Mental Models Impact Security

• Ex. visibility in Facebook– Suppose you have a private

Facebook album, but tag someone. Can that person see it or not?

• Ex. app stores– All apps are vetted by

Google, so they are all safe to download. Correct?

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

25

So, we just have to foster the right mental model and then we’re done?

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

26

There’s not Always a “Right” Mental Model• Experts can disagree on• We asked 10 experts about malware

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

27

Incomplete Mental Models Can Still Be Useful

• Rick Wash’s work on folk models– Hackers are technical geeks that do it for fun– Hackers seek personal info– Hackers only target big fish– Hackers only look for big databases of info– People took different precautions

• Incomplete models may still be an improvement over current state– Degrees of better and worse

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

28

Cybersecurity Training

• Cybersecurity education should foster better mental models– Awareness– Who and why?– Fixing common

misconceptions– Actionable items



Motivated to act?

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

29

Case Study: Phishing Attacks

• Interviewed 40 people as part of an “email study” (Downs et al, SOUPS 2006)

• Only 55% of participants said they had ever noticed an unexpected or strange-looking URL– Most did not consider them to be suspicious

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

30

Example: Phishing Attacks

• 55% of participants reported being cautious when email asks for sensitive financial info– But very few reported being suspicious of

email asking for passwords

• Knowledge of financial phish reduced likelihood of falling for these scams– But did not transfer to other scams, such

as an amazon.com password phish

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

31

• Strategy #2: Tailor delivery of training for your audience– We’re all busy– A lot of training is boring (wall of text)– Little chance to test what you just learned


Teachable Moments Micro-Games

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

32

PhishGuru Simulated Phishing

• Create teachable moments thru simulated phishing emails

• If recipient falls for it, show intervention that teaches what cues to look for– Useful for people who don’t know what

they don’t know (low awareness)

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

33

Subject: Revision to Your Amazon.com Information

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

34

Subject: Revision to Your Amazon.com Information

Please login and enter your information

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

35

• Why am I seeing this?• How was I tricked?• How to protect myself?• Who and how?

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

36

Evaluation of PhishGuru

• Is simulated phishing effective?– We’ve done 4 peer-reviewed studies

showing embedded training works well– About 50% decrease in falling for phish

after one training

P. Kumaraguru et al. Protecting People from Phishing: The Design and Evaluation of an Embedded Training Email System. CHI 2007.

P. Kumaraguru et al. School of Phish: A Real-Word Evaluation of Anti-Phishing Training. SOUPS 2009.

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

37

Results of One Study

• Tested 500+ people in one month– 1 simulated phish at beginning of month,

testing done at end of month

• ~50% reduction in falling for phish– 68 out of 85 surveyed recommend continuing

doing this sort of training in the future

“I really liked the idea of sending [org] fake phishing emails and then saying to them, essentially, HEY! You could've just gotten scammed! You should be more careful – here's how...”

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

38

• Strategy #2: Tailor delivery of training for audience– Create “teachable moments”– Micro-games for training– Just sending training via email (ineffective) – Attending all day classes (boring, can’t test

skills)– Watching videos (can’t test skills)


©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

39

Strategy #3: Use Concepts from Learning Science

• Area of research examining learning, retention, and transfer of skills

• Example principles– Learning by doing– Immediate feedback– Conceptual-procedural– Reflection–… many others

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

40

What About Motivation?



Motivated to act?

• Training also needs to address motivation

• Open question as to best approaches for cybersecurity

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

41

What Motivates People?

• Extrinsic factors (outside factors)– Pay– Privilege, Reputation– Certificates, trophies– Punishment

• Can’t just slap it on, has to be appropriate and thought through

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

43


• Intrinsic value of task– Fun– Curiosity– Challenge, mastery

• Same as before, can’t just slap it on• Cybersecurity and intrinsic motivation

may be hard to reconcile• Intrinsic and extrinsic may conflict

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

44


• Social factors– Reciprocity (you help me, I help you)– Altruism– Norms– Social proof– Identification with group

• Large untapped potential, but open question as to how to best leverage

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

48

Summary

• Better user interfaces• Cybersecurity training

model– Better mental models– Tailor delivery– Learning science

• Lots of opportunitiesfor motivating people, but still open question

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

49

Thanks, where can I learn more?

Find more atwombatsecurity.com

[email protected]

©2

01

3 C

arn

eg

ie M

ello

n U

niv

ers

ity :

51

Timing Matters Too

• Teachable moments• Right after training• Repeat enough times, becomes habit

(don’t have to appeal directly to individual motivation anymore)

leveraging human factors for effective security training, for issa 2013 ciso forum, in pittsburgh...

Technology

security warnings

security issue

security problems model

cybersecurity awareness

mental models impact

smartphone security

end of training

mental models people