perceived risk in mobile authentication and interaction · • [sbw01] angela sasse, sacha...

© 2013 IBM Corporation

Perceived Risk in Mobile Authentication and Interaction

Shari TrewinRachel BellamyCal SwartLarry Koved

This work is supported by a grant from the Department of Homeland Security under contract FA8750-12-C-0265.

IBM T.J. Watson Research Center

© 2013 IBM Corporation2

Mobile Is Becoming A Primary Computing Platform

Physical

FinancialBusiness Personal


Current State of Mobile Authentication

Mobile Devices Are Authentication Tokens Mobile Wallets are Proliferating

•Starbucks – 26M transactions and growing

•Square - $4B/year CC transactions

•Visa’s payWave mobile payments system

•Barclaycard

•Schlage – Door locks

•Craftsman – Garage Door Opener

•Google Wallet, including Citi

•EnStream (Canada)

•Sprint

•Alcatel-Lucent

•Square

•O2 UK

•New Zealand

•Use of NFC as part of mobile wallet technology


Emerging Security Environment

Increased use of mobile smartphones to access enterprise data increases

potential for data loss

Mobile smartphones are increasingly used as authentication devices

Payments and micro-payments through mobile devices is an emergent

phenomenon Personal = Business

BYOD

• Enterprise enablement of mobile devices requires “strong passwords”

• Enterprise passwords are

• Hard to enter on mobile devices

• Disruptive to short term memory

• Strong dissatisfaction with enterprise authentication requirement for mobile


Authentication in Context

Interaction with mobile devices is brief– Often interrupt driven

PIN and gesture are most common

Passwords using reduced size keyboard– Entry of corporate compliant passwords

consumes a large fraction of theinteraction time

– User frustration • Removal of security profile• Avoid corporate compliance

Bio authentication – Typically face or voice– E.g., Android Ice Cream Sandwich

now has a face recognizer for device unlock– Few, if any, fingerprint readers on current

generation smartphones(Apple acquired AuthenTec – fingerprint)


Weak Protection of Credentials

•Smartphones unlockable in < 2 minutese.g., XRY, Fraunhofer.

Exploits known device vulnerabilities

•Browser and platform vulnerabilities

•Side channel attack using motion sensors(sp)iPhone (GaTech), etc.

•Credentials stored in the clear / decrypted

•Mobile devices are shared amongst friends and family


Smartphones: Multi-channel Input / Output Devices

NFC

Gyro

High res display

SMS/Text

Cell towersNetwork access

Soft keyboard

Voice

Multi-touch sensitive display

Accelerometer

Bluetooth

Cameras

GPS

Wi-fi/WiMax

Temperature Sensor

Pointing devices

Fingerprint

People are using Skype, Facetime, Siri, maps and other media-rich applications where biometric data is being acquired.


Why do we care about risk perception?

8

• Mobile transaction risks: eavesdropping, lost / stolen devices, man-in-the-middle

• Willingness to perform security actions is a tradeoff between cost to the individual and perceived benefit [SBW01]

• Mismatch of perceived risk and system determined risk leads to poor user acceptance of technology [BSW08]

• Experts and non-experts respond differently [KST82]

• Experts use statistical reasoning to assess risk; non-experts rely on affect [ECH08]

• Characteristics that influence perception of risk [L76]:voluntariness, immediacy of affect, knowledge about the risk, available alternatives, and consequences


• Mobile transaction risks: eavesdropping, lost / stolen devices, man-in-the-middle

• Willingness to perform security actions is a tradeoff between cost to the individual and perceived benefit [SBW01]

• [SBW01] Angela Sasse, Sacha Brostoff, and Dirk Weirich. Transforming the 'weakest link' a human /computer interaction approach to usable and effective security. BT Technology Journal, 19(3):122-131, 2001.

• Mismatch of perceived risk and system determined risk leads to poor user acceptance of technology [BSW08]

• [BSW08] Adam Beautement, M. Angela Sasse, and Mike Wonham. The compliance budget: Managing security behaviour in organisations. In NPSW'08, September 2008.

• Experts and non-experts respond differently [KST82]• [KST82] Daniel Kahneman, Paul Slovic, and Amos Tversky. Judgment under uncertainty: Heuristics and

biases. Cambridge University Press, 1982.

• Experts use statistical reasoning to assess risk; non-experts rely on affect [ECH08]• [ECH08] Serge Egelman, Lorrie Faith Cranor, and Jason I. Hong. You've been warned: an empirical study of

the effectiveness of web browser phishing warnings. In Proceedings of the 2008 Conference on Human Factors in Computing Systems, CHI 2008, 2008 Florence, Italy, April 5-10, 2008, pages 1065-1074. ACM, 2008.

• Characteristics that influence perception of risk [L76]:voluntariness, immediacy of affect, knowledge about the risk, available alternatives, and consequences

• [L76] William Lowrance. Of Acceptable Risk: Science and the Determination of Safety. William Kaufmann, 1976.

Why do we care about risk perception?


A Few of the Related IT Security Works

10

• Online risk assessment [Garg & Camp ‘12]• Viruses, phishing, ID theft. Familiarity of the risk and degree of dread.

• Willingness to perform sensitive activities on a phone vs. laptop [Chin, Felt, Sekar & Wagner ‘12]• Greater concern about smartphone as compared to laptop

• Smartphone app risks [Felt, Egelman & Wagner ‘12]• Greater concern about apps with greater perceived impact (e.g., financial vs. social)• Age was a significant factor


Related IT security work

11

• Online risk assessment [Garg & Camp ‘12]:• Viruses, phishing, ID theft. Familiarity of the risk and degree of dread.

• [GC12] Vaibhav Garg and Jean Camp. End user perception of online risk under uncertainty. InProceedings of HICCS 2012. IEEE. pp 3278-3287.

• Willingness to perform sensitive activities on a phone vs. laptop [CSFW12]• Greater concern about smartphone as compared to laptop

• [CFSW12] Chin, E., Felt, A., Sekar, V., and Wagner, D. (2012) Measuring user confidence in smartphone privacy and security. Proceedings of SOUPS 2012.

• Smartphone app risks [Felt, Egelman & Wagner ‘12]• Greater concern about apps with greater perceived impact (e.g., financial vs. social)• Age was a significant factor

• [FEW12] Felt, A., Egelman, S., and Wagner, D. (2012) I’ve got 99 problems, but vibration ain’t one: A survey of smartphone users’ concerns. Proceedings of Workshop on Security and Privacy in Smartphones and Mobile Devices (SPSM 2012), ACM Press


How Does Context Influence Risk Perception in Mobile Tasks?

12

User groups studied:

• IT workers

• Individuals (Amazon Mechanical Turk)

• Doctors (not fully reported here)

• IT Security Experts


Method – Mobile tasks in 6 locations

At home by yourself

In a crowded local street

On a quiet train at night with no-one nearby

In your office at your desk

In a very busy café in an unfamiliar neighborhood

In a Beijing hotel room

13


Mobile tasks

14

For Individuals: using your mobile device,• Look up your account balance using the bank’s app• Look up your account balance on the bank’s web site• Make a $100 emergency purchase using the web site of an

unfamiliar retailer entering your credit card information

IT Workers & Security Experts, using your mobile device,• Use company app to look up information about an unannounced

acquisition• Look up your personal retirement benefit information on a trusted

3rd party web site• Make a $100 emergency purchase using the web site of an

unfamiliar retailer entering your company’s credit card information


18 Combinations of Task and Location

15

Would [your / the] information be safe if you did that in these places?


Method – Open Questions

“What else would you want to know about the situations described in this study to decide whether it is safe to access or enter sensitive information on your smartphone there?”

• Responses to this question reveal factors that the individual would consider when evaluating risk, such as the type of network connection or presence of other people.

“What, if any, are the security risks you see in these situations?”

• Responses here indicate the specific threats that the individuals are aware of, such as device theft or network eavesdropping.

“What factors affect your decision whether to access sensitive information in a given situation?”

• This question goes beyond risk perception to reveal other factors that people will take into account when deciding whether to accept the perceived risk, such as the urgency of the need to access the information, and ability to go to a safer place.

16


Method – Open Questions Analysis

• An initial set of codes was derived for each question by starting from the responses given by the IT security experts, and adding or subdividing codes as necessary to cover themes emerging from the remainder of the data.

• The three open questions were analyzed by post-coding all responses from IT Workers and Individuals.

• Two independent coders

• After the independent coding, for each question, an inter-rater reliability analysis using the Kappa statistic was performed to determine consistency between the coders. After two iterations of coding, the Kappa values achieved were:

• ‘what else’ question, Kappa = 0.875 (p <.0.001); • ‘security risks’ question, Kappa = 0.917 (p <.0.001); • ‘what factors’ question, Kappa = 0.879 (p <.0.001).

• Items coded inconsistently were discarded from further analysis. • For the ‘what else’ question we threw out 20% (51 out of 258 comments). • For the ‘security risks’ question we threw out 8% (24 out of 297 separate

comments) of the data. • For the ‘what factors’ question we threw out 11% (30 of 261 separate comments).

17


Method and Participants - Individuals

18

• Methods

• Amazon Mechanical Turk hosted a questionnaire

• Limited to Turkers who had at least 1000 completed “tasks” with at least 95% tasks accepted as quality work

• US-based workers (for legal reasons)

• Three test questions to ensure Turkers had read and understood the scenario

• Limited to participants who owned a smartphone > 6 months

• Participants

• 54 of 76 respondents qualified (owned a smartphone > 6 months)• 38 male, 16 female• Device ownership: 34 Android, 20 iPhone, 2 Blackberry, 1 other• Device unlock: 12 4-digit PIN, 5 gesture, 22 swipe, 10 no lock, 5 PIN (unspecified length)


Method and Participants - IT Workers

19

• Method• Questionnaire distributed on paper and web form at a technology company

• Distributed shortly after annual certification of business conduct guidelines• Protection of company data was fresh in their minds

• Limited to participants who owned a smartphone > 6 months

• Participants• 46 male, 7 female. Between 23 and 67 years old, mean=44.7, (Std dev. = 12.4)

• 7 participants did not divulge their age• Self reported security expertise:

1=minimal, 26=average, 24=knowledgeable, 2=expert


Method and Participants - Security Experts

20

• Method• Group discussion using the same materials as the IT workers

• Participants – IT Security Experts• 11 security researchers, all male, 10 with over 10 years of IT security research

experience, one with 5 years of experience


Method and Participants – Doctors*

21

• Method• Paper questionnaire

• Included risk perception questions

• Participants• 11 hospital-affiliated doctors (10 male)

• A range of different specializations• All were smartphone users

• Identified needs and current practices

Doctors are not included in the following results(Included here for later comparison)

*Not funded by the Department of Homeland Security


Contrasting the user groups

22

Self-reported security expertise for Individuals and IT Workers

Mobile devices owned by Individual and IT Worker participants


Risk Assessments – Information Safety in Different Locations

23

• Different access tasks, 6 locations• Asked to assess the safety of performing the task in that situation.

• There was a significant effect:• Task (Kruskal-Wallis test, Chi-Square=94.918, p<0.001) • Location (Kruskal-Wallis test, Chi-Square=639.032, p<0.001)

Summary of responses indicating safety in different situations. X-axis indicates percentage of participant responses for each location.


Risk Assessments – Information Safety by Task when Home Alone

24

• Significant effect on assessments of safety • Kruskal-Wallis test, Chi-square = 67.995, p<0.001.

• Pairwise comparisons with Bonferroni correction to adjust for multiple tests indicate that the ‘unfamiliar retailer’ is significantly different from all other tasks

• Mann-Whitney test, p<0.001• No other differences are significant (p>0.4 in all cases)

Summary of responses indicating safety of different transaction types performed from home


Risk Assessments - Gender

25

• Significant overall effect of gender on responses (Mann-Whitney test, U=275776, p=0.014): • The central tendency was ''unsure/depends” for both groups• The average response for women was below this value

• 3.97, where the ‘unsure/depends’ value is numbered 4, and higher values indicate greater safety

• Men provide higher assessments of safety (4.18). • Consistent with Felt et al’s [FEW12] findings on ratings of concerns over smartphone

application privacy-related actions.

• Individuals• The gender difference was small and not statistically significant

(women: 4.06, men: 4.21, p=0.125).

• IT Workers • The difference was statistically significant (women: 3.79, men: 4.16, p=0.010).

• These findings suggest there may be gender differences in risk perception for mobile device transactions, and are worthy of further exploration with a larger sample.

• Too few women to be conclusive.


Perceived Security Risks: “What, if any, are the security risks you see in these situations?”

26

• IT Security Experts identified the following security risks in the scenarios.Risks are presented in the order they were provided:

• Shoulder surfing – direct observation of either sensitive information or passwords, potentially using a camera from a distance

• Man in the middle attack – where communications are routed through an attacker

• Network snooping – leaking information from Bluetooth, WiFi or NFC networks

• Automatic backup of sensitive data to a cloud owned by an external organization

• Data left on the device – vulnerable if the device is compromised, stolen, or used by another person.

• Loss or theft of the device


Percentages of respondents from each group mentioning a specific class of risk

27

Overall % % Individuals accessing

personal info.

% IT Workers accessing company

info.

Identified as a risk by Security

ExpertsObservationDirect observation 33 26 50 YesRemote observation 7 2 17 YesOther observation risks 1 0 2NetworkMan-in-the-middle attack 8 8 12Network snooping 23 25 29 YesInsecure data transmission 14 11 21 YesUnsafe WiFi 7 8 7Unsafe cellular 2 2 2Trust 6 4 10Other network risks 2 4 0DeviceLoss of device 5 0 12 YesTheft of device 11 2 26 YesMalware on device 8 6 14 YesDevice access through hacking 7 8 7 YesStorage of sensitive data on device 3 0 7 Yes

Physical device access 3 0 7 YesRemote ServiceTrust in the remote service 12 15 12Theft of information 7 9 7Loss of information 4 4 5Cloud backup 0 0 0 YesOther remote service risks 1 0 2SituationImmediate personal/social/physical risks 7 6 10

Loss of informationData being leaked 8 9 10Access to accounts 12 15 12Other information loss risks 4 2 7

Risks reported by more than 15% of people: (1) someone might see the user’s screen or interaction (33%), and (2) data might be intercepted over the network (23%).


Perceived Security Risks

28

• Both IT Workers and Individuals faced with the prospect of entering credit card information at an unknown retailer’s web site had many concerns relating to the retailer, their honesty, and their security.

• IT Workers (and IT Security Experts) saw many threats related to • Their device being infected with malware or hacked, or • The risk of storing sensitive information on their devices.

• Only 1 of the Individuals mentioned the risk of losing their device or having it stolen• At least 10% of IT Workers expressed this concern.

• IT Workers more often mentioned device theft (26%).

• IT Security Experts cited cloud-based backup services as a risk, • The scenarios presented did not bring this to mind for any of our respondents.

• All groups perceived network eavesdropping as a risk:• Several specific forms of network risk were mentioned by IT Workers and Individuals, most

often ‘snooping’, ‘insecure transmission’, and ‘man-in-the-middle’ attacks. • Often, these groups would identify a network risk as being present “if it’s not a secure

channel”.


Important Information When Assessing Risk:“What else would you want to know about these situations, to decide whether it is safe to access or enter sensitive information on your smartphone there?”

29

Participants already knew the type of location, the task they would be performing, and in some cases whether other people were present.


personal info.

% IT Workers accessing

company info.

Security Experts wanted to know

ObservationPossibility of being observed 3 0 6Possibility of interaction being observed 7 2 11 YesNetworkWho owns the network 6 4 8 YesWho else has network access 3 2 4Risk of data being intercepted 10 9 11Security of the connection 13 17 9Encryption of network traffic 29 19 40 YesConnection method 15 17 13Other network information 1 0 2DeviceRisk of device loss 1 0 2Risk of device theft 2 0 4Prior/future access by others 2 0 4Device security 1 2 0Risk of malware on device 2 0 4 YesApplication security 3 2 4 YesLocal data storage 2 2 2 YesRemote ServiceSecurity certificates used 6 4 8Remote service security 5 0 9Remote service honesty 23 33 13Other remote service information 1 2 0SituationOther information about the situation 8 13 4Other people's experiences 3 4 2Legal recourse or protection 2 4 0TaskTime the task will take 1 0 2Sensitivity of information being accessed 3 0 6 YesAttention demands of the task 1 0 2Other task information 2 4 0 Yes (urgency)Percentages of respondents from each group who wanted to know each kind of additional information, in order to decide whether to perform a mobile transaction


Perceived Risks –Risks perceived by Individuals accessing and using personal financial information

3030


Perceived Risk - Risks perceived by IT Workers accessing company information

31


Perceived Risk: Risks Perceived by Doctors Accessing Medical Information

32


Factors Influencing Mobile Access Decisions –“What factors affect your decision whether to access sensitive information in a given situation?”

33

Experts considered the following three factors:

1.The consequences of the data being compromised

2.The urgency of the need to access the data

3.Whether they can protect the informatione.g., by hiding the screen from observers or cameras


Factors Influencing Mobile Access Decisions

34


personal info.

% IT Workers accessing

company info.

Security Experts

considered

ObservationRisk of being observed 31 28 34Ability to protect against observation 2 2 2 YesNetworkRisk posed by the network 37 39 36Network security protections in place 11 6 17 YesDeviceRisk of device loss/theft 7 2 11Risk of device being accessed 14 17 11Remote ServiceProbability of third party losing or stealing data 10 15 6SituationFeelings about the location/situation 20 26 13Ability to go to a safer location 1 0 2Other probability of data loss 2 2 2Magnitude of risk 5 2 8Liability for lost information 4 4 4 YesTaskTime the task will take 1 0 2Sensitivity of information being accessed 11 7 15 YesImportance of accessing the data 6 6 6Time constraints 11 7 15 YesNeed for the data 5 6 4 Yes

Percentages of respondents from each group who considered each factor when deciding whether to perform a mobile transaction.


Use of Phone

35

• 46% of the participants did not lock their phones

• 33% used a PIN code lock• 11% used a gesture lock• 7% used a password• 4% used some other method (e.g., face reco.)

Phone locking for the Individuals:• 53% of women • 37% of men

• Phone locking for the IT Workers:• 57% of women • 70% of men

We did not ask security experts whether they locked their phones, and how.


Choice of Authentication Level

36

After the first set of scenarios was presented, participants were asked what kind of sign-in (authentication) they thought should be needed.

The specific questions asked depended on the specific scenario (Individuals, IT Workers)

Individuals: “In general, what kind of sign in do you think the bank's app should require for access to your financial information?”

IT Workers: “In general, what kind of sign in should the <<company>> application require for access to confidential <<company>> information from a non-<<company>> location? Assume that <<company>> does not also require a password on the whole device.”

The response choices were:

• None• Minimal (e.g. 4-digit PIN)• Regular (e.g. 8 character mixed alpha-numeric password) • High security (e.g. password AND face recognition)• It should depend on the kind of data I am accessing


Choice of Authentication Level

37

Authentication level considered appropriate for accessing sensitive information on a mobile device.


Tentative Conclusions

38

• Peoples’ risk perception appears to be influenced by the presence of other people in close proximity.

• Untrusted locations increase peoples’ perception of risk (crowded streets, Beijing vs. quiet train, home alone)

• Reducing authentication demands when people are in locations perceived to be safe may align well with people's perceptions of authentication needs.

• People typically trust both app and web sites, in general, if they are familiar with the service provider.

• People are concerned about network security.• User concern over network security could be addressed by providing users with

information about network encryption of the authentication credentials being provided, and the application as a whole.

• Peoples’ perception of risk does not extend to locking their mobile devices.• Although note the difference in behavior between IT Worker vs. Individual men.


Discussion and Design Implications

39

• Design implication: Seek opportunities to make users aware of the risk of device theft.

• Probable design implication: Provide education or intervention that increases users’ realization of the benefits of locking their phone.

• Design implication: Users need to have multiple different authentication options for any transaction, to accommodate variation.

• This also addresses situational impairments, for example, a noisy train station where voice recognition is likely not to be successful.

• Design Implication: When our design reduces authentication demands when people are in familiar locations, this may align well with people's perceptions of authentication needs.

• Design implication: Use risk communication that is not location-specific

• Design implication: If a design places authentication demands on people when in locations they visit infrequently, but they perceive to be safe (e.g. quiet train) our user interface needs to help people understand the risks involved as there is mis-alignment of risk perceptions.

• Design implication: Risk communication (either via educational materials or in the UI) could highlight the observability of the available authentication methods, or ways of protecting against observation risk.

• Design implication: Risk communication to address user concern over network security could provide users with information about network encryption of the authentication credentials being provided, and the application as a whole.

• Design implication: Risk communication should provide some indication of the need for additional security, but do so without revealing the details of how risk is calculated to minimize creating security cues exploitable by attackers.

• Design implication: Authentication methods should leverage opportunities to maintain authentication status while the phone is in use, reducing the frequency of authentication challenges, and the effort involved.

perceived risk in mobile authentication and interaction · • [sbw01] angela sasse, sacha...

Documents