user studies motivation
DESCRIPTION
User Studies Motivation. January 30, 2007. How do we know whether security is usable?. Need to observe users. We are not our users! (you may be surprised by what users really do). Wireless privacy study. Many users unaware that communications over wireless computer networks are not private - PowerPoint PPT PresentationTRANSCRIPT
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/1
User Studies MotivationUser Studies Motivation
January 30, 2007
How do we know whether How do we know whether security is usable?security is usable?
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/3
Need to observe usersNeed to observe usersWe are not our users!
(you may be surprised by what users really do)
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/4
Wireless privacy studyWireless privacy study Many users unaware that communications over
wireless computer networks are not private How can we raise awareness?
B. Kowitz and L. Cranor. Peripheral Privacy Notifications for Wireless Networks. In Proceedings of the 2005 Workshop on Privacy in the Electronic Society, 7 November 2005, Alexandria, VA.
Wall of sheepWall of sheep
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Photo credit: Kyoorius @ techfreakz.org http://www.techfreakz.org/defcon10/?slide=38
Defcon 2001
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Photo credit: http://www.timekiller.org/gallery/DefconXII/photo0003
Defcon 2004
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/8
Peripheral displayPeripheral displayHelp users form more accurate
expectations of privacy
Without making the problem worse
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/10
Experimental trialExperimental trialEleven subjects in student workspace
Data collected by survey and traffic analysis
Did they refine their expectations of privacy?
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/11
ResultsResultsNo change in behavior
Peripheral display raised privacy awareness in student workspace
But they didn’t really get it
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/12
Privacy awareness increasedPrivacy awareness increased “I feel like my information /activity / privacy
are not being protected …. seems like someone can monitor or get my information from my computer, or even publish them.”
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/13
But only while the display But only while the display was onwas on
“Now that words [projected on the wall] are gone, I'll go back to the same.”
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/14
Security and privacy Security and privacy indicatorsindicators
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/15
Evaluating indicatorsEvaluating indicatorsCase study: Privacy Bird
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/16
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Platform for Privacy Preferences Platform for Privacy Preferences (P3P)(P3P)
2002 W3C Recommendation
XML format for Web privacy policies
Protocol enables clients to locate and fetch policies from servers
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/17
Privacy BirdPrivacy Bird P3P user agent
Free download http://privacybird.org/
Compares user preferences with P3P policies
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/20
Critique Privacy BirdCritique Privacy Bird Security people
• Can attackers spoof it?• What if P3P policy
contains lies?• Can P3P policies be
digitally signed?• What about main-in-
the-middle attacks?
Usability people• Green/red color blind
problem• Do people notice it in
corner of browser?• Do people understand
privacy implications?• Why a bird?
Typical Typical securitysecurity
evaluationevaluation
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/22
Does it behave correctly when Does it behave correctly when notnot under attack? under attack?
No false positives or false negatives
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/23
Anti-phishing toolsAnti-phishing tools
Y. Zhange, S. Egelman, L. Cranor, and J. Hong. Phinding Phish: Evaluating Anti-Phishing Tools. In Proceedings of NSSS 2006, forthcoming.
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/24
Does it behave correctly when Does it behave correctly when under attack?under attack?
Can attackers cause wrong indicator to appear?
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/25
Correct indicator
Wrong indicatorAttacker redirects through CDN
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/26
Can it be spoofed or Can it be spoofed or obscured?obscured?
Can attacker provide indicator users will rely on instead of real indicator?
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/27
Usability evaluationUsability evaluation
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/29
C-HIP ModelC-HIP Model Communication-
Human Information Processing (C-HIP) Model• Wogalter, M. 2006.
Communication-Human Information Processing (C-HIP) Model. In Wogalter, M., ed., Handbook of Warnings. Lawrence Erlbaum Associates, Mahwah, NJ, 51-61.
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/30
Do users notice it?Do users notice it?If users don’t notice indicator all bets are
off
“What lock icon?”• Few users notice lock icon in browser chrome,
https, etc.
C-HIP model: Attention switch, attention maintenance
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/31
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/32
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/33
Do users know what it means?Do users know what it means?Web browser lock icon:
“I think that it means secured, it symbolizes some kind of security, somehow.”
Web browser security pop-up:“Yeah, like the certificate has expired. I don’t actually know what that means.”
C-HIP Model: Comprehension/Memory
J. Downs, M. Holbrook, and L. Cranor. Decision Strategies and Susceptibility to Phishing. In Proceedings of the 2006 Symposium On Usable Privacy and Security, 12-14 July 2006, Pittsburgh, PA.
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/34
Netscape SSL icons
Cookie flag
IE6 cookie flagFirefox SSL icon
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/35
Privacy Bird iconsPrivacy Bird icons
Privacy policymatches user’s
privacy preferences
Privacy policydoes not match user’s privacy
preferences
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/36
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/37
Do users know what to do when Do users know what to do when they see it?they see it?
C-HIP Model: Comprehension/Memory
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/38
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/39
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/40
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/41
Do users believe the Do users believe the indicator?indicator?
“Oh yeah, I have [seen warnings], but funny thing is I get them when I visit my [school] websites, so I get told that this may not be secure or something, but it’s my school website so I feel pretty good about it.”
C-HIP Model: Attitudes/Beliefs
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/42
Are users motivated to take Are users motivated to take action?action?
May view risk as minimal
May find recommended action too inconvenient or difficult
C-HIP Model: Motivation
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/43
Do they actually do it?Do they actually do it?“I would probably experience some brief, vague sense of unease and close the box and go about my business.”
C-HIP Model: Behavior
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/44
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/45
Do they keep doing it?Do they keep doing it?Difficult to measure in laboratory setting
Need to collect data on users in natural environment over extended period of time
C-HIP Model: Behavior
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/46
How does it interact with How does it interact with other indicators?other indicators?
Indicator overload?
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/47
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/48
Summary: Security evaluationSummary: Security evaluationDoes indicator behave correctly when not
under attack?•No false positives or false negatives
Does indicator behave correctly when under attack?•Can attackers cause wrong indicator to
appear?
Can indicator be spoofed or obscured?•Can attacker provide indicator users will rely
on instead of real indicator?
Usable Privacy and Security • Carnegie Mellon University • Spring 2007 • Cranor/Hong• http://cups.cs.cmu.edu/courses/ups-sp07/49
Summary: Usability evaluationSummary: Usability evaluation Do users notice it? Do they know what it
means? Do they know what they
are supposed to do when they see it?
Do they believe it? Are they motivated to do
it? Will they actually do it? Will they keep doing it? How does it interact with
other indicators?