an online shopping search engine user study
TRANSCRIPT
![Page 1: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/1.jpg)
CMU Usable Privacy and SecurityLaboratory
http://cups.cs.cmu.edu/
An OnlineAn Online Shopping SearchShopping SearchEngine User StudyEngine User Study
Janice Y. TsaiJanice Y. Tsai
February 27, 2007February 27, 2007
![Page 2: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/2.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 2
Research MotivationResearch Motivation Privacy ConcernsPrivacy Concerns• Studies consistently show that people have
privacy concerns.
Privacy Policies Privacy-ProtectingPrivacy Policies Privacy-ProtectingBehaviorBehavior• Difficult to read and understand.• Dichotomy between stated privacy
preferences and observed behavior.
!
"
![Page 3: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/3.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 3
Research QuestionResearch Question
Will the prominent display of privacyinformation cause consumers to take
privacy into account when making onlinepurchasing decisions?
![Page 4: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/4.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 4
Understandable Privacy PoliciesUnderstandable Privacy Policies• Platform for Privacy Preferences (P3P)
Communicate privacy policies in a standardmachine-readable format.
• Enables development of toolsSummarize privacy policies.Compare policies with user preference.Alert and advise users.
![Page 5: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/5.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 5
Privacy FinderPrivacy Finder• Privacy-enhanced P3P search engine
![Page 6: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/6.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 6
![Page 7: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/7.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 7
![Page 8: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/8.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 8
Research ObjectivesResearch Objectives Privacy Concerns• Determine the level of concern in various
online scenarios.
• Determine if Privacy Finder addressesrelevant concerns.
Privacy-Related Behavior• Determine if users will select websites with
P3P policies and pay a premium for privacy.
• Investigate the effectiveness of Privacy Finderas a risk communication tool.
![Page 9: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/9.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 9
SurveysSurveys Survey DesignSurvey Design• Decide what you want out of the survey.• Decide what questions to ask.
Conduct a literature review for examples.• Write out some analysis procedures.• Select a format for your survey.
PaperEmailWebpage
• Submit an IRB application.• Pilot the study with friends.• Edit/Update the survey.
![Page 10: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/10.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 10
Online Concerns SurveyOnline Concerns Survey• Logistics
Designed survey on paper and ported toSurveyMonkey.com
Raffled off an iPod Nano• Methodology
Duration: 1 week Participants solicited via Craigslist n = 276
• Evaluated Perceived Likelihood of PrivacyViolations
• Developed a Risk Score• Selected Products for Purchasing
![Page 11: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/11.jpg)
Addressed byPrivacy Finder
11
![Page 12: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/12.jpg)
Would NotPurchase
Purchase,No Concerns
12
Purchase,Very Concerned
Purchase,Somewhat Concerned
![Page 13: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/13.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 13
User StudiesUser Studies Experimental DesignExperimental Design• What are you testing?• How can you design your study to get the biggest effect?
Is there a target user population? Will you be able to generalize your results?
• What kind of user study? Online One-on-one Group Interview
• How many sessions? One session? Many sessions?
• IRB Application
![Page 14: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/14.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 14
LogisticsLogistics• Recruit Users
Misc. Market (CMU)Pittsburgh CraigslistCMU Center for Behavioral and Decision Research Flyers around town
• Select UsersScreening survey
• Schedule UsersCreate several sessions Takes longer than you thinkNo shows
![Page 15: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/15.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 15
• Running the User StudyReserve spacePrepare materialsCreate a scriptPrepare computer(s)
• CompensationCashEntry into a raffleGift certificate
![Page 16: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/16.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 16
Online Shopping User StudyOnline Shopping User Study Experimental DesignExperimental Design• Framing: Usability Test of a New Search Engine• Price Incentive for Purchases
Provide lump sum payment Participants keep “remainder” after purchase
• Screening Survey
![Page 17: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/17.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 17
The User StudyThe User Study Three ConditionsThree Conditions• Condition 1: No Information• Condition 2: Irrelevant Information• Condition 3: Privacy Information
Two ProductsTwo Products• Privacy Sensitive• Non-Privacy Sensitive
16 Participants in each Condition16 Participants in each Condition
![Page 18: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/18.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 18
eCommerce website selection factorseCommerce website selection factorsFactors Mean p Value
Price 5.61 <.0001
Return Policy 4.72 0.009
Shipping Speed 4.46 0.37
Customer Service 4.44 0.45
Privacy Policy 4.27 -
Website Design 4.11 0.53
Customer Reviews 3.9 0.18
Software Compatibility 3.69 0.02
Webpage Load Speed 3.63 0.009
Popularity 3.55 0.0058
Physical Location 2.48 <.0001
Cell Phone Compatibility 0.46 <.0001
Handicapped Accessibility 0.3 <.0001
![Page 19: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/19.jpg)
Condition 1: No InformationCondition 1: No Information
19
![Page 20: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/20.jpg)
Condition 2: Handicap InformationCondition 2: Handicap Information
20
![Page 21: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/21.jpg)
Condition 3: Privacy InformationCondition 3: Privacy Information
21
Privacy Premium $0.69 4.8%
![Page 22: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/22.jpg)
H5: Product DifferencesH5: Product Differences
Batteries Sex Toy
H1: Privacy Influences PurchasesH1: Privacy Influences PurchasesH2: High PrivacyH2: High PrivacyH3: Lowest PriceH3: Lowest PriceH4: Icon ImpactsH4: Icon ImpactsResultsResults
22
![Page 23: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/23.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 23
Price PremiumPrice Premium Impact• Price premiumbatteries< Price premiumsex toy
• People may have been more willing to pay forhigh privacy for batteries for $.50 than $1.50for the sex toy.
![Page 24: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/24.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 24
Privacy PremiumPrivacy Premium
Condition 1:
No Information
Condition 2:
Handicap
Information
Premium p Value
Average
Price
Batteries
$14.64 $14.69 $0.05 0.64
Average
Price
Vibrator
$15.26 $15.30 $0.04 0.65
No significant price differences
![Page 25: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/25.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 25
Privacy Premium cont.Privacy Premium cont.
Condition 1:
Privacy
Information
Condition 3:
Privacy
Information
Premium p Value
Average
Price
Batteries
$14.64 $15.23 $0.59 0.0007
Average
Price
Sex Toy
$15.26 $15.88 $0.62 0.00005
Significant price differences
![Page 26: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/26.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 26
Risk CommunicationRisk Communication• Notice
92% noticed the Privacy Icons
• Read 32% of participants read the Privacy Report
• Shop For 60%, privacy information influenced the sites
visited and the sites from which purchases weremade.
![Page 27: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/27.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 27
“Which factor had the most influence on your decision?”• Open Ended Response of “Privacy Policy”• Batteries
32% (Privacy Condition) 6% (“Handicap Information” Condition) 0% (“No Information” Condition) Privacy information heightened awareness!
• Sex Toy 36% (Privacy Condition) 11.8% (“Handicap Information” Condition) 14.8% (“No Information” Condition) Privacy concern already existed with privacy-sensitive item.
![Page 28: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/28.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 28
Lessons LearnedLessons Learned Have a Backup PlanHave a Backup Plan• Users do crazy things....
Try to use Google to search for items Try to search for coupon codesGo directly to their favorite online stores
• Out of Stock!Various websites ran out of items during the studyNew results selected and presented Toss out data and find new participants
![Page 29: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/29.jpg)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/ • Janice Y. Tsai 29
ConclusionConclusion Accessible privacy information affectsAccessible privacy information affects
consumer behaviorconsumer behavior
Consumers willing to pay for betterConsumers willing to pay for betterprivacyprivacy
Privacy FinderPrivacy Finder helps helps users make privacyusers make privacyinformed decisionsinformed decisions
![Page 30: An Online Shopping Search Engine User Study](https://reader031.vdocuments.us/reader031/viewer/2022012514/618dcfe22e878a008e551b69/html5/thumbnails/30.jpg)
CMU Usable Privacy and SecurityLaboratory
http://cups.cs.cmu.edu/