User Perception of Facebook App Data Access: AComparison of Methods and Privacy Concerns

Jennifer GolbeckUniversity of Maryland, HCIL

College Park, MD 20742jgolbeck@umd.edu

Matthew Louis MaurielloUniversity of Maryland, HCIL

College Park, MD 20742mattm@cs.umd.edu

ABSTRACTDespite privacy concerns, social media users continue toshare vast amounts of personal information online, and to useservices that can access this data. But are users fully aware ofwhat information they are sharing when they install an app,or are they making these decisions from a weak position? Inthis paper, we focused on Facebook apps and set out to un-derstand how well informed users are about the informationthey are sharing and how concerned they are about privacy.

We recruited 120 subjects for an experiment. Subjects com-pleted a survey about their beliefs and concerns regardingthe information Facebook apps could access in their pro-files. They were shown additional information from differ-ent sources that explained what data apps could access, andthen asked to re-take the survey. We found that after viewingthe information about app data access, overall concern aboutprivacy on Facbeook increased, as did concern about iden-tity theft, and unauthorized people gaining access to subjects’data. Furthermore, after viewing the data, subjects had a bet-ter understanding of what data apps were able to access fromtheir profile. At the same time, even after viewing explicitinformation on the topic, many subjects still did not fully un-derstand what data apps could access.

We present the results of our study, address how these resultscan inform future work on educating users about privacyrisks and policies, discuss and the implications this has forcybersecurity, social media, and HCI.

Author Keywordsprivacy, Facebook, social networks, apps, cybersecurity

ACM Classification KeywordsH.5.m. Information Interfaces and Presentation (e.g. HCI):Miscellaneous

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than theHCIL must be honored. Abstracting with credit is permitted. To copy otherwise,or republish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee. Request permissions from jgolbeck@umd.edu.

Copyright c© HCIL Tech Report (2014), College Park, Maryland, USA.www.cs.umd.edu/hcil/pubs/tech-reports.shtml

Figure 1. A screen capture of the creepy Facebook Stalker looking at aperson’s profile, taken as a screen capture from the Take This Lollipopvideo, one of the apps used in our study.

INTRODUCTIONSocial media users are concerned about privacy. In the studywe present here, only 4 out of 120 subjects reported they were“Not at all” concerned about privacy on Facebook. But de-spite a wide range of media reports about the consequencesof oversharing and stories about how data can be used in un-intended ways, people continue to interact, share data, anduse apps online despite their concerns.

Are users continuing to share because they are aware of theprivacy risks and have made an informed choice about whatthey are comfortable sharing, or are they operating under falseassumptions or without the knowledge they need to make aninformed choice?

In this study, we focus specifically on the personal informa-tion the Facebook apps can access on a user’s profile. Theseapps serve as an example and case study for broader questionsabout user understanding of privacy in social media.

We set out to answer three research questions:

RQ1: How informed are users regarding the privacy risks ofusing Facebook apps?RQ2: Will viewing additional information about what Face-book apps can access affect the way users feel about the re-lated privacy risks?RQ3: Are some methods of sharing this information more ef-fective at educating users about the information that apps canaccess?

1

To answer these questions, we conducted a between-subjectsexperiment with 120 subjects. Subjects completed a surveyindicating their concern about Facebook privacy and their be-liefs about what data Facebook apps could access. Then, theywere divided into conditions where they viewed either theFacebook privacy policy, the Take This Lollipop interactivehorror film (Figure 1), our own app designed to show per-sonal data, or a combination of all three. Then, subjects re-took the survey so we could judge if their feelings and beliefshad changed.

We found substantial increases in privacy concerns and in thenumber of personal data attributes that our subjects believedapps could access. However, we also found that many userswere still unclear about what data Facebook apps could ac-cess.

In this paper, we discuss related work on this topic. Then,we lay out our experimental design with the foundations forthe survey questions we used. We present an overview of theprivacy information sources that subjects viewed, and discussthe results of our study. We conclude with a discussion ofimplications for the design of privacy policies and tools forcommunicating privacy issues to users. In addition, we lookat the broader issue of cybersecurity as it relates to HCI andthe challenges that come with helping users understand theimplications of their data sharing choices.

RELATED WORKSince social media started emerging as a major paradigm ofinternet use in the early-to-mid 2000s, there have been stud-ies of users’ concerns and privacy preferences. Studies ofhow people choose to share information in social media, theirprivacy concerns, and their awareness about these issues hasbeen covered in [1, 4, 14] to name a few. These studies tookplace in 2006 and 2007, and the landscape has changed a lotsince then, particularly in terms of the privacy controls avail-able to users and in how publicly information is shared bydefault.

As privacy options became more sophisticated, researchfound that users were underutilizing these controls [13].Tools to help users manage their privacy settings were evenproposed [6].

More recent work has looked specifically at users’ privacyconcerns and utilization of privacy settings in these environ-ments. In 2009, Debatin et al. [3] studied users’ privacy con-cerns in Facebook. Their subjects claimed to understand pri-vacy issues, but generally believed others were more at riskfor privacy invasion than they themselves were. The authorsattributed users’ “lax attitude” to a number of factors, includ-ing psychological effects, the way people used the sites, andthe fact that there was high gratification from use which ledto downplaying the risks. Litt [11] found differences in uti-lization of privacy features of social networking sites basedon factors like age, gender, and experience.

In [10], the authors built a model of social network usageconsidering users’ privacy concerns and trust. They showedthat social network use was negatively correlated with privacyconcern, but that his could be mitigated by building trust with

users. These results echo earlier work from 2009 that showedusers’ trust in one another and in an online communities’ in-formation sharing norms mitigated privacy concerns [12].

[5] discusses a privacy feedback tool designed to help usersbetter understand their identity exposure online. While wedo not have users interact with this tool specifically, we dohave them look at information sources that show users whatdata Facebook apps can collect, thereby revealing informa-tion about how exposed a user’s data is to these apps.

Most closely related to our work is a 2011 study that inves-tigated how users interacted with and understood Facebookapps’ data access [8]. The study found that many users mis-understood or were confused by the policies that governedapps’ access to their data. They also found that subjects’ be-havior or knowledge were not necessarily predictive of theirlevel of privacy concern. Instead, privacy attitudes tended tochange if the user personally experienced an adverse privacyevent on Facebook. This is similar to the results from [3] dis-cussed above; they found that personal, negative events had abig impact on users, but knowledge of bad events happeningto others generally did not have much impact.

This could be because there is not a good understandingamong users about what information apps access from auser’s profile, and only adverse events really make the pointto users. Facebook reveals some information about what dataapps will have access to when users install it, and users are leftto rely on this and on their own understanding of Facebook’sprivacy policies in this context. Generally, this informationseems to be insufficient for users to fully understand the in-formation they are sharing with an app. This motivated thework in [15], which studied a large number of apps to under-stand the type of access they requested to users’ data. Thistype of profiling could be important for understanding if appsare over-requesting data access and thus whether users needto be warned about agreeing to more expansive permissions.

EXPERIMENTAL DESIGNIn this experiment, we wanted to understand how informedusers are about Facebook app data access privileges, and tostudy the impact that viewing information about these privi-leges would have on users’ privacy concerns and perceptions.

To achieve this, we administered a pre-test to gauge subjects’privacy concerns and perceptions. The answers from this ini-tial survey are used to help us answer RQ1 described above.Then, subjects viewed information about how Facebook appsaccess personal data in one of four different conditions. Af-ter that, we re-administered most of the questions from thepre-test to see if and how user perception and concern hadchanged.

Our exact process was as follows:

1. Obtain informed consent

2. Collect basic demographic information

3. Administer pre-test survey questions

4. Privacy Information Conditions - subjects were randomlyassigned to one of four conditions (described below) that

2

showed them information about what data Facebook appscan access.

5. Re-administer surveys as post-test questions to see how op-tions may have changed after seeing the privacy informa-tion.

Pre- and Post-Test QuestionnairesOur pre-test was broken into two main sections: demographicand background questions, and Facebook-specific questionsabout privacy concern and the perception of what is shared.This latter group of questions was repeated in the post-test.

Demographic and Background QuestionsWe collected basic demographic information about our sub-jects: age, gender, the country where they grew up, and ed-ucation level. We used this to determine if any demographicfeatures impacted subjects’ privacy attitudes.

Next, we wanted to gain a basic understanding of subjects’privacy attitudes. We included the Westin/Harris Privacy Seg-mentation Model [7, 9], which uses three questions to groupusers into three categories: privacy fundamentalists, privacypragmatists, and privacy unconcerned.

This survey has three statements, to which users re-spond “strongly disagree”, “somewhat disagree”, “somewhatagree”, or “strongly agree”:

1. Consumers have lost all control over how personal infor-mation is collected and used by companies.

2. Most businesses handle the personal information they col-lect about consumers in a proper and confidential way.

3. Existing laws and organizational practices provide a rea-sonable level of protection for consumer privacy today.

Privacy fundamentalists agree with statement 1 and disagreewith statements 2 and 3. Privacy unconcerned subjects dis-agree with statement 1 and agree with statements 2 and 3. Allother subjects are considered privacy pragmatists [7, 9].

In addition, we included questions adapted from the surveyinstrument in [2], designed to elicit general attitudes and be-havior related to privacy online. We adapted some questionsto be Facebook specific. These questions are shown in table1 and were rated on a 5-point Likert scale from “Never” to“Always” in accordance with the initial survey design.

In the second half of the pre-test, we asked users about theirFacebook-specific questions. These were also adapted fromquestions in the [2] survey instrument. Subjects answeredeach on a 5-point Likert scale ranging from “Not at all” to“Very much” in accordance with the original survey design.These questions are shown in table 2.

Finally, we asked subjects to tell us if they believed Facebookapps could or could not access specific types of personal in-formation These are shown in table 3.

With the right permissions, apps can access any of the datapoints listed here, though not all apps can do it all the time,and users have the ability to deny this access in some cases.

Some information, like name, gender, and profile photo, canbe accessed by all apps with basic permissions. Other infor-mation can be accessed, but only when apps explicitly requestextended permissions.

There are also varying levels of extended permissions. Forsome extended permissions, a user has no way to deny theaccess except by refusing to install the app. In other cases,the user can install the app but deny access. For very sensi-tive types of information, like access to a user’s inbox, clearadditional dialogue boxes are shown to alert users about thepermission they are granting.

In table 3, items highlighted in green can be accessed by allapps. Items highlighted in yellow can be accessed throughextended permissions that the user cannot opt out of; i.e. ifan app requests it, the user cannot install the app withoutgranting permission. Items highlighted in pink can only beaccessed through an optional opt-in extended permission re-quest.

Thus, we might expect some users to say apps cannot accesscertain data points, even if they know that apps can techni-cally get them, because the users would choose to deny ac-cess.

ConditionsSubjects viewed information about what data apps could ac-cess on Facebook. We had three diverse conditions and afourth condition that combined the first three. Note that ourgoal was not to isolate and measure the impact of specificattributes of each condition, but rather to survey a range ofapproaches for communicating privacy information.

Condition 1: Privacy Policy - Subjects were instructed toread the Facebook Data Use Policy regarding personal infor-mation1 and Websites and Applications2.

Condition 2: Take This Lollipop - “Take This Lollipop” isan interactive horror movie that integrates a viewer’s Face-book profile information. A creepy stalker views the user’sprofile before mapping a route to their current location anddrive there in a frenzy with the user’s picture taped to hisdashboard. The movie illustrates the wealth of profile infor-mation that an app can access (see Figure 1) in a context thatconnects it with risk.

Although the app requests many extended permissions whena user installs it, and these are explicitly enumerated (see Fig-ure 2), when it launched, even users experienced with Face-book privacy setting expressed surprised at how much data itdisplays. For this reason, we chose it as an app that illustratesthe data that can be obtained through Facebook as well as onethat presents it in an engaging, entertaining, and risk-orientedcontext.

Condition 3: Facebook App Permissions App - In this con-dition, participants were asked to install an app that we devel-oped. This provided an interface to explore their data, and itfocused on what facebook calls “basic info”: Name, Profile1https://www.facebook.com/about/privacy/your-info2https://www.facebook.com/about/privacy/your-info-on-other

3

Table 1. Questions regarding general online privacy concern. Each question is rated on a 5 point Likert scale from “Never” (1) to “Always” (5), exceptfor the final question which is shown as the percentage of yes answers.

Question Average ScoreDo you...read a websites privacy policy before you register your information? 2.21look for a privacy certification on a website before you register your information? 2.48read license agreements fully before you agree to them? 1.79block people on Facebook that you do not want to see your information? 3.63use apps on Facebook? 59.1%

Table 2. Answers pre- and post-test to questions about concern Facebook apps. Values are on a 5-point Likert scale ranging from “Not at all” (1) to“Very much” (5). Significant increases are marked * for p < 0.05 and ** for p < 0.01.

Concern Pre-testAvg.

Post-testAvg.

Change

In general, how concerned are you about your privacy while you areusing Facebook?

3.533 3.925 0.392 **

Are you concerned...that Facebook will sell or release your personal information? 3.483 3.833 0.350 **that you are asked to grant access to too much personal information whenyou install a Facebook app?

4.283 4.258 -0.025

about online identity theft via Facebook? 2.867 3.508 0.642 **about people you do not know obtaining personal information about youfrom your Facebook use?

3.683 4.058 0.375 **

that content you post on Facebook may be seen by people other thanthose for whom you intended it?

3.858 4.183 0.325 **

that a Facebook app could post to your timeline in your name? 4.125 4.183 0.058that a seemingly legitimate Facebook app may be fraudulent? 3.833 4.042 0.208 *that Facebook apps could collect your personal data? 4.058 4.158 0.100that Facebook apps could sell your personal data? 4.050 4.158 0.108

Table 3. Fact-based questions about information Facebook apps can access. The percentage of “yes” responses are shown. Significant increases aremarked * for p < 0.05 and ** for p < 0.01.

Question Pre-test Post-test ChangeFrom what you know right now, when you install aFacebook app, do you think it has access to the fol-lowing information?Your personal information (name, gender, profilephoto)

98.3% 99.2% 0.8%

Your friend list 92.5% 95.8% 3.3%Your profile information (education and work history,relationship status, religion, etc.)

88.3% 91.7% 3.3%

Your birthday 90.8% 90.8% 0.0%Your likes (i.e. pages you like) 85.0% 94.2% 9.2% *Your status updates 69.2% 85.0% 15.8% **Photos and videos you upload 65.8% 88.3% 22.5% **Comments on your content (e.g. photos or status up-dates)

62.5% 79.2% 16.7% **

Your friends’ profile information 66.7% 80.8% 14.2% **Your friends’ posts that appear on your news feed 57.5% 85.8% 28.3% **Comments on your friends posts (from you and others) 53.3% 72.5% 19.2% **Posts you made on your timeline 74.2% 91.7% 17.5% **Comments on your timeline posts (by others) 66.7% 85.0% 18.3% **Your current location 82.5% 84.2% 1.7%Your contact information (email, phone, address) 85.0% 83.3% -1.7%Your private messages to friends 26.7% 40.0% 13.3% **Your chat logs 23.3% 40.0% 16.7% **

4

Figure 2. Data access requested by Take This Lollipop

Picture, Gender, User ID, Friends List, and any other infor-mation made public. Beyond this, we only request two othertypes of data: access to the users news feed as well as anattribute that shows if the user owns any Facebook Pages orApplications. Users can deny this request.

The approach of our app differs from Take This Lollipop,which requests extensive extended permissions. The “basic”information we use is commonly accessed by all applications,so our app essentially shows what almost every app is able toaccess.

After a user installs the app, the most recent data from theirprofile is collected and then redisplayed to the user in differ-ent interactive ways using jQuery and standard HTML/CSSdisplay widgets. The final information display is organizedinto five areas: basic information, a friends list, recent newsitems, recent page likes, and ownership permissions on pagesor groups within the facebook platform. Figure shows twoexample tabs from this app.

Condition 4: All of the above - Subjects in condition 4viewed all of the information described above: the privacypolicies, Take This Lollipop, and the App Permissions App.

SubjectsWe solicited for subjects through mailing lists and posts tosocial media channels. We received complete data from 120subjects. Gender breakdown includes 71 female, 47 male,and 2 unreported. Age ranged from 18- to 66-years-old withan average of 32.3 years and a median of 30. Our subjectsskewed high on the educational background, with 1 havingcompleted some high school, 6 completing some college, 37with bachelors degrees, 58 with masters degrees, and 18 withdoctorates.

RESULTSFor clarity in the text, significance levels will be indicatedwith a * for p < 0.05 and ** for p < 0.01. All tests unlessdescribed otherwise are paired t-tests comparing scores fromthe pre-test and post-test. Since each set of data requiredmany comparisons, we used a Bonferroni correction whereappropriate.

Figure 3. The interface of the Facebook Data App

Overall user understanding of privacy risksThe results in table 2 show subjects’ opinions about Facebookapps’ access rights before and after viewing the information.On average, subjects generally believed the basic informa-tion was accessible more than the non-optional extended per-mission data, and both were available more than the optionalextended permission data. That said, even for the basic in-formation that all apps can access, we did not see 100% ofusers believing that apps could access it, even after viewingthe information in the conditions that clearly indicated whatwas given to apps.

For the non-optional extended permission data, which anyapp can request access to and for which users cannot denypermission (except by refusing to install the app), we sawsignificant increases in the percentage of users who believedapps could access this data. However, few of these saw over90% of users believing apps could access the data, let alone100% of users.

Finally, for the most private data (accessible to apps, but onlythrough extended permission requests that the user can deny),a high percentage of users believed contact information wasmade available to apps. Many fewer users believed chat logsand messages (both of which reside in a user’s Facebook in-box) were accessible. It is possible that some users wouldsay this information is inaccessible since they would person-ally deny access to apps that requested it. It is also possiblethat the way in which apps can get to this information is notmade clear by any of the privacy information sources.

5

Table 4. This table shows differences in the pre-test percentage of appusers and non-users who believe Facebook apps can access certain typesof data. Differences are significant in all cases for p < 0.05. We see thatapp users are significantly less likely to believe that apps have access tothis data.

Data Non-users

Appusers

Status Updates 81.3% 60.6%Photos 77.1% 57.7%A user’s content and comments 79.2% 52.1%Newsfeed data 72.9% 46.5%Comments on friends’ posts 79.2% 36.6 %Timeline posts 87.5% 54.8%Others’ comments on timelineposts

83.3% 56.3%

Looking at all of these data points, we see that users areunder-informed about what information apps can access, andthough education about the data access permissions can help,users still underestimate the amount of information that appscan access. This, in turn, has implications for how they makedecisions. We discuss this further below.

We then divided subjects into two groups: those who reportusing Facebook apps to those who report not using them.Forty-eight subjects claimed to not use apps (“non-users”’),while 71 do use them (“app users”).

Non-users were significantly* more likely than app users tobelieve that apps could access Status Updates, photos, a user’scontent and comments, newsfeed data, comments on friends’posts, timeline posts, and others’ comments on timeline posts.This data is shown in table 4. This shows that people whoare using apps are generally quite under-informed about whatpersonal information they are handing over.

Among app users, post-test perception about what data face-book can access significantly increased** for nearly all items:accessing likes, photos, content and comments, friend lists,friend information, newsfeed data, private messages, chatlogs, comments on friends’ posts, timeline posts, and time-line comments. For non-app users, concern increased regard-ing apps accessing status updates**, photos**, chat longs*,and timeline posts**.

We also hypothesized that pre-existing concern about privacymight influence the results. We applied the Westin/HarrisPrivacy Segmentation Model described above and found oursample contained 31 Privacy Fundamentalists, 88 PrivacyPragmatists, and no Privacy Unconcerned subjects.

Not surprisingly, Fundamentalists started out with more over-all concern about privacy on Facebook compared with Prag-matists (3.935 vs 3.375*), concern that Facebook would selltheir data (3.935 vs 3.307*) and that unintended users wouldsee their data (4.27 vs. 3.7*).

In the pre-test, significantly* more Fundamentalists thanPragmatists believed apps could access their content andcomments, friends’ information, newsfeed, private messages,and contact information. In the post-test survey, fundamen-

talists saw no significant changes in their concerns or beliefsabout what data apps could access, while Pragmatists saw sig-nificant increases on 17 out of 27 survey questions.

It is worth noting that privacy Pragmatists and Fundamental-ists are not parallel groups to app users and non-users. Therewas no significant difference in the percentage of app usersbetween Fundamentalists and Pragmatists (50% vs 65% re-spectively).

Overall user privacy concernOverall, we saw significant increases in concern about a num-ber of factors after subjects had viewed the privacy informa-tion (across all conditions). This data is presented in table 2along with significance levels.

Overall concern about privacy on Facebook significantly in-creased 9.8% on the post-test, from an average of 3.53 to3.93. Increases also appeared in concern about facebook sell-ing / releasing data, identity theft, unknown people access-ing a subjects’ data, unintended people viewing the data, andseeming legitimate apps being fraudulent. It is worth notingthat for the concerns that did not see a significant increase,initial concern was quite high, over 4 on a 5 point scale for allfactors. This left little room for increase.

The impact of privacy communication techniquesIncreases in concern and perception varied based on whichcondition subjects interacted with between the pre- and post-tests. Table 5 shows the data for each group on the questionsdescribed above.

The “Privacy Policy”, “Take This Lollipop”, and “All of theabove” conditions all resulted in significantly higher concernsand perceptions about what data apps on Facebook could ac-cess. Specifically, overall concern and concern about identitytheft and unknown people accessing personal information in-creased for all three conditions.

The “Take This Lollipop”, and “All of the above” conditionsheightened user concern and awareness the most often. “TakeThis Lollipop” led to increases in 5 of 10 concern questions,and “All of the above” brought about increases in 8 of 10;the two exceptions were items on which the initial concernratings were very high.

Similarly, “Privacy Policy”, “Take This Lollipop”, and “All ofthe above” conditions all saw significantly higher percentagesof users believing apps could access specific data. For each,7 out of 17 perceptions significantly increased.

The exception here is our App Permissions App, which re-sulted in no increased concern and two significant decreasesin perception about what data apps can access.

6

Tabl

e5.

Ans

wer

sto

surv

eyqu

estio

nspr

e-an

dpo

st-t

estg

roup

edby

cond

ition

.Sig

nific

antc

hang

esfr

ompr

e-to

post

-tes

tare

show

nw

ithan

*fo

rp<

0.05

and

**fo

rp<

0.01

.Whe

na

cond

ition

indi

cate

stha

tapa

rtic

ular

data

poin

tcan

beac

cess

edby

anap

p,it

ishi

ghlig

hted

inye

llow

.Pr

ivac

yPo

licy

Lol

lipop

App

Perm

issi

ons

App

All

ofth

eab

ove

Que

stio

n

Pre-Test

Post-test

Change

Pre-Test

Post-test

Change

Pre-Test

Post-test

Change

Pre-Test

Post-test

Change

Inge

nera

l,ho

wco

ncer

ned

are

you

abou

tyo

urpr

ivac

yw

hile

you

are

usin

gFB

?A

reyo

uco

ncer

ned.

..

Inge

nera

l,ho

wco

ncer

ned

are

you

abou

tyo

urpr

ivac

yw

hile

you

are

usin

gFB

?3.

417

3.77

80.

361

**3.

471

3.97

10.

500

**3.

462

3.57

70.

115

**3.

875

4.45

80.

583

**

that

FBw

illse

llor

rele

ase

your

pers

onal

info

rmat

ion?

**3.

556

3.75

00.

194

**3.

412

3.97

10.

559

**3.

385

3.61

50.

231

**3.

583

4.00

00.

417*

*th

atyo

uar

eas

ked

togr

anta

cces

sto

too

muc

hpe

rson

alin

form

atio

nw

hen

you

inst

alla

FBap

p?**

4.25

04.

083

-0.1

67**

4.38

24.

500

0.11

8**

4.23

13.

923

-0.3

08**

4.25

04.

542

0.29

2**

abou

tonl

ine

iden

tity

thef

tvia

FB?

**2.

583

3.16

70.

583

**2.

882

3.67

60.

794

**3.

077

3.23

10.

154

**3.

042

4.08

31.

042

**ab

outp

eopl

eyo

udo

notk

now

obta

inin

gpe

rson

alin

form

atio

nab

outy

oufr

omyo

urFB

use?

**3.

417

3.75

00.

333

**3.

706

4.20

60.

500

**3.

846

3.80

8-0

.038

**3.

875

4.58

30.

708

**

that

cont

enty

oupo

ston

FBm

aybe

seen

bype

ople

othe

rth

anth

ose

for

who

myo

uin

tend

edit?

**3.

944

4.00

00.

056

**3.

588

4.14

70.

559

**3.

846

4.00

00.

154

**4.

125

4.70

80.

583

**

that

aFB

app

coul

dpo

stto

your

timel

ine

inyo

urna

me?

**4.

222

4.33

30.

111

**4.

029

4.11

80.

088

**3.

962

3.84

6-0

.115

**4.

292

4.41

70.

125*

*th

ata

seem

ingl

yle

gitim

ate

FBap

pm

aybe

frau

dule

nt?

**3.

889

4.00

00.

111

**3.

882

4.17

60.

294

**3.

769

3.73

1-0

.038

**3.

750

4.25

00.

500

**th

atFB

apps

coul

dco

llect

your

pers

onal

data

?**

4.02

84.

056

0.02

8**

4.00

04.

088

0.08

8**

4.15

44.

000

-0.1

54**

4.08

34.

583

0.50

0**

that

FBap

psco

uld

sell

your

pers

onal

data

?**

4.05

64.

111

0.05

6**

4.00

04.

147

0.14

7**

4.03

83.

885

-0.1

54**

4.12

54.

542

0.41

7**

From

wha

tyou

know

righ

tnow

,whe

nyo

uin

stal

laF

Bap

p,do

you

thin

kit

has

acce

ssto

the

follo

win

gin

form

atio

n?(v

alue

sgi

ven

inpe

rcen

tage

s)

You

rper

sona

linf

orm

atio

n**

100.

010

0.0

0.0

**94

.197

.12.

9**

100.

010

0.0

0.0

**10

0.0

100.

00.

0**

You

rpro

file

info

rmat

ion

**88

.994

.45.

6**

85.3

94.1

8.8

**96

.280

.8-1

5.4

**83

.395

.812

.5**

You

rsta

tus

upda

tes

**69

.488

.919

.4**

73.5

88.2

14.7

**76

.973

.1-3

.8**

54.2

87.5

33.3

**C

omm

ents

onyo

urco

nten

t**

69.4

83.3

13.9

**61

.888

.226

.5**

61.5

61.5

0.0

**54

.279

.225

.0**

You

rfri

end

list*

*91

.791

.70.

0**

88.2

94.1

5.9

**96

.210

0.0

3.8

**95

.810

0.0

4.2*

*Fr

iend

spo

sts

that

appe

aron

your

new

sfe

ed69

.486

.116

.7**

41.2

85.3

44.1

**69

.280

.811

.5**

50.0

91.7

41.7

**C

omm

ents

onyo

urfr

iend

spo

sts

**63

.980

.616

.7**

47.1

76.5

29.4

**53

.857

.73.

8**

45.8

70.8

25.0

**Po

sts

you

mad

eon

your

timel

ine

**80

.694

.413

.9**

70.6

88.2

17.6

**80

.884

.63.

8**

62.5

100.

037

.5**

Com

men

tson

your

timel

ine

post

s**

75.0

94.4

19.4

**64

.788

.223

.5**

69.2

61.5

-7.7

**54

.291

.737

.5**

You

rlik

es**

83.3

97.2

13.9

**79

.494

.114

.7**

92.3

92.3

0.0

**87

.591

.74.

2**

Phot

osan

dvi

deos

you

uplo

ad**

77.8

91.7

13.9

**58

.888

.229

.4**

69.2

73.1

3.8

**54

.210

0.0

45.8

**Y

ourb

irth

day

**91

.797

.25.

6**

85.3

91.2

5.9

**96

.276

.9-1

9.2

**91

.795

.84.

2**

You

rfri

ends

profi

lein

form

atio

n**

75.0

80.6

5.6

**61

.879

.417

.6**

73.1

73.1

0.0

**54

.291

.737

.5**

You

rpriv

ate

mes

sage

sto

frie

nds

**36

.150

.013

.9**

17.6

44.1

26.5

**26

.923

.1-3

.8**

25.0

37.5

12.5

**Y

ourc

hatl

ogs

**33

.350

.016

.7**

14.7

50.0

35.3

**23

.119

.2-3

.8**

20.8

33.3

12.5

**Y

ourc

onta

ctin

form

atio

n**

83.3

83.3

0.0

**85

.385

.30.

0**

92.3

69.2

-23.

1**

79.2

95.8

16.7

**Y

ourc

urre

ntlo

catio

n**

83.3

86.1

2.8

**76

.579

.42.

9**

84.6

73.1

-11.

5**

87.5

100.

012

.5**

7

Figure 4. Data access requested by our App Permissions App

To understand what users were actually learning, we firstlooked at what information each condition communicated tousers. If a condition explicitly states that apps can accesscertain data points, ideally all users will understand the appcan access that info. If a condition does not mention or illus-trate access to a particularly data point, then any significantincreases in user perception about app access may be a resultof paranoia or some other effect.

In table 5, rows have been highlighted indicating the infor-mation a condition specifically showed that apps could accessthat data point.

The Facebook privacy policies clearly state that all appscan access a user’s Name, Profile Picture, Gender, User ID,Friends List, any other information a user has made public,and this same information for each friend. It also says thatan app may access “additional information, such as stories,photos or likes” with additional permission, but it does notenumerate the data points beyond the examples.

Take This Lollipop requests a long list of permissions (seefigure 2). News Feed access includes access to commentsfrom friends and those that a user has posted. Facebook userswould know that their News Feed includes this information,and the Take This Lollipop video clearly shows it.

Our App Permissions App requests only a basic set of infor-mation (see Figure 4). Note that the details of “public profile”appear when the user hovers over that link in the privacy in-formation. This notification makes it clear that we will accessthe user’s Name, age range, profile picture, gender, language,country, friend list, and News Feed.

There is a difference in the way Take This Lollipop and theApp Permissions App show access to the News Feed. In TakeThis Lollipop, it is shown as the News Feed appears to userson the Facebook Site. In the App Permissions App, statusupdates from the News Feed are shown individually, but theyare shown out of the context of the Facebook interface.

Seventeen out of 20 significant increases in user perceptionabout what data apps can access occurred when a conditionshowed that the data could be accessed. At the same time,both the privacy policy condition and Take This Lollipop hadlarge significant increases in the percentage of users who be-lieved apps could access their chat logs, even though neithercondition showed that such data was available. There was

also a large significant increase in the percentage of subjectswho believed apps could access private messages to friendsafter watching Take This Lollipop; again, the video does notshow that such access is possible.

The App Permissions App had two significant decreases inthe percentage of users who thought apps could access theuser’s profile information and contact information. The appdid not show that it could access contact information. It didshow access to a user’s profile information (education workhistory, relationship status, religion, etc.), but only if the in-formation had privacy settings that made it publicly visible.For users with privacy restrictions, this information may nothave appeared thus leading users to conclude apps could notaccess it.

DISCUSSIONA number of insights arise from the results of this study. First,we found that users are concerned about privacy on Facebook,particularly with respect to the information apps can access.At the same time, users were generally under-informed aboutwhat information Facebook apps can access. There were nocases where 100% of users believed apps could access theinformation we asked about, and in many cases, a large per-centage of users were unaware that apps could access certaindata. Even more dramatically, our subjects who reported us-ing apps on Facebook were significantly less likely to knowwhat data the apps could obtain. In many cases, fewer than60% of app users were aware that apps could access particulardata points.

We found that education can improve user understandingof what apps can access. In parallel, this education alsoincreased users’ concerns about the accompanying privacyrisks. The more information people saw, the more concernedthey became. Subjects who looked at information from allthree information sources had the highest post-test levels ofconcern.

At the same time there were some “unjustified” increases inuser perception. For three data points, we saw large increasesin the percentage of users who believed apps could accessthem, even though the information subjects looked at betweenthe pre- and post-test did not indicate that apps could in factget to this data. While these users were not incorrect in think-ing apps could see those data points, we have no clear ex-planation for why their opinions changed between the tests.Similarly, we saw significant decreases in the percentage ofusers who thought apps could access two types of data afterthey viewed the App Permission App.

These issues lead to important areas of future work. Each ofthe information sources we used had different attributes bothin terms of the information they presented, the detail that wasused, and the style of presentation. We did not examine thisissue specifically, but extrapolating from these results, it ap-pears that the more dramatic style of Take This Lollipop didnot lead to greater understanding of what apps could access;we saw more significant increases with the privacy policycondition. However, there were more increases in concernabout privacy with Take This Lollipop. In addition, we saw

8

some significant increases in user belief that apps could ac-cess certain data points, even when the information sourcesdid not show that these could be accessed.

We hypothesize that this could be because the personalizedand horror-oriented nature of the Take This Lollipop videomimics some of the effects of a personal negative privacyevent. As discussed in [3] and [8], when a user personally ex-periences a negative consequence related to privacy, it is moreeffective at increasing users’ privacy awareness and concern.

At the same time, the App Permissions App, which showedsamples of what data an app could access, was less effectiveat increasing user understanding of data access policies. Sub-jects in this condition also had no significant increases in pri-vacy concern. It is unclear why users’ perception of what dataapps could access decreased in some cases, but understand-ing this is an important question to follow up on. If certainaspects of an interface can mislead users into falsely believingtheir data is private, those should be isolated and included asimportant attributes to avoid in design guidelines going for-ward.

A controlled study that looks at information provided to users,the presentation style, and the effect it has on both concernand perception will likely yield important insights about howto best inform users about the information they are sharing.It will also help researchers and designers understand whatmight cause the most concern in users, which should be bal-anced with the risks of the behavior.

Who exactly will provide this information to users is anotherimportant issue to consider. There is a question about whethersocial media sites like Facebook have an incentive to let usersknow exactly what apps can access and how. We found an in-crease in privacy concerns after users became more informedabout data access policies. This could lead users to interactless with apps, thus decreasing engagement with a site and,in turn, reducing its profits. If an under-informed user baseis more profitable, it may fall to third parties to fully educateusers about what they are sharing and what the risks are whenthey make the choice to install.

Finally, we see this study as one step in a greater researchagenda of integrating HCI and cybersecurity. Privacy and se-curity research has often overlooked the human element. Us-ability of security features can be low, leading to riskier be-havior from users. Poor presentation of privacy informationcan lead to uninformed users making bad choices about whatpersonal information to share and what apps to interact with.Better understanding users and their behavior in context canlead to improved security and privacy.

CONCLUSIONSIn this study, we looked at users’ concerns and perceptionsabout privacy, particularly in the context of Facebook apps.We conducted an experiment where subjects were surveyedabout their concerns and perceptions regarding what dataFacebook apps could access. They completed a survey, thenviewed information about Facebook app data access, and re-took the survey.

While subjects did report concern about privacy, we foundthey do not have a full understanding of how that informationis shared with apps. Subjects who reported using Facebookapps were significantly less informed about what data thoseapps could access when compared with subjects who did notuse them.

We compared the effectiveness of three methods for educat-ing users about app data access in four conditions. We foundthat overall, viewing this material increased privacy concernand understanding about what information apps could access.At the same time, some methods were more effective thanothers. Future work is needed to understand the dynamicsof communication mechanisms, design, attention, style, andother factors that affect understanding and concern.

Finally, this study is a first step in a larger space of work con-necting humans, HCI, and cybersecurity. People ultimatelychoose what data to share and how to protect it. They can-not do this in the optimal way without a full understand-ing of how it is shared, with whom, and what could resultfrom that sharing. There are important research questions inthis area for psychology, cognitive science, communications,computer science, and design. We hope this work is a step inthe direction of better understanding initial questions in thisexpanding space of research.

REFERENCES1. Acquisti, A., and Gross, R. Imagined communities:

Awareness, information sharing, and privacy on thefacebook. In Privacy enhancing technologies, Springer(2006), 36–58.

2. Buchanan, T., Paine, C., Joinson, A. N., and Reips,U.-D. Development of measures of online privacyconcern and protection for use on the internet. Journalof the American Society for Information Science andTechnology 58, 2 (2007), 157–165.

3. Debatin, B., Lovejoy, J. P., Horn, A.-K., and Hughes,B. N. Facebook and online privacy: Attitudes, behaviors,and unintended consequences. Journal ofComputer-Mediated Communication 15, 1 (2009),83–108.

4. Dwyer, C., Hiltz, S. R., and Passerini, K. Trust andprivacy concern within social networking sites: Acomparison of facebook and myspace. In AMCIS(2007), 339.

5. Emanuel, L., Bevan, C., and Hodges, D. What does yourprofile really say about you?: privacy warning systemsand self-disclosure in online social network spaces. InCHI ’13 Extended Abstracts on Human Factors inComputing Systems, CHI EA ’13, ACM (New York, NY,USA, 2013), 799–804.

6. Fang, L., Kim, H., LeFevre, K., and Tami, A. A privacyrecommendation wizard for users of social networkingsites. In Proceedings of the 17th ACM conference onComputer and communications security, CCS ’10, ACM(New York, NY, USA, 2010), 630–632.

9

7. Interactive, H. Privacy on and off the internet: Whatconsumers want. Tech. rep., Harris Interactive,Conducted for Privacy and AmericanBusiness,November 2001.

8. King, J., Lampinen, A., and Smolen, A. Privacy: is therean app for that? In Proceedings of the SeventhSymposium on Usable Privacy and Security, SOUPS’11, ACM (New York, NY, USA, 2011), 12:1–12:20.

9. Kumaraguru, P., and Cranor, L. F. Privacy indexes: Asurvey of westins studies. Institute for SoftwareResearch International (2005).

10. Lin, S.-W., and Liu, Y.-C. The effects of motivations,trust, and privacy concern in social networking. ServiceBusiness 6, 4 (2012), 411–424.

11. Litt, E. Understanding social network site users privacytool use. Computers in Human Behavior 29, 4 (2013),1649–1656.

12. Nov, O., and Wattal, S. Social computing privacyconcerns: antecedents and effects. In Proceedings of the

SIGCHI Conference on Human Factors in ComputingSystems, CHI ’09, ACM (New York, NY, USA, 2009),333–336.

13. Strater, K., and Lipford, H. R. Strategies and struggleswith privacy in an online social networking community.In Proceedings of the 22nd British HCI Group AnnualConference on People and Computers: Culture,Creativity, Interaction - Volume 1, BCS-HCI ’08, BritishComputer Society (Swinton, UK, UK, 2008), 111–119.

14. Strater, K., and Richter, H. Examining privacy anddisclosure in a social networking community. InProceedings of the 3rd symposium on Usable privacyand security, ACM (2007), 157–158.

15. Wang, N. Third-party applications’ data practices onfacebook. In CHI ’12 Extended Abstracts on HumanFactors in Computing Systems, CHI EA ’12, ACM (NewYork, NY, USA, 2012), 1399–1404.

10