“I Didn’t Sign Up for This!”Informed Consent in Social Network Research

Luke Hutton and Tristan HendersonSchool of Computer Science

University of St Andrews, St Andrews, Fife, United Kingdom{lh49, tnhh}@st-andrews.ac.uk

Abstract

The issue of whether, and how, to obtain informed con-sent for research studies that use social network data hasrecently come to the fore in some controversial cases.Determining how to acquire valid consent that meetsthe expectations of participants, while minimising theburden placed on them, remains an open problem. Weapply Nissenbaum’s model of contextual integrity to theconsent process, to study whether social norms of will-ingness to share social network data can be leveragedto avoid burdening participants with too many interven-tions, while still accurately capturing their own sharingintent. We find that for the 27.7% of our participants(N = 109) who conform to social norms, contextualintegrity can be used to significantly reduce the timetaken to capture their consent, while still maintainingaccuracy. Our findings have implications for researchersconducting such studies who are looking to acquire in-formed consent without having to burden participantswith many interactions.

IntroductionIn recent years, many scientific disciplines have become in-terested in using data collected from social network sites(SNSs), such as Facebook and Twitter. Such data are con-sidered a rich source for understanding social dynamics,from analysing discussion of sleep disorders (Powell et al.2012), to exploring how bereaved people use Facebook togrieve (Getty et al. 2011); a comprehensive review of howFacebook has been used in social science research is pro-vided by Wilson et al. (2012). Such research can be of highvalue, but raises myriad ethical questions. Studies leveragingsocial network data may do so without the people who origi-nally published it knowing. With such studies conducted re-motely with no direct contact between the researchers andthe content creators whose data are used, it is uncertainwhether they should be considered participants, and experi-mental procedures subject to the same scrutiny as other hu-man subjects research. There is disagreement about whetherdata derived from public sources such as Twitter should befair game for researchers, or whether repurposing such datafor research violates the expectations of content creators.

Copyright c© 2015, Association for the Advancement of ArtificialIntelligence (www.aaai.org). All rights reserved.

The debate came to a head when researchers at Facebookpublished a study exploring emotional contagion throughmanipulation of Facebook’s News Feed product (Kramer,Guillory, and Hancock 2014). When it emerged that Face-book did not seek informed consent from participants, therewas widespread condemnation of the ethical standards ofthe experiment (Hill 2014). Since the study was conducted,Facebook has added the word “research” to a clause in theirData Use Policy regarding “how we use the information wereceive”, one of three policy documents users are asked toagree to before registering.1 While Facebook considers thisclause to be an acceptable proxy for gaining informed con-sent for individual studies, it betrays the expectation thatparticipants engage in research knowing what informationis collected, and for what purpose, which Warrell and Ja-cobsen (2014) contend is essential when conducting suchstudies.

The attraction of such “big data” research can be diffi-cult to reconcile with the logistics of gaining informed con-sent from participants. Big data that are public are not nec-essarily fair game for research (boyd and Crawford 2012).Even if someone agrees to participate in such an experi-ment, they may not be willing to share all of their socialnetwork data, some of which could be considered highlysensitive. The state of the art is to largely adopt one of twostrategies for gaining informed consent. Building on Lugerand Rodden’s (2013) recommendations for considering con-sent in ubiquitous computing, we term these secured andsustained consent. Secured consent is the traditional check-box or “end-user license agreement (EULA)”, where peopleprovide consent at a single point in time to an unqualifiedcollection of data, without respect to the nature or purposeof the study. While this approach is quick and trivial forparticipants, it risks violating their expectations if data arecollected and processed against their wishes. At the otherextreme, we consider sustained consent, where participantsare continuously probed about their willingness to share dis-crete pieces of data over the course of an experiment, charac-terised by studies such as Sleeper et al.’s (2013) examinationof self-censorship on Facebook, where participants chosewhen to share information with researchers over a periodof time. Such a consent strategy is likely to engender more

1Facebook Data Use Policy: facebook.com/policy.php

Proceedings of the Ninth International AAAI Conference on Web and Social Media

178

support from participants who have actively chosen what todisclose, at the risk of burdening them with a large numberof interventions which may cause frustration, and ultimatelylead to attrition, a problem for all longitudinal studies.

Previous social network studies have involved the col-lection and processing of large quantities of data from theusers of social network sites. In a recent study, we havesurveyed the literature to find that the extent to which peo-ple consented to their data being used is generally not col-lected or reported, with only 28 of 505 (5.5%) of papersdescribing their consent procedures (Hutton and Henderson2015). Therefore, we have little understanding as to whetherparticipants would have consented to their data being usedfor such purposes, or appreciate the implications of expos-ing their data to researchers. Studies which rely on a singlecheckbox in a boilerplate consent form risk exposing partic-ipants to inappropriate levels of disclosure, with the benefitof reduced cognitive and temporal load on the participant.Conversely, studies which ask the participant to explicitlysanction the disclosure of each piece of data may better meetthe expectations of participants, but at the cost of a signifi-cantly greater burden on the participant.

In this paper, we propose a third way of acquiring con-sent which aims to maximise the accuracy of the sustainedapproach, while achieving a low burden on participants, aswith the secured approach. We use Nissenbaum’s (2004)model of contextual integrity to leverage context-specific so-cial norms of willingness to share social network data in anexperiment which asks participants to consent to disclosinga subset of their Facebook data for a hypothetical study. Bydetecting whether users conform to such norms, we aim toreduce the number of times we need to explicitly ask fortheir consent, with a minimal loss in accuracy.

Specifically, we hypothesise the following:

• H1: Acquiring consent to share social network data withresearchers using contextual integrity reduces the burdencompared to current methods of acquiring explicit sus-tained consent, while more accurately reflecting people’sintent than secured consent methods which involve nosuch interventions.

• H2: Acquiring consent with contextual integrity is as ro-bust to temporal changes in willingness to share data assustained consent.

This paper makes the following contributions:

• We develop the first means of quantifying norms for shar-ing social network data.

• We apply contextual integrity to the study of informedconsent, with the first study to measure whether contex-tual integrity provides an acceptable trade-off between ac-curacy and the burden on people to acquire their consent.

Contextual integrityNissenbaum (2004) proposes contextual integrity as a theo-retical framework for considering information privacy. Sheargues that information is not inherently public or private,but governed by context-specific norms, which determine

to whom it is appropriate for information to be transmit-ted to, and for what purpose. Individuals, organisations, andsections of society each have their expectations about whatconstitutes the appropriate flow of information, and any ac-tor can perceive a privacy violation if these expectations arenot met. For example, in order to receive a diagnosis for amedical condition, a patient accepts that they must commu-nicate sensitive information about their health to a doctor.The information might generally be considered “private”,but both actors have an understanding of the norms govern-ing this sharing of information, and thus there is no privacyviolation. If, however, that doctor was to subsequently gos-sip about this condition to someone else, the expectations ofthe patient have been violated, and so has their privacy.

This concept, while simple, has profound implications forhow we consider information privacy. Our specific focus ison whether we can accurately infer context-specific normsby sampling people’s willingness to share social networkdata. We then identify if people conform to such norms, anduse the “wisdom of the crowd” as a proxy for individualdisclosure decisions, thus reducing the time spent collect-ing explicit consent from participants. Our aim is to demon-strate the usefulness of contextual integrity in this domainby illustrating what proportion of the population are norm-conformant, and can benefit from such inferences.

MethodWe conducted a study to investigate whether acquiring sus-tained informed consent based on contextual integrity per-forms better than two other strategies, summarised as:

• C1 Secured consent: Participants provide up-front con-sent to data acquisition in an informed consent form

• C2 Contextual integrity consent: Participants are askedexplicitly about their willingness to share each piece ofdata, unless they clearly conform to or deviate from a so-cial norm for sharing such data.

• C3 Sustained consent: Participants are asked explicitlyabout their willingness to share each piece of data.

Deriving willingness-to-share normsTo evaluate contextual integrity (C2), we first need to derivesocial norms so that we can detect conformity and deviation.To do so, we use a dataset from a previous study that exam-ined people’s willingness to share social network data withresearchers (McNeilly, Hutton, and Henderson 2013). In thestudy, participants were asked which of 100 pieces of theirFacebook data they were willing to share with researchers,of the following types:

• Location check-ins by the user• Names of Facebook friends• “Liked” Facebook pages• Photos uploaded by the user• Photo albums created by the user• Biographical attributes• Status updates

179

0.00

0.25

0.50

0.75

1.00

Friends Likes CheckinsStatus updates Photos Photo albums

Data type

Pro

po

rtio

n s

ha

red

Data type

Friends

Likes

Checkins

Status updates

Photos

Photo albums

Figure 1: Rates of sharing of different types for participantsin C3. The white diamonds represent the mean sharing rateof that type in the 2012 dataset, constituting the prevail-ing norm. Willingness to share data with researchers has in-creased on all fronts in this period.

We consider the proportion of shared content for each at-tribute to be the prevailing norm (Figure 1). We draw onthese norms in our study to determine the extent to whichour participants conform with them.

User studyParticipants were recruited online (through advertisementsin university mailing lists, Twitter, and Reddit) for a studywhich modelled the three consent strategies and evaluatedtheir performance, with participants randomly assigned toone of these strategies. The study comprised two parts: con-sent acquisition and a performance evaluation.

In the consent acquisition phase, participants were askedabout their willingness to share up to 100 of pieces of datarandomly extracted from their Facebook profile with re-searchers conducting a hypothetical study into “how emo-tions spread on Facebook” (Figure 2). These data were ofthe types found in the norms dataset, with the exception ofbiographical attributes. The strategy the participant was as-signed to affected the presentation of this step. The purposeof this step was to infer a “consent policy”, which repre-sented the subset of the data the participant was willing toshare, according to the rules of that strategy.

As we are unaware of other attempts to quantify the normsfor sharing SNS data, we used a simple method for determin-ing participants’ conformity to the norms derived from thesource dataset. After each participant answered a question,we performed a series of chi-square equality of proportionstests, comparing the distribution of responses by the partic-

Figure 2: A screenshot of a question in the study. In thiscase, the participant is asked to share the fact that they like aFacebook page with the researchers.

ipant for each data type to the distribution of the norm. Ifp ≤ 0.1, we considered the participant to conform to thenorm, and all further questions of that type were removed. Ifp ≥ 0.9, we considered the participant to deviate from thenorm, and again all questions of that type were removed. Asthe chi-square test assumes there are at least five counts ineach cell, we did not attempt to calculate norm conformityuntil the participant shared five items of a given attribute.Therefore, the method is not robust to those who share verysmall amounts of data. We chose to test at the 0.1 signifi-cance level, as early piloting of the method allowed a deter-mination to be made about conformity within a small num-ber of questions while maintaining accuracy. The results ofthis study will be used to vindicate this design decision.

Secured consent Participants in all conditions were askedto complete a boilerplate informed consent form, in accor-dance with current best practices and the ethical require-ments of our institution. For participants in this condition,an additional question was injected in to the consent form,adapted from a clause in Facebook’s Data Use Policy: “Iunderstand the researchers may use Facebook data they re-ceive about me for data analysis, testing, and research.” Forthese participants, completing this form was treated as theparticipant giving consent to all 100 pieces of content beingprocessed for the purpose of the study, forming the consentpolicy, and ending the consent acquisition phase.

180

Contextual integrity consent Participants were shown, inturn, each of the 100 pieces of data collected, and askedwhether they were willing to share it with the researchers.This strategy aimed to reduce the number of questions, how-ever, by constantly evaluating whether the participant con-formed to the prevailing social norm of willingness to sharethat type of data. Each time a participant answered a ques-tion, their norm conformity was measured for each of thesix data types they were asked about, removing redundantquestions if conformity was established. After the partici-pant completed this phase, the consent policy was based onthe proportions of each data type the participant was willingto share. The final policy consisted of the content explicitlyshared by the participant, augmented by additional contentconsistent with these proportions.

Sustained consent The presentation of this method wassimilar to the contextual integrity condition, in that all ques-tions were shown in turn to the participant, but no questionswere removed, as no norm conformity calculations weremade. The consent policy contained the subset of attributesexplicitly shared by the participant.

Figure 3: A screenshot of the performance evaluation step.Participants are shown the content collected according totheir consent policy, and asked to remove any items theydeem inappropriate to share. The proportion of items se-lected determines the accuracy of the method.

The second phase of the study, the performance evalu-ation, was the same for all conditions. Participants wereshown all of the data in their consent policy, and asked to

click on any items which they do not wish to have shared(Figure 3).

A week after the study was completed, participants wereasked to complete the study again. Assigned to the samecondition, participants completed the same process, with adifferent random subset of data.

We evaluated our results using the following metrics:

• Burden: How long the participants spent participating inthe study. This is measured as the percentage of potentialquestions that were presented to the participant. For se-cured consent this is 0%, as the method assumes that par-ticipants were willing to share everything. For sustainedconsent, this is 100%, as all participants were asked themaximum number of questions. The burden of the con-textual integrity condition lies somewhere between thesetwo extremes, with fewer questions asked depending onthe participant’s norm conformity throughout the study.

• Accuracy: How successfully the consent strategy meetsthe intent of users. This is measured as the percentage ofdata in the consent policy which the participant was alsowilling to share in the performance evaluation step. Weexpect the accuracy of the sustained consent condition tobe very high as all participants explicitly chose which datato share, while in the secured condition, accuracy is likelyto be much lower and more variable between participants,as they did not choose which content would be in the con-sent policy.

• Robustness: Previous work shows that willingness toshare social network data is temporally sensitive (Baueret al. 2013; Ayalon and Toch 2013), so we repeated thestudy after a week to observe the extent to which this ef-fect manifests, and to determine which methods are morerobust to it. The accuracy of using answers from the firstweek to predict responses to the second week providesthis measure, as it will highlight whether each method issensitive to changes in willingness to share informationover time.

Ethical considerationsBefore conducting the study, our experimental design wasscrutinised and approved by the relevant ethics commit-tee (our institution’s equivalent of an Institutional ReviewBoard). The experiment used our previously developedframework for privacy-sensitive handling of social networkdata (Hutton and Henderson 2013) which ensured only thedata necessary to execute the study was collected, and thatall data were appropriately sanitised and deleted during thelifecycle of the experiment.

Results154 people began the study, of whom 109 completed partic-ipation in the first week. 71 of these participants also com-pleted the second part of the study a week later. As shownin Table 1, participation was broadly equal across all threeconditions. In our analysis, we consider the responses ofall 109 participants, with only those who completed both

181

Condition Started Completedweek 1

Completedweek 2

Securedconsent

41 32 22

Sustainedconsent

55 39 24

Contextualintegrity consent

58 38 25

Table 1: Participants were assigned to one of the three con-ditions at random, and participation was broadly equal.

Category Response Study%

Facebook%

Gender Male 33 43.6Female 66 46.3

Age 18-24 72.3 20.525-34 21.8 24.735-44 7 17.945-54 2 14.255+ 0 13.7

Education High School 3 20Undergraduatedegree

66.3 32.1

Postgraduatedegree

31.7 3.6

Table 2: Comparison of demographics in our study to thoseof Facebook in the UK. Demographics are derived from datamade available to Facebook advertisers, correct as of Jan-uary 2015.2

weeks considered in our temporal analysis of the robust-ness of contextual integrity. Table 2 shows the demograph-ics of our sample population, compared to data Facebookmake available to their advertisers.2 As 77.2% of our partic-ipants live in the UK, we compare our sample to the UK’sdemographic make-up. A side-effect of primarily promotingthe study in university mailing lists is that we oversampleyounger, university-educated people. 52.6% of participantswere affiliated with our institution. To determine whethersuch participants would be more likely to share data withresearchers at their own institution, and thus skew the re-sults, we compared their overall willingness to share datawith other participants, finding no differences.

Is there a relationship between burden andaccuracy?We are interested in the relationship between burden – thepercentage of potential questions a participant is asked abouttheir willingness to share data with researchers, and accu-racy – whether inferences based on these questions satisfythe expectations of the individual.

As discussed earlier, participants in the secured consentcondition were not asked any questions about their willing-ness to share individual pieces of data, representing a burden

2Facebook Ads Manager: facebook.com/advertising

of 0%. Conversely, participants in the sustained consent con-dition were asked whether they were willing to share eachindividual piece of data collected in the study, a potentialburden of 100%.3 For users in the contextual integrity con-dition, we expected burden to lie somewhere in the middle.In the worst case, the burden would match that of sustainedconsent, however the more norm-conformant a participantwas, the fewer questions they were asked.

We expected that participants in the sustained consentcondition would yield the highest accuracy. As these partic-ipants were explicitly asked about their willingness to shareeach piece of data, the acquisition of these data should meettheir expectations. Conversely, we expected the secured con-sent condition to exhibit lower, and more variable accuracy.As this method does not account for differences betweenparticipants’ willingness to share data, it is unlikely to meetmost people’s expectations.

0%

25%

50%

75%

100%

0% 25% 50% 75% 100%Burden

Accura

cy Condition

Secured

Contextual integrity

Sustained

Figure 4: Scatterplot showing the relationship between accu-racy and burden in all three conditions. Accuracy is the mostvariable for those in the secured consent condition, whereasin the case of contextual integrity, a small loss in accuracy ismet with a greater time burden saving. Note that the pointshave been jittered to improve readability.

Figure 4 shows the relationship between burden and ac-curacy in all three conditions. As expected, participants inthe sustained consent condition mostly saw perfect accuracy.While secured consent does indeed exhibit the most variableaccuracy, it performs better than we originally anticipated;we discuss the implications of this later.

Behaviour in the contextual integrity condition is morevariable. While there is a similar tendency towards higheraccuracy, there is a notable drop in performance. As shownby the regression line, accuracy surprisingly drops with in-creased burden. This exposes an important detail about the

182

0%

25%

50%

75%

100%

Conforms Deviates Undetermined

Condition

Accu

racy

Figure 5: Boxplot showing how accuracy differs betweenparticipants in the contexutal integrity condition dependingon their norm conformity. Norm-conformant people achievehigher, and less variable, accuracy rates than those who de-viate from norms.

applicability of the contextual integrity method. When weexpand this condition to show the accuracy of participantsbased on their norm conformity, this trend is easier to under-stand. The technique attempts to detect the norm devianceand conformity of participants. As shown in Figure 5, theformer was not useful for maximising accuracy. As deviancerequires more questions to detect, this drags down overallaccuracy as burden increases. Later in this section we dis-cuss the implications of contextual integrity better servingcertain types of people.

When combining responses from all conditions, a one-way ANOVA shows no statistically significant effect of bur-den on accuracy (F(1,171)=0.903, p>0.1), suggesting thatas people’s behaviour is so diverse, improving the accuracyof consent acquisition simply through increasing the numberof interventions may not be sufficient.

Does contextual integrity reduce participantburden?Contextual integrity is used in the informed consent pro-cess to determine if people conform to social norms, andleverage this conformity to ask fewer questions about theirwillingness to share data. Participants in the sustained con-sent and contextual integrity conditions were asked a max-imum of 100 questions.3 In the contextual integrity condi-tions, questions were not asked if the participant was found

3For some, this number was smaller if they did not have enoughpieces of data in their Facebook profile of each type.

to be norm-conformant or deviant with respect to a particu-lar data type. On average, participants in the sustained con-dition were asked 81.9 questions, while contextual integrityparticipants were asked 67.2, an 21.9% decrease in burden.When comparing the distribution of burden between the twoconditions, a one-way ANOVA shows a statistically signif-icant difference (F(1,122)=25.15, p<0.05). This significantreduction is useful when conducting longitudinal studies, asit suggests the technique may allow fewer disruptive inter-ventions to acquire consent. This finding is only useful, how-ever, if accuracy is not compromised.

Does contextual integrity significantly reduceaccuracy?

0%

25%

50%

75%

100%

Secured Contextual integrity Sustained

Condition

Accu

racy

Figure 6: In all three conditions, median accuracy is high,although variability increases as the number of questionsasked is reduced.

In all conditions, mean accuracy is very high, only drop-ping from 99.3% in the sustained condition, to 92.7% in thecontextual integrity case, and 91.2% for secured consent.Accuracy is most variable in the contextual integrity and se-cured conditions, as depicted in Figure 6. Surprisingly, me-dian accuracy in the secured consent condition is close to100%, suggesting a large proportion of the sample were will-ing to share all of their data with researchers. While identi-fying the motivations for this are beyond the scope of thisstudy, previous work has shown that people are more will-ing to share social network data with academic researchersdepending on the purpose of the study (McNeilly, Hutton,and Henderson 2013).

As expected, variability for contextual integrity partici-pants lies between the two extremes of the other conditions.Despite the similarly high medians, an ANOVA comparingaccuracy between the contextual integrity and sustained con-

183

ditions suggests that the former exhibits a significantly loweraccuracy distribution, failing one of our performance metrics(F(1,120) = 7.45, p<0.05) The plurality of a high accuracycluster, and very low-performing outliers merits further ex-planation, and goes some way to explaining this apparentfailure.

Who does contextual integrity work for?

0%

25%

50%

75%

100%

5% 10% 15% 20% 25%Burden

Accura

cy

Conformity determination

Conforms

Deviates

Undetermined

Figure 7: Scatterplot showing the relationship between normconformity and accuracy. As indicated by the cluster after 5questions, high accuracy can be preserved when norm con-formity is detected quickly, although the technique is notuseful for people who are highly norm deviant. Note thatthe points have been jittered to improve readability.

We classify participants in the contextual integrity con-dition based on their norm conformity. Figure 8 shows thetime taken to determine which of these groups participantsbelong to. If a participant is norm-conformant, in the vastmajority of cases this is identified within just 7 questionsper attribute, allowing us to skip all further questions anduse this conformity as a proxy for their willingness to sharediscrete items. Figure 7 illustrates how this purging of ques-tions for norm-conformant and deviant participants affectsaccuracy, indicating the types of user that the contextual in-tegrity technique can support. The technique used to detectnorm conformity requires a person to agree to share at least5 pieces of data before being able to calculate whether ornot they conform to the norm. The large cluster of greenpoints here with high accuracy indicates that people who arenorm-conformant can be asked a small number of questionswhile maintaining accuracy. Indeed, when questions are re-moved at this point, the technique performs better than forpeople whose conformity could not be established (and forwhom no questions were removed), suggesting that people

0.00

0.25

0.50

0.75

1.00

0 10 20

Questions to determine conformity

Cum

ula

tive

pro

babili

ty

Conformity determination

Conforms

Deviates

Undetermined

Figure 8: Cumulative distribution function of the number ofquestions needed to determine whether participants in thecontextual integrity condition conform or deviate from thenorm, or if this could not be determined. If people conformto norms, this is detected after a small number of interven-tions.

who behave similarly to their peers can be asked fewer ques-tions and achieve higher accuracy.

27.7% of participants’ conformity was detected within 6questions while achieving more than 96.5% accuracy, the in-tercept with the undetermined regression line at this point.Across the whole condition, the same number of norm-conformant participants achieved this degree of accuracyor higher than undetermined participants, while on averageachieving a 41.1% reduction in burden, indicating that forsuch people, contextual integrity achieves both a reductionin burden and an increase in accuracy. This is an impor-tant result, as although it indicates that contextual integrityis not be appropriate for all people, within just 6 questionswe can detect whether a user’s consent can be accuratelycaptured, and apply an appropriate strategy based on theirbehaviour. Based on the slope of the regression lines, wecan determine best practices for application of the contex-tual integrity technique. If a person’s norm conformity canbe detected within 6 questions, then trusting this as a proxyfor their explicit performs very well. We also determinedwhether participants were significantly norm-deviant. Thatis, if their behaviour significantly deviated from the socialnorms, we would also remove questions and take their cur-rent sharing ratios as a proxy for willingness to share. Wefound that this approach does not perform well, as deviancerequires at least 15 interventions to determine, and is a verylow-performing proxy. As such, if people are significantlydeviant from social norms, it is best to continue asking them

184

questions, as high accuracy is maintained.

Is contextual integrity robust to temporal changesin willingness to share?

0.0

0.2

0.4

0.6

0.8

Secured Contextual integrity Sustained

Condition

Ove

r−sh

ari

ng

ra

tio

Figure 9: Boxplot showing how robust each condition is totemporal change by using the consent policy from the firstweek to predict the participant’s responses in the secondweek. All conditions exhibit over-sharing, highlighting thedifficulty of capturing consent at a single point in time.

Previous work has shown that attitudes towards pri-vacy and sharing social network data are temporallyvolatile (Bauer et al. 2013), and as such, decisions aboutwillingness to share data with researchers may only repre-sent thinking at a single point in time, and not a person’s“true” intent. This may cause regret, and perceived leak-age of social network data (Wang et al. 2011). The con-sent methods we examine in this study consider the tempo-ral issue differently. The secured consent method, perhapsthe most common in social network studies, assumes thata participant’s willingness to participate in a study is carteblanche to collect any data associated with them, and dis-regards any temporal effects. Conversely, sustained consentrelies on constant interventions to mitigate any drift in a per-son’s willingness to share data, which achieves high accu-racy at the cost of a significant burden on the participant.As a goal of the contextual integrity method is to reduce theburden on participants, we hypothesise that leveraging so-cial norms is more robust over time. If a user is found tohighly conform to social norms at one point in time, we ex-pect this to hold true, as a proxy for willingness to share dis-crete pieces of data, better than the secured consent method.As we have discussed earlier, we expected a small decreasein accuracy compared to sustained consent as the significantnumber of interventions ensures accuracy. By repeating the

study over a week, we capture changes in behaviour to de-termine the robustness of these techniques. To do this, weapply the consent policy of the first week’s results to predictthe participant’s responses in the second week. These predic-tions are validated by the participant’s responses to the per-formance evaluation questionnaire, the same way accuracyis measured. Figure 9 illustrates the extent to which thesepredictions would have led to over-sharing of data. Theseresults suggest that privacy attitudes do indeed change overthe course of a week. Across all conditions, trying to usepredictions from the previous week performs quite poorlyin many cases. This is most problematic in the case of se-cured consent because there is no way of accommodatingsuch changes in intent, suggesting that consent acquisition ina single moment in time is not sufficient. The sustained con-sent condition shows a very similar distribution, however inpractice this would be mitigated by continuing to interveneto capture consent, dismissing the need to rely on old data, atthe cost of higher participant burden. Surprisingly, the con-textual integrity condition performs poorly by this measureof robustness, however this is understandable in the contextof our previous result that the technique is only applicablefor about a quarter of the population who are highly norm-conformant, and an ANOVA suggests that there is no signifi-cant difference in over-sharing between conditions (F (2,65)= 0.168, p>0.1). As this condition includes participants ofvarying degrees of conformity, attempts to leverage this tomake longitudinal predictions for non-conformant partici-pants performs very poorly. In practice, users who have notbeen identified themselves as norm-conformant within 6 in-terventions would be excluded from a contextual integrity-based solution in favour of a sustained consent approachwhich would better capture their intent.

Related workThe ethical collection of data from Facebook and other pub-lic sources of online social network information has beenwidely discussed in the literature. Zimmer (2010) discussesproblems with the well-known T3 Facebook study con-ducted at Harvard (Lewis et al. 2008), and concludes thatsimply because a social network user chooses to share dataon an SNS, this does not give a researcher the right to collectsuch data; on the contrary, researchers must take even morecare with such data. Neuhaus and Webmoor (2012) simi-larly argue that academic researchers should take more carewith SNS data than commercial data processors, and pro-pose a set of best practices termed “agile ethics”. Such ag-ile ethics assume that data collection practices may changeover the course of research, thus implying that traditional se-cured consent may be insufficient. Solberg (2010) considersthat Facebook research has low risks, and that such researchdoes not require any ethics approval. Moreno et al.(2008) ex-amines SNS research involving adolescents, and argues that(secured) informed consent may be required but should bedecided on a case-by-case basis. The British PsychologicalSociety (2013) provide a detailed set of guidelines on obtain-ing “valid consent” from participants in Internet research.

Recent work by Morrison et al. (2014) is perhaps the clos-est to our own. They look at obtaining consent from partic-

185

ipants in mobile HCI experiments. They propose that onceparticipants have shared a particular quantity of data withresearchers, they should be presented with a personal rep-resentation of their shared data to determine whether theycontinue to grant the researchers consent to access thesedata. Such representations are found to be useful in a largeuser study. Our work differs from this in that we attempt touse contextual integrity as a more efficient way of detectingchanges in consent.

Contextual integrity has been widely discussed since itsintroduction over ten years ago (Nissenbaum 2004). Em-pirical evaluations of contextual integrity include studiesof Google Books (Jones and Janes 2010), cloud comput-ing (Grodzinsky and Tavani 2011), and blogging (Grodzin-sky and Tavani 2010), while Barkhuus (2012) argues thatit should be more widely used in HCI. To the best of ourknowledge, we are the first to apply contextual integrity inan empirical evaluation of informed consent.

Conclusions and future workWe have presented the first application of contextual in-tegrity to the acquisition of informed consent for sharingsocial network data. We conducted a study with 109 partic-ipants to identify whether contextual integrity could reducethe burden on participants while maintaining accuracy.

Returning to our hypotheses, we find qualified supportfor H1. On average, contextual integrity reduces burden by21.9%. While median accuracy is not significantly betterthan secured consent, for 27.7% of participants, contextualintegrity delivered perfect accuracy with a 41.1% reductionin burden. We also find support for H2, as the contextual in-tegrity method is not significantly less robust than sustainedconsent over time. Our results vindicate our decision to testnorm conformity at the 0.1 significance level, but furtherwork can demonstrate whether this boundary is appropriatein all cases.

We conclude that as human behaviour is so diverse, thereis no one-size-fits-all approach to consent that achieves op-timal results. We believe it is important for organisations ac-quiring consent for social network data to consider the im-plications of these results. The attraction of our use of con-textual integrity is that as norm conformity can be quicklyestablished, if a person clearly does not conform to suchnorms, it is possible to transparently change strategy to asustained approach and maximise accuracy. We demonstratethat while the low-burden secured consent approach may besufficient for some people, it can not be relied on to maintainaccuracy in most cases.

We acknowledge that our measure of accuracy is not thesole means to determine that informed consent has beensought. This metric allows us to confirm that the partici-pant disclosed the SNS data that they were willing to, whichwe believe is important to establish. It does not, however,investigate whether the participant understands the implica-tions of sharing their data, or the purpose of the research. Inbiomedical studies, consent comprehension tests are com-monly used to determine that participants’ consent is in-formed, but their effectiveness has been questioned (2007).Investigating the wider implications of assessing consent

comprehension is important further work, where again weanticipate contextual integrity could be leveraged. For ex-ample, while we found that our semi-automated approach todetermining consent was appropriate for some people, oth-ers might find it invasive, and striking this ethical balance isa sensitive topic.

We would like to investigate other applications to whichwe can apply our findings. For example, we are interested inapplying contextual integrity to mobile HCI studies whichwould benefit from gaining informed consent in such a waythat reduces the burden on participants. We are also in-terested in exploring other methodological implications ofadopting contextual integrity, for instance studying whetherthe method is liable to introducing bias as in other forms ofinformed consent (Rothstein and Shoben 2013).

AcknowledgmentsThis work was funded by the Engineering and Physical Sci-ences Research Council (EPSRC) through a Doctoral Train-ing Grant.

ReferencesAyalon, O., and Toch, E. 2013. Retrospective privacy: Man-aging longitudinal privacy in online social networks. In Pro-ceedings of the 9th Symposium on Usable Privacy and Se-curity.Barkhuus, L. 2012. The mismeasurement of privacy: usingcontextual integrity to reconsider privacy in HCI. In Pro-ceedings of the SIGCHI Conference on Human Factors inComputing Systems, 367–376.Bauer, L.; Cranor, L. F.; Komanduri, S.; Mazurek, M. L.;Reiter, M. K.; Sleeper, M.; and Ur, B. 2013. The postanachronism: The temporal dimension of Facebook privacy.In Proceedings of the 12th ACM Workshop on Privacy in theElectronic Society, 1–12.boyd, d., and Crawford, K. 2012. Critical questions forbig data. Information, Communication & Society 15(5):662–679.British Psychological Society. 2013. Ethics guidelinesfor Internet-mediated research. INF206/1.2013. Availableonline at http://www.bps.org.uk/publications/policy-and-guidelines/research-guidelines-policy-documents/research-guidelines-poli.Cohn, E., and Larson, E. 2007. Improving participant com-prehension in the informed consent process. Journal of nurs-ing scholarship 39(3):273–280.Getty, E.; Cobb, J.; Gabeler, M.; Nelson, C.; Weng, E.; andHancock, J. 2011. I Said Your Name in an Empty Room:Grieving and Continuing Bonds on Facebook. In Proceed-ings of the SIGCHI Conference on Human Factors in Com-puting Systems, 997–1000.Grodzinsky, F. S., and Tavani, H. T. 2010. Applying the“contextual integrity” model of privacy to personal blogs inthe blogosphere. International Journal of Internet ResearchEthics 3(1):38–47.

186

Grodzinsky, F. S., and Tavani, H. T. 2011. Privacy in“the cloud”: applying Nissenbaum’s theory of contextual in-tegrity. ACM SIGCAS Computers and Society 41(1):38–47.Hill, K. 2014. Facebook Added ’Research’ To User Agree-ment 4 Months After Emotion Manipulation Study. Forbes.http://onforb.es/15DKfGt.Hutton, L., and Henderson, T. 2013. An architecture for eth-ical and privacy-sensitive social network experiments. SIG-METRICS Perform. Eval. Rev. 40(4):90–95.Hutton, L., and Henderson, T. 2015. Towards reproducibil-ity in online social network research. IEEE Transactions onEmerging Topics in Computing. To appear.Jones, E. A., and Janes, J. W. 2010. Anonymity in a worldof digital books: Google Books, privacy, and the freedom toread. Policy & Internet 2(4):42–74.Kramer, A. D. I.; Guillory, J. E.; and Hancock, J. T. 2014.Experimental evidence of massive-scale emotional conta-gion through social networks. Proceedings of the NationalAcademy of Sciences 111(24):8788–8790.Lewis, K.; Kaufman, J.; Gonzalez, M.; Wimmer, A.; andChristakis, N. 2008. Tastes, ties, and time: A new so-cial network dataset using Facebook.com. Social Networks30(4):330–342.Luger, E., and Rodden, T. 2013. An informed view on con-sent for UbiComp. In Proceedings of the ACM internationaljoint conference on Pervasive and Ubiquitous computing,529–538.McNeilly, S.; Hutton, L.; and Henderson, T. 2013. Un-derstanding ethical concerns in social media privacy studies.In Proceedings of the ACM CSCW Workshop on MeasuringNetworked Social Privacy: Qualitative & Quantitative Ap-proaches.Moreno, M. A.; Fost, N. C.; and Christakis, D. A. 2008.Research ethics in the MySpace era. Pediatrics 121(1):157–161.Morrison, A.; McMillan, D.; and Chalmers, M. 2014. Im-proving consent in large scale mobile HCI through person-alised representations of data. In Proceedings of the 8thNordic Conference on Human-Computer Interaction: Fun,Fast, Foundational, 471–480.Neuhaus, F., and Webmoor, T. 2012. Agile ethics for mas-sified research and visualization. Information, Communica-tion & Society 15(1):43–65.Nissenbaum, H. F. 2004. Privacy as contextual integrity.Washington Law Review 79(1):119–157.Powell, S. J.; Linehan, C.; Daley, L.; Garbett, A.; and Law-son, S. 2012. “i can’t get no sleep”: Discussing #insomniaon Twitter. In Proceedings of the SIGCHI Conference onHuman Factors in Computing Systems, 1501–1510.Rothstein, M. A., and Shoben, A. B. 2013. Does consentbias research? The American Journal of Bioethics 13(4):27–37.Sleeper, M.; Balebako, R.; Das, S.; McConahy, A. L.; Wiese,J.; and Cranor, L. F. 2013. The post that wasn’t: Ex-ploring self-censorship on Facebook. In Proceedings of

the ACM Conference on Computer Supported CooperativeWork, 793–802.Solberg, L. 2010. Data mining on Facebook: A free spacefor researchers or an IRB nightmare? University of IllinoisJournal of Law, Technology & Policy 2010(2).Wang, Y.; Norcie, G.; Komanduri, S.; Acquisti, A.; Leon,P. G.; and Cranor, L. F. 2011. “I regretted the minute Ipressed share”: a qualitative study of regrets on Facebook.In Proceedings of the 7th Symposium on Usable Privacy andSecurity.Warrell, J. G., and Jacobsen, M. 2014. Internet researchethics and the policy gap for ethical practice in online re-search settings. Canadian Journal of Higher Education44(1):22–37.Wilson, R. E.; Gosling, S. D.; and Graham, L. T. 2012. Areview of Facebook research in the social sciences. Perspec-tives on Psychological Science 7(3):203–220.Zimmer, M. 2010. “But the data is already public”: on theethics of research in Facebook. Ethics and Information Tech-nology 12(4):313–325.

187