RedditUsability Test

560

Submitted by: Jessica Chambers, Brooke Hundley Sasha Campbell

COM 560 XU Date: 12.6.13

method test results conclusion

search reddit

username password

remember me reset password login

MY SUBREDDITS FRONT • ALL • RANDOM PICS • FUNNY • GAMING • ASKREDDIT• WORLDNEWS • NEWS • VIDEOS • IAMA • TODAYILEARNED • WTF • AWW • TECHNOLOGY • MORE

introduction want to join? login or register in seconds Englishappendices

2

EXECUTIVE SUMMARY

Reddit is a social news aggregation site that allows registered users to submit links and

comments in a variety of categories (SubReddits). Registered users are also able to send

messages and share articles with other users and can customize which SubReddit’s they are

subscribed to. In order to test the functionally and user satisfaction of Reddit, a usability test was

administered to 10 participants who had very little or no previous experience with Reddit. The

selected participants fell within the primary and secondary age demographic for the website and

were all active Internet users (PewResearchCenter). The usability test consisted of two

questionnaires (pre-survey and post-surveys) and nine tasks that were timed. All tests were

screen captured with Quicktime and were later analyzed. The data was broken down into three

categories: number of errors (wrong clicks), time-per-task and whether the tasks were completed.

Furthermore, the information was then compared with the questionnaire results to analyze and

better understand the usability of Reddit.

Primary findings:

• Users were able to complete a majority of the tasks but did not find Reddit to be user

friendly.

• Tasks that were familiar to other websites (i.e. search bar, logging in, etc.) were more

easily achieved than those specific to Reddit.

• Users had trouble locating information and utilizing the main navigation menu.

• Participants had difficulty customizing their Reddit subscriptions in the SubReddits

section.

• Only 40% of users said they would use Reddit again.

3

INTRODUCTION

Reddit is a social news and entertainment, bulletin-style site in which users submit

content (primarily links or comments), and other users vote up or down to rank the post and

establish its position on the home page. Reddit.com is ranked in the top 100 sites used globally

(alexa.com) and in the top 50 used in the United States, and had more than 27 million users each

month (quantcast.com).

According to the PewResearchCenter’s July 2013 report (Appendix A), 6% of all adult

online users visit reddit.com, and the top demographic of users on the site consists of 18-29 year

olds, with an emphasis on male users. The site has a slogan of “the front page of the internet”

and is often used as a news aggregator or a personal bookmark collection. Content is organized

by a series of “subreddits,” otherwise known as categories of user interest. In order to submit

content, users must have a registered account. Registered users can also post comments on any

submission, and exchange information back and forth through these comments. The comments

can then be voted up or down. Reddit has a series of customizable options including a personal

front page in which users can choose which subreddits appear, as well as the ability to “friend”

other redditors, giving the user quicker access to postings and comments on their friend list via

these subscriptions. The community aspect of Reddit extends to a reward system in which

redditors accumulate “karma” (points) for highly rated comments and links. The site also offers

a subscription service known as Reddit Gold, which adds a number of features to the interface,

including the ability to display more comments on a page, exclusive access to the private

subreddit lounge, and notifications whenever the user’s username is mentioned in a comment.

Reddit requires users to create an account in order to access many of the interactive

features of the site, however, users who are not members still have access to the most

4

informational features. Users who create an account have the ability to comment on

articles/links, vote up or down on other users’ comments, share content via email or Reddit

inboxes, and subscribe/unsubscribe to “subreddit” categories of interest. Without an account,

visitors can still navigate to any posting, view the existing comments on any thread, and view the

subreddit categories. A Reddit account unlocks features that allow users to participate in the

Reddit community and leave more feedback. The interface relies solely on user-generated

content, making members part of a collective community that allows Reddit to provide a

thorough and engaging experience.

The creation of a Reddit account does not require a valid email address, although one is

suggested. Instead the user only needs to register a login name and password and answer a

captcha element to verify the user is human. Once the account is created, users can participate in

social interactions on the interface by submitting a “new link” or a “new text post.” Users can

share content with other Reddit users or via email addresses, can save content of interest to a

“saved” section of their account, and can connect with other users by leaving comments on

threads or voting other comments up or down. These up or down votes are aggregated to move

the posting’s rank higher or lower on the Reddit homepage. Reddit.com also provides incentive

for their members to be active on the site. Users are given karma points for voting on links or

comments, as well as leaving comments for others to view. Karma points are used to establish a

member’s position in the community.

Reddit houses a variety of features on their site, but for the purposes of this usability test

we focused on specific tasks that were more integral to the average user’s needs. The search,

commenting, voting, sharing, and customizing subreddit lists were tested. While these functions

do not cover every capability of the site, they are representative of the purpose of the site as a

5

whole. A typical user would frequently utilize these core functions in order to navigate the site,

and the usability of these features would weigh heavily on recruiting and retaining users.

METHODS

Our objective in this usability test was to measure the navigability, efficiency, and user

satisfaction with the Reddit browsing experience. The test was completed with ten participants

consisting of six females, and four males between the ages of 22 and 45. The participants were

chosen via a convenience sample of Elon University students. All but one participant fit into the

largest demographic of Reddit users, which are men and women between the ages of 18-29. The

remaining participant fit into the second highest demographic, that of men and women aged 30-

49. Although the study was limited through a convenience sample of university-educated

individuals, the participants should be considered a relatively accurate representation of Reddit

users. Our pre-survey results revealed that 90% of our participants typically browse the Internet

for more than 3 hours a day, with the remaining 10% still browsing for 1-3 hours a day, thus our

sample population should be considered power users. Although participants are advanced

Internet consumers, 70% had no experience with Reddit.com, and there were no participants

within the remaining 30% that ranked their Reddit experience level higher than a 2 on a scale

from 0 to 5, with “0” representing no experience.

Before our team could administer the usability test, a set of required materials were

developed to ensure consistency in our results. A participant consent form was generated

(Appendix B). A pre-survey and post-survey were created for participants to take before

beginning the test and after exiting the test. Before we could start the test, 10 Reddit accounts

with unique usernames and passwords were created. Quicktime’s screen recording was set up to

capture each participant’s actions and testing time on every testing computer. The usability test

6

was composed of three stages. After participants were read the script detailing the procedure,

they signed a consent form and began the test. During the test, participants were first asked to

take a survey. Once they finished the initial survey, participants worked through the set of tasks

provided. Finally, participants took an exit survey regarding their experience. Administers

created a reward system to entice volunteers to participate which included brownies, cookies and

cupcakes. The aforementioned sustenance was given upon completion of the required tasks and

subsequent exit survey.

Alexa.com data states that people who access Reddit from school are greatly

overrepresented relative to the general Internet population. Given this information researchers

wanted to simulate a collegiate environment by holding the survey in a similar area. Therefore,

researchers held the survey at Elon University, which allowed for a simulated real world exp-

erience. Researchers administered the usability test to participants in groups of four in a leisure

space in an Elon University Building. Each participant had a computer and chair with space

between test takers to minimize distractions. The screen recording software was running when

users arrived so as not to distract the participant.

During the usability test, a set procedure for each of the ten participants was

followed. One researcher read an instructional script to ensure that each participant had the same

understanding of the rules and requirements of the usability test (Appendix C). The script

details that researchers were focused on studying the interface and not the users, and asked the

participants to honestly answer any questions about their experience. To best capture the typical

browsing experience of users, participants were encouraged to complete the entire test without

help, just as they would in a non-test-taking setting.

7

The usability test began with a pre-survey of eight questions hosted on

surveymonkey.com (Appendix D) that gauged our participants experience with Reddit.com, as

well as their Internet consumption, and practice of sharing multimedia content. Out of 10

participants, 70% reported that they typically use the Internet to access news, half reported that

they were “very likely” to like or dislike media on the Internet, and half rated that they were

“very likely” to share media on the Internet.

Once they completed the pre-survey, participants were asked to move onto a set of

provided scenario tasks (Appendix E). The scenarios were designed to illustrate the

effectiveness and navigability of the site’s browsing capabilities. The scenarios are

representative of the context and situations in which participants would explore most of the

available features on Reddit including signing in, searching for a topic, leaving comments, voting

on content, sharing content, and subscribing and unsubscribing to content streams (subreddits).

After completion of the scenario tasks, participants were asked to answer a post-survey

(Appendix F) gauging their ease of use with the site and overall opinions to include future use

and suggested design changes. Once all of the data was collected, the videos of our participants

were recorded noting the tasks completed, errors, and time of each task (Appendix G). The

researchers measured this information and cross-referenced it with the post-survey results in

order to determine overall effectiveness and user satisfaction with the Reddit interface.

RESULTS

The results of the usability test were obtained from screen recordings and the

participants’ post-survey responses (Appendix H). The results are assembled based on averages

amongst all test takers however additional insights were gained from individual experiences and

are mentioned in further detail throughout this results section. While reviewing the videos,

8

results were organized into three different categories: error rate (number of incorrect clicks), time

per task, and if the task was actually completed or not. Researchers established that the error rate

would denote unnecessary clicks to propel the user forward in completing the task. However,

error rate did not include clicks for viewing purposes given that the website’s main function is to

click and view the content.

Nine separate tasks were observed in total including: finding a specified (Thor) article,

signing in to Reddit, adding a comment, rating/voting a comment up or down, going to a

specified (Technology) SubReddit, saving a found article, unsubscribing from SubReddits,

subscribing to SubReddits, and sharing the previously saved article. With 10 participants and

nine categories, approximately 78% of the tasks were completed successfully. Only two

participants were able to complete all nine of the tasks with the lowest success rate at five out of

nine completed tasks. In the post survey, users were asked to rate if they agreed or disagreed

with the question, “I was able to successfully complete all the tasks.” Participants responded,

with 60% feeling they were able to complete all of the tasks.

Actual task completion data revealed that some participants were able to quickly and

efficiently accomplish the tasks, and that three tasks: finding an article on a specific topic,

signing into Reddit, and saving an article, had a perfect success rate of 100% completion. This is

to be expected since search bars and account logins are a part of most websites, and a familiar

part of the web browsing experience. However, while all the participants were able to save an

article, the time to complete this task ranged from 14 seconds to 1 minute and 50 seconds.

Reddit’s interface provides links to content that is hosted on other sites, thus many

participants failed to realize that they had to go back to Reddit to save. Overall, saving did not

have a high number of wrong clicks, but we observed confusion on navigability given the

9

cluttered interface design through viewing our screen recordings and time lapsed

documentation. Despite a great deal of aptitude in completing three of the tasks, a hierarchy of

tasks became evident as two tasks fell under a 50% successful completion rate among all

participants. Rating a comment was only accomplished by 50% of the group and only 40% were

able to go to the SubReddit category. Since the participants revealed at least 60% were ‘Likely’

or ‘Very Likely’ to like/dislike content online, it was surprising to see this had such a low

success rate.

Our observations and screen recordings revealed that some of the participants were

unable to clearly locate the “Up” or “Down” buttons and could not complete this task. It was not

as surprising to see users having difficulty accomplishing our task of going to the technology

SubReddit. From any page, participants had two ways to go to this area; either by visiting the

navigation that ran across the top of the page or utilizing a drop down menu on the top left of

page. Of the four users who completed this task, three used the navigation menu at the top of the

page. The six individuals who failed to compete this task did so with mixed experiences. Two of

the users simply searched for ‘new technology’ or ‘iPhone 6’ in the search bar rather than trying

to locate the SubReddit menu. Several individuals also went to other SubReddits instead. It was

inconclusive just from watching the screen recordings if this was due to an inability to locate the

specified technology SubReddit or simply choosing to disregard the instructions. SubReddit’s are

the main navigation for the website and are difficult to notice. Overall, dealing with the

SubReddit’s was one of the more difficult tasks given.

Eight individuals were able to successfully unsubscribe from different SubReddit’s while

only seven were able to subscribe. This is in part due to confusion on where users should go to

change this information. Many of the errors occurred when users went to their username’s

10

preference section, which doesn’t allow you to customize SubReddits. The navigation bar offers

an edit button that will take a user to their personal subscription page. The navigation bar also

features a drop down navigation area titled ‘My SubReddit’s’ with the last option listed as ‘Edit

Subscriptions.’ Even once users arrived on the edit page, there was confusion on what should be

done. Some participants went to specific SubReddits to try and unsubscribe/subscribe. While this

is a possible solution, 30% of users were not able to locate subscribe and 20% could not locate

unsubscribe. Two users had an extended subscribe time because they looked for specific topics

they were interested in rather than subscribing to the suggested SubReddits on the edit page.

When it came to our final tasks, 80% of the participants were able to locate their saved

article and share it, yet it remained the second most difficult task according to the post-survey

results. This task had the highest error rate and average of 1.25 incorrect clicks. It was clear that

sharing the article was not actually the most difficult part of this portion of the test. Instead,

participants had high difficulty in understanding where to recover their saved articles. One

individual was able to share an article but their response was disregarded because they had been

unable to locate their specific saved article to share. This task was the most time consuming with

the fastest time taking 1 minute and 6 seconds. Overall, the average user took 1 minute and 45

seconds to complete the task.

When completing the post-survey, participants were asked if they would use Reddit

again, 60% of the individuals said they would not. The four individuals who said they would use

Reddit stated they would utilize it to find funny pictures and look at specific topics/SubReddits.

One individual commented they would not use it for, “anything serious.” It is unclear if that

result would have been different if more individuals had been able to locate the technology

SubReddit and explore non-entertainment content. As previously shown, most participants did

11

not like the layout and navigation of the website. Furthermore, when asked what were the most

challenging parts of the user experience with Reddit, 70% of participants wrote “not able to find”

the required links to complete the tasks. Navigability and design were the top reasons why users

said they would not use the site again. While the users were able to complete most Reddit tasks,

overall the non-user friendly interface alienates a majority of its target audience.

CONCLUSION

Reddit’s current website has a large global audience and while current use of the site is

widespread and the interface is deemed suitable, our study showed a significant opportunity for

improvement. Researchers suggest that more thought should be given to navigation and design of

the interface for ease of use. One of the complaints from 50% of users was the small, illegible

font size. This is a simple fix that will give the interface and navigability a boost in usability.

Furthermore, the visual aesthetics of the site should be addressed. The site could be greatly

improved from the muted tones and cluttered, monotone text. Specifically, users had issues

finding the ‘rate a comment’ button and therefore the design and labeling should be altered to

make it more readily noticeable. Moreover, the articles that were archived were not marked

appropriately which gave users difficulty in trying to comment or rate items. Given that articles

of this nature are not interactive in terms of common user interaction they should be separated in

some way. As a whole the interface is poorly designed and gave our participants a reason to

never visit the site again. Therefore, Reddit should consider altering its interface for

functionality and aesthetics to best serve the user and the site.

The study conducted was informative, however in future testing we suggest the following

edits and additions to gather more conclusive data. In future testing of Reddit usability,

researchers suggest a larger quantity of participants, as well as a more diverse demographic from

12

which a various skill level of previous use could be compared. Testing should be conducted in a

more secluded area to simulate a personal computing environment. Furthermore to avoid

distractions, standardized computer settings are advised given that our participants met with

distractions due to the computers being personal in ownership and other programs signaling for

attention. It is also highly advised to obtain a better plan regarding how to time the participants

as they conduct both the tasks and survey. This piece of the usability test was met with some

confusion for researchers to review because some tasks were abandoned or done out of order and

thus it was hard to calculate accurate times. By implementing these suggestions, more

conclusive data can be obtained to develop an accurate depiction of Reddit’s usability.

 

method test results conclusion

Appendix(A-H)

560

Submitted by: Jessica Chambers, Brooke Hundley Sasha Campbell

COM 560 XU Date: 12.6.13

search reddit

username password

remember me reset password login

MY SUBREDDITS FRONT • ALL • RANDOM PICS • FUNNY • GAMING • ASKREDDIT• WORLDNEWS • NEWS • VIDEOS • IAMA • TODAYILEARNED • WTF • AWW • TECHNOLOGY • MORE

introduction want to join? login or register in seconds Englishappendices

www.pewresearch.org

JULY 3, 2013

6% of Online Adults are reddit Users Young men are especially likely to visit the “front page of the internet.”

Maeve Duggan Research Assistant, Pew Internet Project

Aaron Smith Senior Researcher, Pew Internet Project

http://pewinternet.org/Reports/2013/reddit.aspx

FOR FURTHER INFORMATION, CONTACT:

Pew Research Center’s Internet & American Life Project 1615 L St., N.W., Suite 700 Washington, D.C. 20036 Media Inquiries: 202.419.4500

Six percent of online adults are reddit users,1 according to a nationally representative survey by the Pew

Research Center’s Internet & American Life Project. This is our first survey that specifically examines

reddit as a standalone platform, and it finds that young men are especially likely to visit the site. Some

15% of male internet users ages 18-29 say that they use reddit, compared with 5% of women in the

same age range and 8% of men ages 30-49.

Young males are especially likely to use reddit % of internet users in each age/gender grouping who use reddit

Source: Pew Research Center’s Internet & American Life Project Spring Tracking Survey, April 17 – May 19, 2013. N=2,252 adults ages 18+. Interviews were conducted in English and Spanish and on landline and cell phones. The margin of error for results based on all internet users is +/- 2.5 percentage points.

Self-described as the “front page of the internet,” reddit is a bulletin of user-submitted text, links,

photos, and videos. The site hosts active sub-communities on a wide selection of topics, ranging from

world news headlines, to animal GIFs, to fan forums and various niche topics. The popularity and

prominence of material on the site is determined by voting from the reddit community. reddit gained

widespread publicity when President Barack Obama took part in the site’s “Ask Me Anything” series in

August 2012, and again in April 2013 when the site was used as a collaboration space for amateur

sleuths attempting to identify the perpetrators of the Boston Marathon bombings.

1 In a separate survey in December 2012, we asked about the use of a number of other social media platforms. The

proportion of online adults who use reddit is similar to the proportion that use Tumblr (also 6%), and around half the number that use Twitter (16%), Pinterest (15%), or Instagram (13%). Some 67% of online adults are Facebook users. For more information on these sites please read our report, The Demographics of Social Media Users -2012.

15%

8%

3% 3% 5% 5%

2% 1%

0%

10%

20%

30%

18-29 30-49 50-64 65+

Men Women

Overall, men are twice as likely as women to be reddit users, those under 50 are significantly more likely

to use reddit than those 50 or older, and the site is much more common among urban and suburban

residents than among those living in rural areas. Indeed, just 2% of internet users ages 50 and older—

and 2% of rural residents—say they use the site.

reddit usage by demographic group % of internet users in each group who use reddit

% who use reddit

All internet users (n=1,895) 6%

a Men (n=874) 8b

b Women (n=1,021) 4

Race/ethnicity

a White, Non-Hispanic (n=1,331) 5

b Black, Non-Hispanic (n=207) 4

c Hispanic (n=196) 11ab

Age

a 18-29 (n=395) 11bcd

b 30-49 (n=542) 7cd

c 50-64 (n=553) 2

d 65+ (n=356) 2

Education attainment

a No high school diploma (n=99) 9

b High school grad (n=473) 4

c Some College (n=517) 6

d College + (n=790) 7b

Household income

a Less than $30,000/yr (n=417) 6

b $30,000-$49,999 (n=320) 6

c $50,000-$74,999 (n=279) 7

d $75,000+ (n=559) 6

Urbanity

a Urban (n=649) 7c

b Suburban (n=893) 6c

c Rural (n=351) 2

Source: Pew Research Center’s Internet & American Life Project Spring Tracking Survey, April 17 – May 19, 2013. N=2,252 adults ages 18+. Interviews were conducted in English and Spanish and on landline and cell phones. The margin of error for results based on all internet users is +/- 2.5 percentage points. Note: Percentages marked with a superscript letter (e.g.,

a) indicate a statistically

significant difference between that row and the row designated by that superscript letter, among categories of each demographic characteristic (e.g. age).

More about reddit

reddit features a user-voting system (called up- and down-voting) to rank posts, comments, and links

within its sub-communities (known as subreddits). Subreddits and posts can range from the serious to

the absurd, and from general topics to niche interests. The front page of reddit features popular

submissions from across the site and typically rotates top posts regularly throughout the day. People

who use reddit are called redditors.

reddit is often pivotal in the spread of popular memes, like Ridiculously Photogenic Guy, Overly

Attached Girlfriend, and Sudden Clarity Clarence.

One of the most popular subreddits, IamA (“I am a”) features a popular question forum called Ask Me

Anything (AMA). redditors with “something uncommon that plays a central role in your life” or who

have experienced “a truly interesting and unique event” host a session with the prompt “I am a…..ask

me anything.” Other redditors can then submit questions and comments. Examples vary from “IAmA

prison guard at a maximum security prison, AMA” to “IAmA wildlife cameraman. AMA” and “IAmA

Liberal Saudi Woman Living in Riyadh, Saudi Arabia. AMA!” Celebrities have also participated in AMA,

including Stephen Colbert, Judy Blume, Paul Krugman, and Bill Gates.

For a more in-depth overview of the site, visit Mashable’s “Reddit for Beginners” or poke around the site

at reddit.com.

Survey Questions

Spring 2013 Tracking Survey Final Topline 5/21/2013

Data for April 17-May 19, 2013

Princeton Survey Research Associates International for

the Pew Research Center’s Internet & American Life Project

Sample: n=2,252 national adults, age 18 and older, including 1,127 cell phone interviews Interviewing dates: 04.17.2013 – 05.19.2013

Margin of error is plus or minus 2.3 percentage points for results based on Total [n=2,252] Margin of error is plus or minus 2.5 percentage points for results based on all internet users [n=1,895]

INTUSE Do you use the internet, at least occasionally?

EMLOCC Do you send or receive email, at least occasionally?

EMINUSE Do you use the internet or email, at least occasionally?

INTMOB Do you access the internet on a cell phone, tablet or other mobile handheld device, at least occasionally?2

USES INTERNET

DOES NOT USE INTERNET

Current 85 15

December 2012 81 19

November 2012 85 15

September 2012 81 19

August 2012 85 15

April 2012 82 18

February 2012 80 20 INTUSE/EMLOCC/EMINUSE/INTMOB continued...

2 The definition of an internet user varies from survey to survey. In the current survey, half the sample was asked

INTUSE/EMLOCC/INTMOB and half was asked EMINUSE/INTMOB. Current results are for both forms combined. Throughout the current topline, total internet users are defined as those who answered yes to any question INTUSE, EMLOCC, EMINUSE or INTMOB. From April 2012 thru December 2012, an internet user is someone who uses the internet at least occasionally, sends/receives email at least occasionally or accesses the internet a mobile device (three-part definition with question wording “Do you use the internet, at least occasionally?” OR “Do you send or receive email, at least occasionally?” OR “Do you access the internet on a cell phone, tablet or other mobile handheld device, at least occasionally?”). From January 2005 thru February 2012, an internet user is someone who uses the internet at least occasionally or sends/receives email at least occasionally (two-part definition with question wording “Do you use the internet, at least occasionally?” OR “Do you send or receive email, at least occasionally?”). Prior to January 2005, an internet user is someone who goes online to access the internet or to send and receive email (question wording “Do you ever go online to access the Internet or World Wide Web or to send and receive email?”).

INTUSE/EMLOCC/ EMINUSE/INTMOB continued...

USES INTERNET

DOES NOT USE

INTERNET

December 2011 82 18

August 2011 78 22

May 2011 78 22

January 2011 79 21

December 2010 77 23

November 2010 74 26

September 2010 74 26

May 2010 79 21

January 2010 75 25

December 2009 74 26

September 2009 77 23

April 2009 79 21

December 2008 74 26

November 2008 74 26

August 2008 75 25

July 2008 77 23

May 2008 73 27

April 2008 73 27

January 2008 70 30

December 2007 75 25

September 2007 73 27

February 2007 71 29

December 2006 70 30

November 2006 68 32

August 2006 70 30

April 2006 73 27

February 2006 73 27

December 2005 66 34

September 2005 72 28

June 2005 68 32

February 2005 67 33

January 2005 66 34

Nov 23-30, 2004 59 41

November 2004 61 39

July 2004 67 33

June 2004 63 37

March 2004 69 31

February 2004 63 37

November 2003 64 36

August 2003 63 37

June 2003 62 38

May 2003 63 37

March 3-11, 2003 62 38

February 2003 64 36 INTUSE/EMLOCC/EMINUSE/INTMOB continued...

INTUSE/EMLOCC/ EMINUSE/INTMOB continued...

December 2002 57 43

November 2002 61 39

October 2002 59 41

September 2002 61 39

July 2002 59 41

March/May 2002 58 42

January 2002 61 39

December 2001 58 42

November 2001 58 42

October 2001 56 44

September 2001 55 45

August 2001 59 41

February 2001 53 47

December 2000 59 41

November 2000 53 47

October 2000 52 48

September 2000 50 50

August 2000 49 51

June 2000 47 53

May 2000 48 52

WEB1-A Next... Please tell me if you ever use the internet to do any of the following things. Do you ever use the internet to...[INSERT ITEM; RANDOMIZE]?

Based on all internet users [N=1,895]

TOTAL HAVE EVER DONE

THIS

---------- DID

YESTERDAY

HAVE NOT

DONE THIS DON’T KNOW REFUSED

Use Reddit

Current 6 n/a 90 4 *

Methods This report is based on the findings of a survey on Americans' use of the Internet. The results in this

report are based on data from telephone interviews conducted by Princeton Survey Research Associates

International from April 17 to May 19, 2013, among a sample of 2,252 adults, age 18 and older.

Telephone interviews were conducted in English and Spanish by landline (1,125) and cell phone (1,127,

including 571 without a landline phone). For results based on the total sample, one can say with 95%

confidence that the error attributable to sampling is plus or minus 2.3 percentage points. For results

based on Internet users3 (n=1,895), the margin of sampling error is plus or minus 2.5 percentage points.

In addition to sampling error, question wording and practical difficulties in conducting telephone surveys

may introduce some error or bias into the findings of opinion polls.

A combination of landline and cellular random digit dial (RDD) samples was used to represent all adults

in the United States who have access to either a landline or cellular telephone. Both samples were

provided by Survey Sampling International, LLC (SSI) according to PSRAI specifications. Numbers for the

landline sample were drawn with equal probabilities from active blocks (area code + exchange + two-

digit block number) that contained three or more residential directory listings. The cellular sample was

not list-assisted, but was drawn through a systematic sampling from dedicated wireless 100-blocks and

shared service 100-blocks with no directory-listed landline numbers.

New sample was released daily and was kept in the field for at least five days. The sample was released

in replicates, which are representative subsamples of the larger population. This ensures that complete

call procedures were followed for the entire sample. At least 7 attempts were made to complete an

interview at a sampled telephone number. The calls were staggered over times of day and days of the

week to maximize the chances of making contact with a potential respondent. Each number received at

least one daytime call in an attempt to find someone available. For the landline sample, interviewers

asked to speak with the youngest adult male or female currently at home based on a random rotation. If

no male/female was available, interviewers asked to speak with the youngest adult of the other gender.

For the cellular sample, interviews were conducted with the person who answered the phone.

Interviewers verified that the person was an adult and in a safe place before administering the survey.

Cellular sample respondents were offered a post-paid cash incentive for their participation. All

interviews completed on any given day were considered to be the final sample for that day.

Weighting is generally used in survey analysis to compensate for sample designs and patterns of non-

response that might bias results. A two-stage weighting procedure was used to weight this dual-frame

sample. The first-stage corrected for different probabilities of selection associated with the number of

adults in each household and each respondent’s telephone usage patterns.4 This weighting also adjusts

for the overlapping landline and cell sample frames and the relative sizes of each frame and each

sample.

3 Internet user definition includes those who use the internet or email at least occasionally or access the internet

on a mobile handheld device at least occasionally. 4 i.e., whether respondents have only a landline telephone, only a cell phone, or both kinds of telephone.

The second stage of weighting balances sample demographics to population parameters. The sample is

balanced to match national population parameters for sex, age, education, race, Hispanic origin, region

(U.S. Census definitions), population density, and telephone usage. The Hispanic origin was split out

based on nativity; U.S born and non-U.S. born. The basic weighting parameters came from the US

Census Bureau’s 2011 American Community Survey data. The population density parameter was derived

from Census 2010 data. The telephone usage parameter came from an analysis of the January-June

2012 National Health Interview Survey.

Following is the full disposition of all sampled telephone numbers:

Sample Disposition

Landline Cell

41,291 24,698 Total Numbers Dialed

1,755 411 Non-residential

1,516 88 Computer/Fax

12 ---- Cell phone

24,344 9,674 Other not working

2,038 226 Additional projected not working

11,626 14,299 Working numbers

28.2% 57.9% Working Rate

679 75 No Answer / Busy

3,442 3,668 Voice Mail

41 16 Other Non-Contact

7,464 10,540 Contacted numbers

64.2% 73.7% Contact Rate

450 1,537 Callback

5,786 7,097 Refusal

1,228 1,906 Cooperating numbers

16.5% 18.1% Cooperation Rate

45 68 Language Barrier

---- 684 Child's cell phone

1,183 1,154 Eligible numbers

96.3% 60.5% Eligibility Rate

58 27 Break-off

1,125 1,127 Completes

95.1% 97.7% Completion Rate

10.0% 13.0% Response Rate

The disposition reports all of the sampled telephone numbers ever dialed from the original telephone

number samples. The response rate estimates the fraction of all eligible respondents in the sample that

were ultimately interviewed. At PSRAI it is calculated by taking the product of three component rates:

Contact rate – the proportion of working numbers where a request for interview was made

Cooperation rate – the proportion of contacted numbers where a consent for interview was at

least initially obtained, versus those refused

Completion rate – the proportion of initially cooperating and eligible interviews that were

completed

Thus the response rate for the landline sample was 10 percent. The response rate for the cellular sample

was 13 percent.

Appendix B: Informed Consent Form Informed Consent Form Elon University  Title of Project:  COM560A Group Research Projects for Class  Principal Investigator:  Qian Xu 

Assistant Professor School of Communications, Elon University 2850 Campus Box Office: (336) 278­6454 Email: qxu@elon.edu Co­investigator: Brooke Hundley 

 1. Purpose of the study: The purpose of this study is to gather information about Elon students’ and Elon community members’ uses and evaluation on a number of various media interfaces in their daily life. The study has been designed as part of a graduate course project in interactive media strategies (COM560A) and will therefore reflect a number of different research questions being examines by multiple subgroups of students. This project is designed to give practical experience to students in usability test for communication media. This project will look into the uses and effects of communication media on users.  2. Procedures to be followed: You will be asked to first interact with some media and then answer a few questions raised by the student researchers of fill out an questionnaire.  3. Discomforts and Risks: There are no risks in participating in this research beyond those experienced in everyday life.  4. Benefits: There are two potential benefits to participation: (a) You might learn more about yourself by participating in this study. You might have a better understanding of how important communication is to you: and (b) this research might provide a better understanding of how use of media affects college students. This information could help plan programs and make students services better.  5. Duration: It will take about 30 minutes to complete the study.  6. Statement of confidentiality: Only the person in charge and the student researchers will know your identity. All data related to this study will only be accessible to the principal investigator and the student investigators for this study and will be kept in locked closets and/or secured computers of the investigator and student investigator desks, in Elon university offices.  7. Right to ask questions: You can ask questions about this research. Contact Qian Xu at 336­278­6454 with questions. You can also call this number if you have concerns about this research, or if you feel that you have encountered any problem in the study.  8. Compensation: There is no compensation involved in this study. 

 9. Voluntary participation: You do not have to participate in this research. You can end your participation at any time by telling the person in charge. You do not have to answer any questions you do not want to answer. Refusal to take part in or withdrawing from this study will involve no penalty or loss of benefits you would receive otherwise.  10. You must be 18 years of age or older to consent to participate in this research study.  If you consent to participate in this research study and to the terms above, please sign your name and indicate the date below.  You will be given a copy of this consent form to keep for your records.  ________________________________________________     _____________ Participant Signature  Date  _________________________________________________     _____________ Investigator Signature  Date                             

Appendix C: Script  Thank you for agreeing to take part in our study.  My name is _______________.  For your preliminary instructions, I’ll be working from a script to ensure that my instructions to everyone who participates in the study are the same.  I am here to learn about how people use the Urbanspoon website.  When you sit down as your work station there will be two sheets of paper in front of you.  First, please to sign the consent form.  The second sheet will be a copy of your instructions.  The instructions will give you directions to complete a short initial survey, scenarios and tasks on Reddit, and end with a final survey about your experience.  As you try to complete each task, please try to do whatever you would normally do in the given situation.  Please know that we are not testing you, and there is no such thing as a wrong answer.  Your doing this helps us understand what works and doesn’t work about the site.  Should you reach a point in any given task where you feel you would normally stop using the site or use a different site, please just move on to the next task on the list.  We want this to simulate your normal browsing habits under the given circumstances.  And in an effort to simulate a normal browsing situation, I will be unable to give you any assistance with the site.  I am an independent researcher who had nothing to do with the design of the site you are about to try out.  So please be honest in your feedback ­ I need to know exactly what you think, not what you think I want to hear.  The whole session will take about 30 minutes. Do you have any questions before we start?                   

Appendix D: Pre­Survey Survey was administered on SurveyMonkey.com.  

1. What is your username?  

2. How often do you typically browse the Internet?  Never A few times a month A few times a week 1­3 hours a day More than 3 hours a day Always connected  

3. How often do you typically share multimedia content?  Never A few times a month A few times a week  1­3 hours a day More than 3 hours a day Always connected   

4. How often do you typically use the Internet to access news? Never Every other month Every other week Every other day Everyday 

 5. Rate your level of experience with Reddit?  

No Experience        Expert 0       1          2    3 4 5  

6. Rate your likelihood to share images/videos/media? Not likely at all                         Very likely 1       2           3    4         5  

7. Rate your likelihood to comment on media on the Internet? Not likely at all             Very likely 1       2           3    4         5  

8. Rate your likelihood to like or dislike media on the Internet?  Not likely at all             Very likely 1       2           3    4         5    

Appendix E: Tasks  1. Please take the pre­survey questionnaire on Survey Monkey.  2. Please read the following scenarios and complete the tasks to the best of your ability. If you cannot complete a task, please just move on to a new task.  3. A friend has been talking about a social site called Reddit. He mentioned reading an article about the new Thor movie. See if you can find anything on this topic. Once you find an article, sign in and add a comment. 

a. Username (To be provided). b. Password (To be provided). 

 4. After commenting, review some of the other comments on this article. Pick a comment and vote.  5. You want to explore some of the other areas of Reddit and are interested in new technology. Find the technology category (i.e. SubReddit) and find a post you like. You don't have a lot of time to read all the comments and want to visit it later. Save the article so you can view it later.  6. Some of the SubReddit's you are subscribed to are not that interesting. Customize your SubReddit display by unsubscribing from two categories and subscribing to two of your choices.  a. Please list the two SubReddit's you unsubscribe from: ______________________  b. Please list the two SubReddit's you subscribed to: ________________  7. Before you go, you want to share the thread you previously saved. After locating the saved article, find a way to send it to your friend Reddit user Bhundle1.  8. Please take the post­survey questionnaire on Survey Monkey about your experience using Reddit.   Thank you for your time and participation.       

Appendix F: Post­Survey Survey was administered on SurveyMonkey.com.   Rate the following: 1. I was able to successfully complete all the tasks. Strongly Disagree     Strongly Agree 

1          2    3 4 5  2.  I found the site easy to navigate. Strongly Disagree     Strongly Agree 

1          2    3 4 5  3.  I was able to quickly find the topics and articles I was looking for. Strongly Disagree     Strongly Agree 

1          2    3 4 5  4.  What tasks did you consider the most challenging part of your experience with Reddit today?  5. How would you use Reddit in the future?  If you would not use again, respond “would not use” and list the reasons why.  6.  If any, what changes would you like to see implemented in the site to increase the user experience?  7. What username did you use?  8.  What is your age?  9. Select your gender: 

Male Female Other 

            

Error  RateFind  Thor  Article Sign  In Comment Rate  Comment

1 0 0 0 X2 0 0 0 03 3 0 X X4 0 0 2 X5 1 0 2 16 0 0 0 07 0 0 0 X8 0 0 X 09 0 0 0 X0 2 0 0 0

Time-­‐Per-­‐TaskFind  Thor  Article Sign  In Comment Rate  Comment

1 :30 :28 :51 X2 :44   :45 :10 :193 1:06 2 2:10 X4 :47   :24 1:25 X5 1:10 :31 1 X6 :45 1:02 :32 :487 :43 :17 :30 X8 :26 :18 X :559 :27 :31 :19 :07

0 :49 :42 :27 :23

APPENDIX  G

Task  Competion  (Y/N)  Find  Thor  Article Sign  In Comment Rate  Comment

1 YES YES YES NO2 YES YES YES YES3 YES YES NO NO4 YES YES YES NO5 YES YES YES NO6 YES YES YES YES7 YES YES YES NO8 YES YES NO YES9 YES YES YES YES0 YES YES YES YES

Go  to  Technology  SubReddit Save Unsubscribe Subscribe Share  ArticleX 1 0 0 20 0 1 0 10 0 1 0 2X 0 x x 21 X 0 0 10 1 0 X 1X 1 3 X 10 0 0 0 1X 0 X X 00 0 0 4 X

Go  to  Technology  SubReddit Save Unsubscribe Subscribe Share  ArticleX 1:50 1:39 3:26 2:14:18 :14 1:22 :30 1:06:05 :20 :26 :43 1:43X :24 X X 3:06X 0.57 1:05 1:01 1:33:37 1:06 1:17 :32 1:14X 1:01 3:44 X X:33 :20 :47 :39 1:53X 1:29 X X 1:18

X 1:14 :32 X X

Go  to  Technology  SubReddit Save Unsubscribe Subscribe Share  ArticleNO YES YES YES YESYES YES YES YES YESYES YES YES YES YESNO YES NO NO YESNO YES YES YES YESYES YES YES YES YESNO YES YES YES NOYES YES YES YES YESNO YES NO NO YESNO YES YES NO NO

Appendix H: Results  Pre­Survey Responses Survey was administered on SurveyMonkey.com.   1.  What is your username? 

Elontest8 Elontest1 ElonUsability2 ElonTest7 elontest10 Elontest6 ElonUsability3 ElonUsability4 ElonTest9 ElonUsability5 

2.  How often do you typically browse the Internet?  0% ­ Never 0% ­ A few times a month 0% ­ A few times a week 10% ­ 1­3 hours a day 60% ­ More than 3 hours a day 30% ­ Always connected 

 3.  How often do you typically share multimedia content?  

0% ­ Never 0% ­ A few times a month 50% ­ A few times a week  50% ­ 1­3 hours a day 0% ­ More than 3 hours a day 0% ­ Always connected  

 4.  How often do you typically use the Internet to access news? 

10% ­ Never 0% ­ Every other month 0% ­ Every other week 20% ­ Every other day 70% ­ Everyday 

 5.  Rate your level of experience with Reddit?  

70% ­ No Experience (0) 20% ­ 1 10% ­ 2 0% ­ 3 0% ­ 4 

0% ­ Expert (5)   

6  Rate your likelihood to share images/videos/media? 0% ­ Not at all likely (1) 20% ­ 2 10% ­ 3 20% ­ 4 50% ­ Very Likely (5) 

 7.  Rate your likelihood to comment on media on the Internet? 

30% ­ Not at all likely (1) 0% ­ 2 30% ­ 3 20% ­ 4 20% ­ Very Likely (5) 

  

8.  Rate your likelihood to like or dislike media on the Internet?  10% ­ Not at all likely (1) 20% ­ 2 10% ­ 3 10% ­ 4 50% ­ Very Likely (5) 

  

Post­Survey Responses  1. I was able to successfully complete all the tasks. 

10% ­ Strongly Disagree (1) 20% ­ 2 10% ­ 3 40% ­ 4 20% ­ Strongly Agree (5) 

  2.  I found the site easy to navigate. 

50% ­ Strongly Disagree (1) 10% ­ 2 40% ­ 3 0% ­ 4 0% ­ Strongly Agree (5) 

  3.  I was able to quickly find the topics and articles I was looking for. 

0% ­ Strongly Disagree (1) 

20% ­ 2 50% ­ 3 10% ­ 4 20% ­ Strongly Agree (5) 

  4. What tasks did you consider the most challenging part of your experience with Reddit today?  “Finding an article about Thor that wasn't inappropriate.” “Finding the saved reddit” “Finding the bookmarked link.” “Sharing with another user ­ I'm not sure if I typed it in correctly. could have easily shared it with an email address, but wasn't sure what to type for a user.” “I actually had trouble loggin in before clicking an article and finding the sub headings.” “Unsubscribing from the subreddits. I couldn't find the "unsubscribe" button” “Commenting on an article” “Unsubscribing from the two categories” “The location of the subredditt” “Knowing where the important buttons were. The button to find a saved item was hard. I also didn't know what SubReddits are or their purpose”   5.  How would you use Reddit in the future?  If you would not use again, respond “would not use” and list the reasons why.  “Would not use” “Would not use because it seems to me that site has no purposes” “Would not use” “Find funny article” “I would use it to find funny stuff, but probably not for anything serious.” “I don't find the website experience necessary in the future, and it's not navigation friendly to me.” “Maybe to search for interesting pictures, some of the subreddit topics seemed like things I would enjoy” “To follow game­related subreddits” “Never again” “Would not use. Horrible interface, too much clutter.”   6.  If any, what changes would you like to see implemented in the site to increase the user experience?  “Cleaner layout, make it look less like a spreadsheet” “Better navigation” “A larger, easier to use nav” 

“It's a pretty boring website ­ looks wise. it'd be nice for it to look cleaner. also the navigation was pretty tough.” “The SubReddit's should be displayed differently and have larger font. Especially if these are headings that are personal to the user.” “Make it not so condensed and cluttered. It's too hard to navigate right now and the text is too small.” “Make the site look nice” “More intuitive” “Bigger font size and the way the site is used” “Easy navigability. everything is so small!! make the font and important things bigger”   7. What username did you use? 

Elontest8 Elontest1 ElonUsability2 ElonTest7 elontest10 Elontest6 ElonUsability3 ElonUsability4 ElonTest9 ElonUsability5 

 8.  What is your age?  

22 24 45 23 26 27 22 26 29 23 

  9. Select your gender: 

40% ­ Male 60% ­ Female 0% ­ Other