website usability testing report · usability metrics were gathered to assess the usability of each...

Website Usability Testing ReportDecember 2016

worldanimal.net

Lauren ThomasMSIS Graduate StudentUniversity of Texas at AustinSchool of Information

Table of Contents

Executive Summary 3

Overview 4

Background & Goals 5

Test Protocol 6

Usability Metrics 7

Task Results & Redesign Ideas 9

Task 1: Searching the Directory 10

Task 2: Sharing Organization Information 16

Task 3: Reporting a Problem 23

Task 4: Finding an International Organization 27

Task 5: Registering an Organization 30

Task 6: Updating Organization Info 36

Overall Analysis 42

Metrics Analysis 43

Post-Test Questionnaire 47

Site-wide Redesign Ideas 48

Appendix 49

WAN Usability Report | December 2016

Executive Summary

WAN Usability Report | December 2016

3

The World Animal Net website (worldanimal.net) is an important online hub for animal protection societies from all over the globe. Easy navigation of its materials, online directory, and user account functions is crucial to its mission.

In August 2016, WAN requested volunteer help to identify ways to improve the website to better serve users. A usability study was planned and an IRB waiver was obtained from the University of Texas at Austin. Eight representative volunteers were recruited to participate in remote, moderated usability testing sessions. Each participant was asked to complete a set of six common tasks to uncover usability issues. They were also asked think aloud and to complete questionnaires about their experiences.

Usability metrics were gathered to assess the usability of each task and highlight the most problematic areas of the site. Common errors and issues were revealed and used to inform a set specific redesign recommendations for the user interface and information architecture.

This report includes an in-depth analysis of those usability metrics and issues and offers a plan of action to improve the user experience on worldanimal.net. The Appendix includes full documentation of the usability study so that it can be replicated in the future to assess the effectiveness of redesign efforts.

OVERVIEW

4

OVERVIEW

Background & Goals

World Animal Net (WAN) is a 501(c)3 non-profit organization established to improve communication and coordination among the world’s animal protection groups. Volunteers maintain an extensive directory of 17,000 organizations in over 130 countries on the organization’s website, WorldAnimal.net. This directory allows activists and animal protection groups to find and connect with others in order to increase the efficacy of their activities. Maintaining an accurate, easy-to-use, and searchable database is central to the organization’s efforts.

?Business Questions

This study seeks to uncover usability issues on WAN’s online directory through moderated remote usability test sessions with a representative participant sample. The study will culminate in a list of specific redesign recommendations of the information architecture and user interface. These will improve the user experience of the directory and help groups more easily, efficiently, and enjoyably connect and coordinate to protect animals all over the world.

Goals

What usability issues do users experience when finding organizations in the directory?

What usability issues do users experience when creating and managing organization listings in the directory?

What do users like and dislike about the directory and other parts of the website?

5

OVERVIEW

Test Protocol

This was a moderated, remote usability study. Participants logged on to an online conference room on a non-mobile device on join.me. A recording of both their screen and audio was captured. Each session lasted between 30 minutes to 1 hour.

Participants were asked to complete a set of 6 tasks. The moderator read the tasks aloud and instructed the participant when to begin and prompted them to think aloud as they went through a task to reveal their thoughts, emotions, likes and dislikes.

After a participant was finished with each task, they were asked to answer a set of 2 post-task questions: Whether they completed the task, and how easy they thought the task was (1=very difficult, 5=very easy).

After a participant finished all of the tasks, they were asked to complete a post-test questionnaire about their overall experience using the site. These included free-text questions and Likert scales.

8 participants were recruited for this study. They each completed a screener questionnaire and a consent form. They were recruited over social media postings and email. Each used a Chrome browser and indicated interest in volunteering to help animals.

See Appendix for full test protocol information and documents.

?

6

OVERVIEW

Usability Metrics

MEAN TIME ON TASK

How long users who completed the task spent from start to finish, in seconds.

*Because participants were instructed to think aloud while completing the task, these scores may be slower than in real-life user scenarios.

EASE OF USE RATING(1 = very difficult, 5 = very easy)

How “easy” users rated a task in the post-task questionnaire.

% COMPLETION RATE(of users who attempted)

Percentage of users who successfully completed a task and answered that they completed the task in the post-task questionnaire.

Error-free Rate

The metrics included below were recorded for each individual task. These help us understand how user-friendly each task is and they also provide us with benchmarks with which we can compare the results of future usability tests. That way, we can see if our redesign efforts are actually improving the user experience.

This is the percentage of users who completed the task without experiencing any errors. Errors include actions that result in an incomplete or incorrect outcome (critical errors), or actions that produce a less efficient result (non-critical errors). These might include opening the wrong menu, clicking the wrong link, or entering incorrect inputs into form fields. Verbal complaints that don’t reflect a compromise of efficiency are not recorded as errors, but rather ‘pain points.’

Quotes

We include illustrative quotes from participants gathered as they thought aloud while completing the task.

7

OVERVIEW

Usability Metrics (continued)

Error Severity Ratings

!

High

Medium

Low

For each task, an analysis of the most common critical errors, non-critical errors, and pain points are included. They are rated by severity depending on how much they affect the user experience. These are used to help us understand how we can redesign the site for improved user experience.

Error Analysis

These are the most important errors to address. These include critical errors, as well as non-critical errors and especially frustrating pain points that are experienced by a high number of users.

These include non-critical errors and pain points that may be encountered by fewer users but still compromise efficiency and significantly impact the user experience.

These include non-critical errors and pain points that are experienced by fewer users and have a lesser impact on user experience.

8

Task Results

& Redesign Ideas

9

RESULTS | T A S K 1

Searching the Directory

3.4EASE OF USE RATING(1 = very difficult, 5 = very easy)

WAN’s worldwide directory of animal organizations allows activists to find and connect with groups of interest. Users need to be able to efficiently find relevant groups according to a variety of criteria.

Why does it matter?

Users were asked to find a bat-focused group in another U.S. state.

118sMEAN TIME ON TASK

87%COMPLETION RATEof 8 users who attempted

Error-Free Rate

"I'm not sure what the World Animal Net Directory is, but it kind of looks like where I want to be.“

"Usually I look for search in the upper right of the page.“

"I skipped over the choice initially because the link was not highlighted in blue."

25%

10

of users completed the task without performing an error.

RESULTS | T A S K 1 (continued)

Task 1 Error Analysis

100%

Issue #1: Users had trouble figuring out

where to search for organizations.

of users looked under ‘Programs’ and ‘Resources’ in the top menu before trying the Directory. This is a clear issue but wasn’t counted as an error unless the user clicked on one of them.

25%of users unsuccessfully used the site search feature instead of or before trying the directory.

Analysis:

Users are having trouble understanding the purpose of the Directory, and not immediately recognizing it as a way to find organizations. Some hovered over it and decided it wasn’t what they wanted. Their instinct is to look for a function titled ‘Search’, and they start looking for this in the upper-right of the page.

11


Task 1 Error Analysis (continued)

25%

Issue #2: On the Directory search results

page, users are falsely interpreting groups as

inactive.

of users skipped over a correct solution to Task 1 because they thought that the group was inactive, commenting that this was because the name was not a blue hyperlink like other groups.

Analysis:

The way information is presented in the Directory search results page is confusing to some users. Some listings have a blue hyperlink for the name, and others don’t. In reality, a lack of a hyperlink means that the group didn’t provide a website link. But this inconsistency and gray text is leading some users to mistakenly believe that groups are inactive.

!

12



50%

Issue #3: When filtering their criteria on the

Directory search page, users were disappointed

not to be able to filter as desired.

of users looked for a bat-related option in the Select Focus or Activity dropdown menu and expressed dismay when they couldn’t find one.

Analysis:

When users couldn’t find an appropriate option in the dropdown, they opted to use the keyword search. Of course it is impossible to include every single kind of category in this dropdown, but their instinct was to try and find one first and this resulted in a loss of efficiency for half of our users.

13


Task 1 Redesign Ideas

1

Add a large button below the top menu and label it ‘Search for Organizations.’

This will give users a clearer path to the Directory. Even when users see ‘Directory’ in the top menu, they don’t connect that word with what they want to do.

Issue #1

Search for Organizations

14


Task 1 Redesign Ideas (continued)

2

Add buttons or explicit links inside organization previews on the search results page. They should be labeled ‘Go to website’ or ‘More information’.

Remove the ‘Read more’ link and the hyperlink functionality from the title.

This will help users understand where they are going when they click a link.

Issue #2

Visit WebsiteMore info

15

RESULTS | T A S K 2

Sharing Organization Information


Users need a way to share and spread the word about relevant organizations they find in the WAN directory, as well as save their contact information for later use.

Why does it matter?

Users were asked to email info about a group using only the WAN site.



Error-Free Rate

50%

"I would have just copied the information and sent it from my personal email, or saved it in a word document.“

"I don't like how I have to scroll down to view the send button.“

"I'm not sure what they want by saying 'Sender.'"

16

of users completed the task without performing an error.



38%

Issue #1: Users have trouble figuring out how

to email themselves information.

of users scrolled and looked at various other parts of the page in search of an email function, even though they started off with it in the viewport. Not counted as an error unless user clicked on something.

1user was unable to complete the task because they could not find the email functionality; a critical error.

Analysis:

Some users did not immediately notice or see the email icon or did not immediately recognize it as the functionality they wanted to complete the task.

17



1

Issue #2: User could not successfully submit

the form.

user received an error message when they tried to send an email even though all fields were appropriately filled out. This was a critical error that prevented them from completing the task.

Analysis:

Even though only one user experienced this problem, it should still be addressed because it is a system-side critical error that completely prevents users from completing the task.

*I am not including a screenshot of the form fields entered because the Participant entered their own and we need to protect their privacy. They did enter valid inputs for all fields.

18



25%

Issue #3: Users were confused about what to

put in certain fields on the email form.

users put in the wrong kind of information (name or email address) in the ‘Sender’ or ‘Email to’ fields, but recognized their mistakes and fixed them. Counted as a non-critical error.

Analysis:

It was unclear to some users what kind of input (name or email) to input into some of the fields in the email form. Although they were able to quickly recognize and fix their mistake, this still resulted in frustration and a loss of efficiency in completing the task.

!

19



50%

Issue #4: Users indicated that this wasn’t a

useful feature for them.

of users commented that they would not use the email feature in real-life, but would copy and paste the info into their own email or Word document.

Analysis:

The email feature is not something that many of our users would have suspected was available, and they already have more intuitive habits for storing information for their own use.

20



1

Replace the email button with social share buttons on each organization’s page. These can include email, but also platforms like Facebook.

Users indicated that email alone might not be very helpful to them. Social share buttons wouldallow users to share organization info with a much wider audience. It would also help get the word about World Animal Net network organizations out there.

*WAN already has these implemented on various other pages of the site!

Issues #1 and #4

21



2

Add placeholders or more precise labels to the fields in the report submission form. This will help users understand exactly what kind of inputs are expected.

Issue #3

enter email address

enter your name

enter email address

Example: Here’s a great group!

3 Issue #2

Fix the email form functionality (system-side fix).

22

RESULTS | T A S K 3


WAN relies on users and volunteers to maintain the accuracy of information in the Directory. Users need an easy way to report problems to keep the information accurate and useful.

Why does it matter?

Users were asked to report a problem with a Directory listing.



Error-free Rate

Reporting a Problem

“[The ‘report a problem’link] is like almost the first thing you notice on the page.“

"They probably should have an 'other' option because these four don't encompass all of the options.“

"Invalid input? What's these people's problem?"

23

63%of users completed the task without performing an error.



38%

Issue #1: Users have big trouble using the

numeric Captcha when submitting the form.

of users received an error message stating that their Captcha was entered incorrectly when they believed it to be entered correctly.

25%of users were unable to complete the task because of this issue.

Analysis:

At first it seemed as though users could not discern between the letter “o” and zero (0), but the Captcha even failed with more discernable numbers (see screenshot above). There is something wrong with the Captcha functionality for this form.

24



25%

Issue #2: Users were confused by the

checkbox options.

of users expressed confusion or frustration about which checkbox as they felt none of them reflected their problem.

Analysis:

Although there was a free-text option for the user to explain the reason for their problem report, some users still believed that they needed to select a checkbox option. Checking a box was not required but this was not obvious to them. Debating internally as well as checking and un-checking resulted in a loss of efficiency for this task.

!

25



1

Add a checkbox titled ‘Other issue’ to the list of checkbox options. This will stop users from wasting time deliberating and being confused about what to do if their option is not available.

Issue #2

2 Issue #1

Fix the Captcha or replace it with one that works (this is a system-side fix).

26

RESULTS | T A S K 4


WAN’s directory is international, and its listings are in many countries in the organization names may be in other languages. Users should be able to easily find international groups. This puts the ‘World’ in World Animal Net.

Why does it matter?

Users were asked to find a bat-focused organization in Poland.



Error-free Rate

Finding an International Organization

"The could have the contact information on this page." (referring to the search results page)

"Oh, and that link just took me to their website. Whoops!“

“I don’t know bat in Polish. I guess just in case there's an English one, I'll try in 'bat.'"

27




25%

Issue #1: Users ended up on the

organization’s website instead of listing.

of users clicked the organization’s title in the Search Results page. They expected to go to a WAN page but were surprised to find themselves on an external website.

Analysis:

As in Task 1, users are confused by the blue hyperlink on the organization’s title. There is no indication of where this link might go. Some users assume that it will take them to a page with more information on the WAN site.

!

28



1 Issue #1

Add buttons or explicit links inside organization previews on the search results page. They should be labeled ‘Go to website’ or ‘More information’.

Remove the ‘Read more’ link and the hyperlink functionality from the title.

This will help users understand where they are going when they click a link.

Visit WebsiteMore info

29

RESULTS | T A S K 5


WAN relies on members of organizations to successfully register their own groups in the Directory. The registration process should allow users of all technical levels to easily input and submit accurate information.

Why does it matter?

Users were asked to register an organization in the Directory.



Error-free Rate

Registering an Organization

"The Anti-spam key thing is like keeping me from moving on. This would definitely be frustrating in real life, like oh my gosh, come on. I'd rather just type a word or answer a question.“

"Dear God, they make these harder and harder. This is incredibly difficult.“

“Reg complete activate? I THINK I did it…”

30




75%

Issue #1: Users experienced trouble

recognizing and assembling the form’s Captcha.

of users experienced frustration and trouble with assembling the puzzle Captcha on the registration form.

Analysis:

Most users expressed that this kind of Captcha was entirely new for them. Some did not even notice it or recognize it as a Captcha at first. They approached it with curiosity but found it to be very “particular” and difficult to assemble correctly. Users with touchpads had an especially hard time. Struggling with this process was a cost to efficiency for this task.

25%of users did not notice the Captcha at first and tried to submit the form without completing it.

31



50%

Issue #2: Users were unsure whether their

form was successfully submitted.

of users were unsure whether their form was successfully submitted. They were looking for feedback or a confirmation but couldn’t find it.

Analysis:

Users were unsure as to whether their registration had submitted successfully. Some did not understand that it was still processing and clicked ‘Send’ a second time and received an error message that their email address was already in use (since it had already been added to the database with the first click). The confirmation message is small, hard to see, and in technical jargon that is incomprehensible to users. Users who saw this (see screenshot above) felt optimistic but still unsure that they were done.

25%of users clicked ‘Send’ a second time while the form was still submitting. This resulted in an error message that confused them even more.

32



25%

Issue #3: Users believed some active form

fields to be inactive.

of users complained that some of the required fields seemed to be inactive because they were gray instead of white.

Analysis:

Although the form directions do explain the difference between the white and gray fields, users almost never read directions and so we can expect this confusion to continue.

33



1 Issue #1

Replace the puzzle Captcha with a more familiar, numeric Captcha.

34

2 Issue #2

Add a message near the ‘Send’ button on the registration form that says ‘Submitting…’ or ‘Processing…’ once the ‘Send’ button is clicked. This will provide the feedback needed for users to know the system is working.

The user should not be brought to a different page, but the same page should refresh.

A large confirmation message should display the status in human language. For example, ‘Your registration has been received.”

This should not be at the very top of the page, but rather at the top of the form, and in a different color. This is where users typically expect to see confirmation messages. Users did not notice it when it was at the very top.



3 Issue #3

Make all of the input fields white on the registration form.

If we need to distinguish between fields that are publicly visible on the Directory and those that aren’t, they should be distinguished in a different way.

We could add a little icon, such as a person, next to the public fields, and then have a key at the top of each section that reminds the user what this means.

Or we could group all of the public fields together in one section.

35

RESULTS | T A S K 6


For groups to successfully connect and collaborate, they must keep their contact info up-to-date. Users must keep their own info accurate and should be able to easily log in and make these changes.

Why does it matter?

Users were asked to log in and update their organization’s phone number.



Error-free Rate

Updating Organization Info

"I'm very confused as to how they want me to put my phone number in.“

"Why would they put [the login button] down here? It's a terrible idea. Nobody wants to scroll down to log in."

"I can't figure out where to log in, usually it's at the top."

36




100%

Issue #1: Users had trouble entering different

parts of the phone number into the right fields.

of users struggled by trial-and-error to figure out which parts of the phone number (country code, area code, etc.) was supposed to in which field. This counted as a non-critical error.

Analysis:

Most users attempted to put the entire phone number in the second field, but immediately found that it wouldn’t fit. They then struggled with different combinations until they either got it right or gave up and just left it like you see above in the screenshot. This confusion was caused by a 1) lack of labeling of the fields and the 2) misleading sizes of each field.

37



25%

Issue #2: Users could not navigate to the login

page when they tried.

clicked the ‘Login to Update Your Directory Listing’ and nothing happened. They searched for alternative links but could not find any. This prevented them from completing the task. This was a critical error.

Analysis:

All users who experienced this error were using Google Chrome. This may be an issue with pop-up settings for certain users in this browser. There were no alternative ways of getting to the login page available for these users.

38



100%

Issue #3: Users had trouble finding the login

functionality.

of users started looking for the login button in the upper-right of the page and looked all over before finally scrolling down and finding the button. Not counted as an error.

Analysis:

It is a web convention to put login links on the upper-right of the page; this is where users first expect to see it. When that failed, 2 users went directly to the footer to find the link. Some users resorted to more complicated means, such as finding their listing in the directory. Everyone verbally commented that the login button was not where they expected it to be.

25%of users tried to find login functionality by searching for their group in the Directory to see if there would be a link on their listing. Counted as a non-critical error.

25% of users looked for an alternative link in the footer after not finding one on the top of the page. Not counted as an error.

39



1 Issue #2

Fix the login pop-up button for Chrome. Or better yet, scrap the pop-up functionality altogether and make the login button a link to a separate login page. This will ensure that it works for all users.

40

2 Issue #3

Put the login link or button in the upper-right corner of the page. Add an additional link to the footer, as this study told us this is the second most common place that people look. On many sites, log links are coupled with registration.

Log in | Register your Organization



3 Issue #1

Label the phone number fields and make them each a width that is appropriate for the number of characters they expect to take.

It helps to have the cursor automatically leap to the next field when the input is full.

41

Phone number

+1

Country code

( )

Area code Number

-

OVERALL ANALYSIS

42

OVERALL ANALYSIS

Metrics Analysis

Here we compare the results from each task to get a view of site-wide usability. We can also see which areas of the site have the most serious usability issues and should be prioritized in our redesign efforts.

Percent Completion Rate % of users who completed a task

In terms of completion, Task 6 (updating organization info) was the most problematic with only 50% of users able to complete the task successfully.

Task 2 (sharing info) and Task 3 (reporting a problem) were a success for only 75% of users. Task 1 saw only one user unable to complete it successfully. All users were able to complete tasks 4 and 5.

43

OVERALL ANALYSIS

Metrics Analysis (continued)

Mean Ease of Use Rating 1=“very difficult, 5=“very easy”

Users rated Task 6 a 2.9 on average, perceiving it to be difficult. This was also the task with the lowest completion rate. Fixing issues with Task 6 should be prioritized in our redesign efforts.

Tasks 1, 2, and 5 received a neutral rating, as some users found them difficult while others had an easier time. This indicates a mixed experience that we should also look more closely at.

Tasks 3 and 4 received a mean rating of at least 4.0, or “easy.”

44

OVERALL ANALYSIS


Mean Time on Taskhow long (seconds) users took to

complete a task from start to finish

error bars represent a 90% confidence level

[192, 254]

[90, 146]

[66, 116] [59, 109][47, 95]

[149, 193]

This graph compares the mean time (in seconds) spent on each task. These do not mean much by themselves, but are very informative when 1) compared with goals and 2) when compared against results of future usability tests to see if users are able to complete tasks more quickly after redesign efforts.

The error bars give us an estimate of a range of values that would include a true population mean. For example, for Task 5, we can be 90% certain that the true population mean would fall somewhere between 192 and 254 seconds. These ranges (in green above each bar) help us understand how accurate our results are. Wide ranges reflect high variability in the data. We might improve accuracy by increasing the sample size of participants.

45

OVERALL ANALYSIS


Error-free Rate

46

% of users who completed a task without any errors

Task 6 had the lowest error-free rate (0%). No users got through this task without performing a critical or non-critical error.

The best rates were those of Tasks 3 and 4, but these were at 63%. No tasks were completely error-free. This tells us that all of the tasks could stand for some improvement.

RESULTS | POST-TEST QUESTIONNAIRE

Post-Test Questionnaire

After finishing the task portion of the testing session, each user was asked to answer a post-test questionnaire about their overall experience using the site.

3.6Average rating given by users when asked to rate their experience finding information on this site.Likert scale, 1= very difficult and 5=very easy

75%

When asked what they believed the site’s purpose to be, 90% of users mentioned key parts of WAN’s articulated mission for the Directory. They used words such as “coordinate,” “communication,” “social hub,” and “empower one another.”

25%When asked what they like best about the site, the answers were varied. The most common answers were that it was easy to filter and find groups in the directory (25%) and that the layout was simple and wasn’t cluttered (25%).

50%When asked what they like least about the site, the most common answer (40%) was that uses did not like the appearance of the site, e.g. the colors or logo, or thought that the site looked outdated.

47

RESULTS | POST-TEST QUESTIONNAIRE

Post-Test Questionnaire (cont.)

When asked what they would change about the site, 38% of users mentioned changing the colors or logo. Another 38% also asked for larger icons and buttons for key functions.

38%

48

3.63.6 was the average rating when asked “How likely are you to use the World Animal Website in the future?” with 1= very unlikely and 5= very likely. Only the responses of participants (5) who answered that they were active in organizations or lifestyle choices to promote animal welfare, protection, or rights were included in this number. The average across all users was 2.9.

Site-wide redesign ideas:

- Consider a site-wide redesign of colors and graphics- Consider designing a new logo- Keep the simple and uncluttered layout- Increase the size of buttons and links

2.9

APPENDIX

49

SCREENER QUESTIONNAIRE RECRUITMENT FLYER MODERATOR SCRIPT

USABILITY TESTING GUIDE

Thank you for your interest in volunteering to test a non-profit animal protection website. Please answer the following questions. After your answers are submitted, we will contact you about scheduling the usability study. Answers to this questionnaire will be kept completely confidential.

Are you at least 18 years of age?- Yes- No

Do you currently live in the United States?- Yes- No

Do you speak English fluently?- Yes- No

Are you a member of and/or participate in activities of an animal rights, animal protection, animal welfare, or similar group?- Yes (please specify ________)- No- Not sure

NEXT

The Screener Questionnaire was delivered to prospective participants

to ensure they meet the demographic and technology criteria to

complete the usability testing.

APPENDIX | SCREENER QUESTIONNAIRE

50

Please select the choice that best describes you:

- I am a member of an animal rights, animal protection, animal welfare, or similar group

- I am not a member of an animal rights, animal protection, or animal welfare group but I actively participate in activities or lifestyle choices to promote related goals

- I do not actively participate in activities that promote animal rights, animal protection, or animal welfare, but I am looking for opportunities to do so

- I am NOT currently seeking opportunities to promote animal rights, animal protection, or animal welfare

Do you have a Google Chrome browser installed on your computer, or are able to install one on your computer?

- Yes- No- Not sure

We are looking for volunteers to try using a non-profit website animal protection website and give us their thoughts about it in a study (30 minutes to 1 hour in length). It would be scheduled for a time that works for you and done in a private online conference room. Are you interested in participating?

- Yes- No

NEXT

APPENDIX | SCREENER QUESTIONNAIRE (CONT)

51

The user testing session will occur remotely, in an online conference room with a moderator on the other end. Your voice and what you do and see on the screen will be recorded. On the computer you would use, do you have a built-in microphone or other way to speak and record your voice into your computer?- Yes- No- Not sure

What is a good email address at which to contact you? This information will be used for scheduling purposes only, will be kept confidential, and will not be shared with any other parties.______________________________________

SUBMIT

APPENDIX | SCREENER QUESTIONNAIRE (CONT)

52

Thank you so much for coming in today. I wanted to give you a little information about what you will be looking at and give you time to ask any questions you might have before we get started.

Today we are asking you to serve as an evaluator of a Web site and to complete a set of tasks. Our goal is to see how easy or difficult you find the site to use.

You are free to stop this study for any time and for any reason. Please also remember that we are testing the website, and not your abilities.

I am here to record your reactions and comments of the Web site you will view. You will be audio recorded. A recording of your screen will also be captured.

*Please navigate back to the guide and click the link to the consent form. Please read it over and let me know if you have any questions.

Do you have any questions?

Please sign and date the consent form, and click ‘submit.’

[wait for user to complete consent form]

Please close the consent form tab.

More

This script was read aloud by the moderator to each participant during

the usability testing sessions. The tasks were also read aloud (see the

usability testing guide).

APPENDIX | MODERATOR SCRIPT

54

APPENDIX | MODERATOR SCRIPT (CONT)

55

During this session, I would like you to think aloud as you work to complete the tasks. I will not be able to offer any suggestions or hints, but from time to time, I may ask you to clarify what you have said or ask you for information on what you were looking for or what you expect to have happen.

The session will take between 30 minutes and one hour. Feel free to take a break whenever you need to. You are also free to stop at any time.

Do you have any questions before we begin?

Intro to the site

Please navigate back to the guide and click ‘Next’. Click the link to open the website in a new tab. This is the website we are testing today is World Animal Net, a non-profit group that works toward animal protection.

Please remember these tasks are evaluating this site, and not your performance. This is in no way a test of your abilities. Also, I did not design this site, so you cannot hurt my feelings. Please be honest and candid as you think aloud during the tasks.

Are you ready to begin the first task?

Welcome! Thank you for volunteering to participate in this usability study. The insights you provide will help us improve a non-profit website that helps connect animal protection groups all over the world. We also hope you find it to be a fun and interesting way to volunteer.

Please keep this tab and guide open in your browser throughout the testing session. You will refer back to it and navigate through for details and instructions.

At your scheduled testing session time, please navigate to this conference room link to begin. The moderator will be online and waiting for you. Please ensure you are using a Google Chrome browser. If, after you click the link, you're presented with a pop-up window, please click "launch application" to launch Join.me:

Click here to open conference room in new tab

(Please wait for instructions from the moderator before clicking 'Next' below)

NEXT

The Usability Testing Guide was delivered via Qualtrics and was kept

open by users throughout the usability testing session. It contains the

consent form, task instructions, post-task and post-test questionnaires.

APPENDIX | USABILITY TESTING GUIDE

56

APPENDIX | USABILITY TESTING GUIDE (page 2)

Please read and complete this consent form.

During this usability test I agree to participate in an online session using my computer and microphone. During the session I will be interviewed about the site, asked to find information or complete tasks using the site and asked to complete an online questionnaire about the experience.

I understand and consent to the use and release of the recording by World Animal Net. I understand that the information and recording are for research purposes only and that my name and image will not be used for any other purpose. I relinquish any rights to the recording and understand the recording may be copied and used by World Animal Net without further permission.

I understand that participation is voluntary and I agree to immediately raise any concerns I might have.

If you have any questions after today, please contact Lauren Thomas at [email protected].

Please click 'I consent' below to indicate that you have read and understand the information on this form and that any questions you might have about the session have been answered. Then click 'Next' to continue.

I consent

Back

Next

57


Please navigate to the following website. The link will open in a new tab. This is the site we will be evaluating today:

http://worldanimal.net

(Please wait for instructions before clicking 'Next' below)

Back

Next

58

http://worldanimal.net/


Task 1

[Please do not begin this task until I have finished reading the instructions aloud. Please think aloud as you complete the following task. Please let me know when you've completed the task.]

You are an active member of Save Our Bats, a bat conservation and protection group in Austin, Texas. Your group wants to organize a national White-nose Syndrome Awareness Day to help educate people all across the country about this terrible disease that kills bats. You’ve been asked to come up with a list of other bat groups in the U.S. so that your group can contact them to share ideas and coordinate events. Using the World Animal Net website, find one similar bat-focused group in another U.S. state.

Navigate back to the World Animal Net website tab to complete this task. You may refer back to this tab for information as needed.

Back

Next

59


Did you successfully complete this task?

• Yes• No• Not sure

Please rate the difficulty of this task.• 1 (very difficult)• 2 (difficult)• 3 (neutral)• 4 (easy)• 5 (very easy)


Back

Next

60


Task 2


You decide to save the group’s contact information for later. Using the World Animal Net website, email this group’s information to yourself at [email protected].

When you're ready to begin, please click this link and start there (it opens in a new tab).


Back

Next

61






Back

Next

62


Task 3


You notice that the organization's name is spelled wrong in its listing. Report the problem to World Animal Net.

When you're ready to begin, please click on this link (opens in a new tab) and start there.


Back

Next

63






Back

Next

64


Task 4


You start thinking about how great it would be if White-nose Syndrome Awareness Day could go global. You also recall that a member of your group speaks Polish. Using the World Animal Net website, find the phone number of a bat-focused group in Poland.


Back

Next

65






Back

Next

66


Task 5


You realize that your group is not listed in this directory, and you decide to add it so that other groups can find your group, too. Using the World Animal Net website, register your organization (use the following information as required):

Organization name: Save Our BatsFirst name: BatildaLast name: McBatPosition: VolunteerEmail: [email protected]: BatsRule123


Back

Next

67






Back

Next

68


Task 6


You just remembered that you forgot to add your phone number to the listing. On the World Animal Net website, use the information below to log in and add the phone number 512-555-7777 to your group's listing so that other bat activists know how to contact you.

Email: [email protected]: BatsRule123


Back

Next

69






Back

Next

70


You will now be asked to answer a few questions about your overall experience using the World Animal Net website. You may stop sharing your screen while you complete these questions.

Back

Next

71


You will now be asked to answer a few questions about your overall experience using the World Animal Net website. You may stop sharing your screen while you complete these questions.

Back

Next

72


Please rate your experience finding information on this site.1 (very difficult)

• 1 (very difficult)• 2 (difficult)• 3 (neutral)• 4 (easy)• 5 (very easy)

What do you think the World Animal Net website is used for?[Free text response]

What do you like the best about the World Animal Net website?[Free text response]

What do you like the least about the World Animal Net website?[Free text response]

What would you change about the World Animal Net website?[Free text response]

How likely are you to use the World Animal Net website in the future?

• 1 (very unlikely)• 2 (likely)• 3 (neutral)• 4 (likely)• 5 (very likely)

BackSUBMIT

73

website usability testing report · usability metrics were gathered to assess the usability of each...

Documents