USABILITY TESTINGWVU Libraries RoomMe

WVU LIBRARIES WEBSITE STUDYProblem:

We used TechSmith Morae to conduct usability testing of the new WVU Libraries RoomMe web application on desktop, tablet,

and mobile devices. This round of usability testing was internal, utilizing undergraduate student employees, graduate student

employees, and staff. This is the results of the first round of usability testing.

1ST ROUND OF TESTINGTask 1: Go to WVU Libraries' room reservation.

Task 2: Reserve a study room at the Downtown Library for a group that has 10 people.

Task 3: At the Downtown Library, what is the maximum number of hours for one reservation?

Task 4: At the Evansdale Library, is Classroom 130 available at 9:00AM on August 18th? If so, reserve it.

Task 5: How far ahead are you able to make a reservation at the Downtown Library? At Evansdale?

TEST PARTICIPANTS

EvanUndergraduate Student

JessicaLibrarian

RaeUndergraduate Student

AutumnGraduate Student

DavidDeveloper

OVERALL TASK:AVERAGE TIME

* Average time includes delivery of tasks.

OVERALL TASK:SUCCESS

Completed

Completed with difficulty

Completed with ease

Almost Didn’t Completed

TASK 2: Reserve a study room at the Downtown Library for a group that has 10 people.

User Observations: - 60% of users expect that the ‘Find A Room Button’ will change

the contents of the first loaded results, and don’t understand that the information is dynamically loading. A grayed-out button that turns blue once a different selection is made could alleviate this.

- http://community.wvu.edu/~twb004/usability3/t21.wmv- http://community.wvu.edu/~twb004/usability3/t22.wmv- http://community.wvu.edu/~twb004/usability3/t23.wmv

- 80% of people don’t see a way to search specifically for room capacities, or are just selecting rooms one-by-one until they find a listed room capacity, despite the room policies information. Listing a room capacity or suggested room capacity on each room page could fix this issue. An enhancement could also be to add room capacity as a search criteria.

- http://community.wvu.edu/~twb004/usability3/t24.wmv- http://community.wvu.edu/~twb004/usability3/t25.wmv

- Form/input buttons on iPad are too thin vertically. Increase height for iOS devices.

TASK 3: At the Downtown Library, what is the maximum number of hours for one reservation?

User Observations: - 80% of the users are overlooking ‘Room Policies’, especially on

mobile devices. Room Policies should be more visually apparent on mobile devices, and should be better integrated into the individual rooms’ information with better visibility.

- http://community.wvu.edu/~twb004/usability3/t31.wmv- 60% of users are expecting a NUI experience in reserving the

rooms and are confused when met with errors, complications, or policy information differences; users are always going to use the system before reading the policies. If a user should be limited in scheduling to six hour intervals instead of being allowed to schedule 23 hours, or just allowed to schedule six hours from the start time if that is the policy.

- http://community.wvu.edu/~twb004/usability3/t32.wmv

TASK 4: At the Evansdale Library, is Classroom 130 available at 9:00AM on August 18th? If so, reserve it, then Cancel it.

User Observations: - Once the user is on room 130 in the Evansdale Library they are not

seeing a way to reserve the room, so they think it is off-limits or broken as opposed to busy. An error message speaking to specifically to a room being reserved should be implemented, and reservation results should appear at the top of the page in list format, rather than at the bottom in a table format.

- http://community.wvu.edu/~twb004/usability3/t41.wmv- Users can see all of the rooms via the building view, but aren’t able to

reserve those buildings do to being disabled for the summer. If rooms are disabled for renting during the summer, they shouldn’t populate on the building list at all, or she be marked ‘unavailable’.

- http://community.wvu.edu/~twb004/usability3/gen1.wmv- Users are confused when not being able to find the room in the mobile

list even if it is not available. List unavailable rooms on mobile in red and parenthetically cite ‘reserved’ or ‘unavailable’ afterwards.

- http://community.wvu.edu/~twb004/usability3/t42.wmv- Users are confused when search results don’t populate at the top of the

page before search options, and wonder if the search occurred. Make search results always populate at the top of the page.

- Users don’t see changes occurring with dynamic data. Add a loading wheel for queries that dynamically load content in-page.

- http://community.wvu.edu/~twb004/usability3/t43.wmv

TASK 5: How far ahead are you able to make a reservation at the Downtown Library? At Evansdale?

User Observations: - Again, 80% of users are expecting a NUI experience in reserving

the rooms and are confused when met with errors, complications, or policy information differences; users are always going to use the system before reading the policies. If a user should be limited in scheduling only so many weeks or hours into the future, they should not be presented options if policies are limiting, or scheduling is not a possibility.

- http://community.wvu.edu/~twb004/usability3/t51.wmv- http://community.wvu.edu/~twb004/usability3/gen2.wmv

- Users can’t delete reservations on Mobile. Change reservations from displaying in a table format, especially on mobile.

- http://community.wvu.edu/~twb004/usability3/t53.wmv

SYSTEM USABILITY SCALE (SUS) QUESTIONS1. I think that I would like to use this system frequently2. I found the system unnecessarily complex 3. I thought that the system was easy to use 4. I think that I would need the support of a technical person to be able to use this system 5. I found the various functions in this system were well integrated 6. I thought there was too much inconsistency in this system 7. I would imagine that most people would learn to use this system very quickly 8. I found the system very cumbersome to use 9. I felt very confident using the system 10. I needed to learn a lot of things before I could get going with this system

SUS: RESULTS

1

3

3

1

2

3

2

2

1

1

1

1

1

3

2

1

1

1

1

1

2

1

1

2

1

1

1

2

1

1

3

2

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

I think that I would like to use this system frequently

I found the system unnecessarily complex

I thought that the system was easy to use

I think that I would need the support of a technical person to be able to use this system

I found the various functions in this system were well integrated

I thought there was too much inconsistency in this system

I would imagine that most people would learn to use this system very quickly

I found the system very cumbersome to use

I felt very confident using the system

I needed to learn a lot of things before I could get going with this system

Strongly Disagree Disagree N/A Agree Strongly Agree

USABILITY SUGGESTIONSUsers by far have a more favorable desktop and tablet experience - which requires less time and understanding - than a mobile experience. The mobile experience seems to frustrate users and causes task completion and functionality confusion.

UI and UX suggestions to follow…

UX ENHANCEMENTS- Implement a grayed-out button that turns blue once a different selection

is made.

- List a room capacity or suggested room capacity on each room page.

- Add room capacity as a search criteria.

- Increase input heights for iOS devices.

- Room Policies should be more visually apparent on mobile devices and within individual rooms.

- Users should be limited in scheduling to six hour/policy intervals in dropdowns instead of being allowed to schedule the default 23 hours.

- Implement error messaging that speaks to specific room policies within the room being reserved.

- Reservation results should appear at the top of the page in list format rather than at the bottom in a table format.

- If rooms are disabled for renting during the summer, they shouldn’t populate on the building list at all.

- Unavailable rooms should be marked as such.

- List unavailable rooms on mobile in red and parenthetically cite ‘reserved’ or ‘unavailable’ afterwards.

- Make search results always populate at the top of the page.

- Add a loading wheel for queries that dynamically load content in-page.

- Limit users to scheduling only so many weeks or hours into the future based on policies; don’t present options or scheduling if it’s not possible to accomplish.

- Change ‘My Reservations’ from displaying in a table format, especially on mobile.

UI ENHANCEMENTSI would suggest possibly implementing the following user interface changes in the next upgrade or version of RoomMe:

- Remove or minimize website integration on mobile devicesThis is mainly the header and footer and website navigation, which should be secondary to the web application’s core functionality, and takes up too much space on a mobile device.

This could be accomplished in the branding upgrade we do this fall, which will have a simplified navigation and minimal branding.

- Implementing a tab barImplement a sticky tab bar that positions essential and repetitive tasks – searching for a room, login, policies, etc. – through universal iconography. This would afford users the screen space to accomplish the core functions that are integral and necessary, while providing a ease to finding navigation and repetitive tasks.

- Optimize use of interactive contentUse accordions and modals to subdivide content, which will conserve in-page space and group together like content. Examples of this are forms, room information, and in-page policies.

Forms need to be completely standardized and re-worked to be more mobile friendly, as well as work with notification and completion signage to better indicate failure, progress, and success.