usability test findings tch1 version 1.00.02a testing dates: march 27 th – 31 st 2009 russound...
TRANSCRIPT
Usability Test FindingsTCH1 Version 1.00.02A
Testing Dates: March 27th – 31st 2009
Russound"TCH1 TouchPoint" iPhone Interface
Usability Testing ObjectivesCapture the User Experience (UX) for
standard product features through task based exercises
Test Subjects without exposure to project75% iPhone/iPod Touch users to 25% non-
user ratioIdentify Pro’s and Con’s to the User
Interface (UI)Obtain supporting data that may help improve
the UXNote that some of the suggested changes
may be irrelevant if the product has had changes made since testing which are supported by these findings
1. Please start the Russound TouchPoint application to control the audio system.
Suggested Change:
“TCH1” could be replaced with “Russound”
25% of total users had some minor difficulty. Comments were
that the label wasn’t
informative enough
100% of total users
accomplished this task
2. Turn on the Kitchen to play some music.
Suggested Change:
Upon launch of application, possibly first present the user with a zone list to opt for the specific room to control as the last room used is not always one desired for control in a future session
37.5% of total users selected the zone power button for their
first try and averaged 3
attempts before successfully turning the
Kitchen zone on
100% of total users
accomplished this task
62% of total users turned
the kitchen on with their first
attempt
3. Turn up the volume level.
50% of total users selected Next Source
button on their first try
100% of total users
accomplished this task
50% of total users selected the Plus Button on their first try
But, only 12.5% of total users did this on the first try
All users noted that a volume level indicator
should be visible at all times or
have a text label
Suggested Change:
Have a volume range indication shown at all times that is filled as the level increases. Use of different iconography such as speaker symbols with radiant rings for less and more. Labeling would additionally be even more helpful in eliminating the confusion with source selection
4. How would you instantly mute the audio - not just turn the volume down incrementally?
25% of total users
accomplished this task by
selecting the lowest area on the volume bar and 12% turned
the zone off
50% of total users could not find the Mute feature and
averaged 2.25 tries before giving up
Suggested Change:
Have a Mute Icon located at the lower end of the volume level indicator such as a circled speaker with a line through it
50% of total users
accomplished this task on the
first try
Users commented that there was no indication as to
where this feature lived such as an
icon or text
5. Select another source to listen to.
100% of total users
accomplished this task but…
All users had prior
knowledge
100% of users learned this
feature through error on a prior
task
Suggested Change:
While this task was compromised through prior knowledge, labeling the source selection area might have prevented that from happening.
Several users commented that a “Source” label would be helpful or numbering the list (1234...) to prevent confusion with volume
Suggested Change:
Since 50% of total users didn't get this on the first try labeling the source list access area is recommended. Some users mentioned a numbered list instead of dots would be preferred
6. Without scrolling through sources one at a time, directly select the Cable Box.
50% of total users Selected the Source Icon
as their first choice.
12% of total users weren't
able to find the feature at all.
50% of the users who
didn't accomplish this task with their
first choice tried the Source
Name area at some point and
averaged 3 tries
88% of total users
accomplished this task25% of total users
stated at some point -
"I don't know"
25% of total users thought that clicking a
specific dot would select
the corresponding
source
50% of total users
accomplished this task on the
first try but they had prior
knowledge
Suggested Change:
Indication of numeric keypad page existence needs to be shown since it wasn't clear to most users where to go to for this function.
Either it was earlier mistaken for possible access to another feature from which they gained the memory of its location, or they indicated there was no other option left to try.
62% of total users
accomplished this task on
their first try.
7. Go to the AM/FM Radio and directly tune in 107.1 FM
12% of total users who
completed the task selected the data field
where frequency is
displayed
25% of users who completed
this task selected the Source Icon
image37.5% of total users commented
–“There was no
place left to try”
12.5% of total users who
completed the task selected the Source Name field
However, 50% of total users
had previously learned where
this feature lived
100% of total users
accomplished this task
8. You have AM/FM Radio stations preset. How do you access them?
Only 38% of total users did this on their
first try
Users who did not accomplish this on their first attempt averaged 2.2 tries
Suggested Change:
Possibly use text to label preset buttons though this did not seem to be a significant user difficulty.
60% of users who didn't
accomplish this task on their
first try selected the + button as
their first choice.
50% of all users who failed to complete this task selected the Source Name as a
possible option
50% of all users who failed to complete this task selected
the Source Icon as a possible
option
50% of all users who failed to complete this task selected
the Source List as a possible
option
38% of users commented that
they felt an AM/FM selection should exist on
the keypad page for direct tuning
75% of all users failed to perform this
task
12.5% of them had prior
knowledge of this features
use and 12.5% accomplished this task by
accident
Suggested Changes:
Either the play button should be relabeled on this screen to be AM/FM
or an AM/FM selection should exist on the Keypad page
9. How would you select an AM station versus FM? Please tune in AM 1030.
Only 25% of total users
accomplished this task
"I have no idea""I got nothing"
"I wouldn't know how"
50% of total users
accomplished this task
37.5% of total users
accomplished this on their
first try
50% of all users failed to perform this
task
10. Increase the bass level in your room.
60% of users who didn't
complete this task on the first try had in fact selected the
Zones list page then left it
without selecting the blue 'more'
arrow button to reveal the zone settings page.
Suggested Changes:
Settings specifically for a zone were not easily found when living behind a global zone list.
Users struggled with zone specific settings living behind the global zone page instead of accessing the zone settings from the zone’s page directly. Providing this option on the zone page may be more fitting in the overall hierarchy.
Replacing the > button for the zone settings on the zone list page with 'more' or 'settings' might have guided the users to the location of the zone specific features.
Impossible task
Goal:Obtain desired user method
for accomplishing
this feature
Suggested Changes:
Change the Zone List page’s power button to be an All Off feature
11. Turn off all rooms in the system.
38% of all users selected the
Zones page first62% of all users
selected the zone power
button first with over half of
these trying a subsequent
press and hold action.
88% of users believed that the power button on the Zones List
page was the “All-Off” feature
Summary
Obtained fast, qualitative resultsWhile not perfect, the testing prototype
served a valuable role in permitting us to capture the User Experience
Survival of the easiest is driving consumer purchases regarding heavily based UI products
Improvement Priority Review
Task: #9
Item Description: AM/FM Selection Control
Test Failure: 75% did not complete this task
Test Success: 12.5% had prior knowledge & 12.5% did by mistakeSuggested
Improvement:Play/Pause button should be relabeled as 'AM/FM' on Tuner screensNote: Latest build has (>II) button performing pause function which mutes tuner. This needs further resolution
Risk:Low Notes:
Time:3 hrs Notes:
Priority: 1 Notes:Must Have
Improvement Priority Review
Task: #3
Item Description: Volume Adjustment
Test Failure: 87.5% did not do this task on their first try
Test Success:100% accomplished task but only 12.5% did on their first try
Suggested Improvement:
A: Replace the Volume Up and Down arrows with speaker iconsB: Have a visible volume range graph that is filled to selected level
Risk:Low (A) Notes:B: High Risk
Time:30 Min (A) Notes:B: 8 Hrs
Priority: 2 (A) Notes:Icon size: 35x35px PNG format
Improvement Priority Review
Task: #4
Item Description: Perform Mute
Test Failure: 50% did not complete this task
Test Success: 50%Suggested
Improvement:Change the text to 'Muted' from 'Mute'Add an icon (Circled speaker with a line through it) located above the lower end (left) of the volume indicator area
Risk:Med Notes:Text only would be Low risk
Time:3 Hrs Notes:Text only would be 30 min
Priority: 3 Notes:Icon size: 20x20px PNG format
Improvement Priority Review
Task: #10
Item Description: Adjust Zone Settings
Test Failure: 50% did not complete this task
Test Success: 37% success on first attemptSuggested
Improvement:Link to zone specific properties should be reached through the zone page. Suggested to reside behind source name instead of TCH1 version. Zone properties can also remain accessible through current navigation
Risk:Med Notes:Navigation changes
Time:4 Hrs Notes:
Priority: 4 Notes:
Improvement Priority Review
Task: #1
Item Description: Select TCH1 Home Page IconTest Failure: 25% had difficulty
Test Success: 100% completed this taskSuggested
Improvement:TCH1 doesn't mean anything to users. 'Russound' does and matches the icon being the R Logo. Suggestion is to change the icon text to 'Russound'
Risk:Low Notes:
Time:30 Min Notes:
Priority: 5 Notes:
Improvement Priority Review
Task: #7
Item Description: Access the Numeric KeypadTest Failure: 0%
Test Success:62% succeeded on the first try but 50% had prior knowledge
Suggested Improvement:
Provide a visual indication that there is a keypad which exists for that source type. *The (i) is a typical icon used on iPhone / iPod Touch to indicate more information exists or a keypad icon can be used
Risk:Med Notes:Source Type dependent
Time:4 Hours Notes:
Priority: 6 Notes:Icon size: 20x20px PNG format
Improvement Priority Review
Task: #2
Item Description:Select a zone to turn on other than the one the UI was last left on
Test Failure:62% turned on the last zone the TCH1 was left on not the desired one
Test Success:100% were able to complete the task but only 38% did it on the first try
Suggested Improvement:
Provide the user with the list of zones for selection of the desired room to control upon initial launch. This could be an issue for performing simple tasks such as volume adjustment or such as you would have to select the room each time
Risk:High Notes:
Time:10 Hrs Notes:
Priority: 7 Notes:
Roger SoucyApril 3rd 2009
Russound