Usability testing:A reality check for Plain TalkA brief overviewOctober 7, 2008Dana Howard BotkaManager, Customer Communications, L&IPlain Talk Coordinator, Office of the Governor

What is usability testing?

Finding out if your targeted customer can actually use your product, such as a: Set of instructions Form Web site Software application Automobile Cell phone

How does it work?

4-6 people try out your document, web page, or other product one a a time.

One facilitator conducts the try-out, asking open-ended questions.

The usability participant is asked to “think out-loud” while doing tasks.

Up to two program reps observe and take notes – or record the session.

We don’t ask testers what they like

We develop a test to see if the typical “user” can actually do a task, such as: Calculate a late fee. Fill out a form without making errors. Find the answer to a question on a web site. Correctly figure taxes. Turn on a car defroster quickly. Forward a cell phone call.

Why usability testing for Plain Talk?

Because you discover: Terms customers don’t understand. Words that can mean something else. Basic information or context that is missing. Information that is unimportant & distracting. Your customer’s emotional reaction to what you’ve

developed. What you had no idea would happen!

How does usability work?

Recruit 4-6 people who fit the description of your typical user – but who are not familiar with the “product.”

Develop 6-10 tasks that, if performed correctly, will let you know they can “use” your product.

Write a scenario that will allow the test participant to understand what situation they are in.

Write a series of questions that will get the test participant to do a task – without hints.

L&I’s “Quarterly Report” Project:Helping employers self-report hours

Challenge: High customer error rates on this report. Errors often lead to late fees for employers and costly

processing headaches for L&I.

Solution: Make “paper reporting” easier for customers with

instruction sheet and “late fee worksheet.” Encourage employers to use upgraded online report

application.

Our test materials

See the documents in your packet

1. Before & after examples of our test document

2. Pre-test screening questionnaire

3. Instructions and legal sign-off

4. Scenario and test questions

5. Sample “thinking aloud” from a test participant

6. Findings and Recommendations

Form before revision

Form instructions before revision

Setting up the usability test. Plain Talk team members decide what they most need to learn from recruited test participants.

Form after revision

Form instructions after revision

Usability testing on a budget

Recruit test participants from friends, family, agency walk-ins.

Keep the test simple and short. Use results as a guide – not final data. Consider the act of developing the test as a great way

to focus on your project goals.

What about a survey or focus group?

Not a replacement for a test of whether someone can do a task.

People will often say they “like” a document or web site – and not know how to use it. This is because they don’t want to:– Admit they had trouble.– Hurt your feelings.– Go to the trouble of seeing if it actually “works.”

Invite managers and staff to observe testing! It’s a great reality check.

Involve frontline staff in usability testing. They know their customers – and are great at solving problems.