Improving Federal Forms with User Experience Testing and Eye Tracking

Jennifer Romano Bergstrom, Jon Strohl, Andrew Hale, Sarah KeatonMay 17, 2014

AAPOR | Anaheim, CA@romanocog #AAPOR

2

@romanocog #AAPOR

+ emotions and

perceptions = UX

Usability = “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” ISO 9241-11

2

3User Experience Design (P. Morville): http://semanticstudios.com/publications/semantics/000029.php

User Experience

@romanocog #AAPOR

3

4

OBSERVATIONAL+ Ethnography

+ Time to complete task

+ Reaction time+ Selection/click

behavior+ Ability to complete

tasks+ AccuracyIMPLICIT

+ Facial expression analysis+ Eye tracking

+ Electrodermal activity (EDA)+ Behavioral analysis

+ Linguistic analysis of verbalizations

+ Implicit associations+ Pupil dilation

EXPLICIT+ Post-task satisfaction

questionnaires+ In-session difficulty ratings

+ Verbal responses+ Moderator follow up

+ Real-time +/- dial

UX Data

4

@romanocog #AAPOR

Traditional UX research is good at explaining what people say and do, not what they think and feel.

Why should we measure implicit?

@romanocog #AAPOR

6

Modern eye tracking

Modern Eye Tracking

Co

pyri

ght ©

201

3 T

he N

iels

en

Com

pan

y. C

on

fiden

tial a

nd p

rop

rieta

ry.

7

Example MethodologyParticipants:• N = 74 | Average Age = 37• Mix of gender, ethnicity, income• Random assignment to diary

condition• New, Old, Prototype, Bilingual

Usability Testing session:• Participants read a description of the

study.• The moderator gave instructions and

calibrated the eye tracker.• Participants completed Steps 1-5 in the

diary at their own pace. • End-of-session satisfaction

questionnaire• Debriefing interview

Eye Tracker

Moderators worked from another room.

Control Room

Slide from Walton, Romano Bergstrom, Hawkins & Peirce – AAPOR 2014

8User Experience Design (P. Morville): http://semanticstudios.com/publications/semantics/000029.php

@romanocog #AAPOR

Plain language

*Public Law 111-274. Retrieved from http://www.plainlanguage.gov/plLaw/

• President Obama signed the Plain Writing Act of 2010 requiring federal agencies to use “clear Government communication that the public can understand and use.”

– Find what they need

– Understand what they find

– Use what they find to meet their needs

8

Jarrett, C., & Gaffney, G. (2009) Forms that Work: Designing Web Forms for Usability. Morgan Kaufmann

Government forms matter!

@romanocog #AAPOR

9

Poor forms lead to: • Increased errors• Lower response rate• Higher nonresponse• Loss of time • Increased

frustrations• Poor user

experiences.

Jarrett, C., & Gaffney, G. (2009) Forms that Work: Designing Web Forms for Usability. Morgan Kaufmann

Good forms = good UX

@romanocog #AAPOR

10

Issue: Instructions are not read, and the starting point is not clear

*Hochheiser, H. & Shneiderman, B. (2000). Performance benefits of simultaneous over sequential menus as task complexity increases. International Journal of Human-Computer Interaction, 12(2), 173-192.

UX Best Practice: Use step-by-step menus for simple forward-moving tasks.*

“It’s not intuitive at all for where I’d go to find information.”

Users select their desired job to plan a career path.• Keyword search • Select from a series of

dropdowns.

UX Findings:• Instructions are not read • Users enter their current job, not

desired job.• It is not clear that users have the

option to either search or enter job details when entering a starting job.

Fixation count gazeplot shows the participant looks back and forth between the two input options

@romanocog #AAPOR

11

Issue: The layout does not encourage left-to-right processing

UX Best Practice: Allow users to enter information in ways that are logical to them.

@romanocog #AAPOR

12

Version 1 PrototypeVersion 2

Users enter their household TV viewing for one week.

Issue: The location of navigation buttons cause confusion

*Niemela, M. & Saarinen, J. (2000). Visual search for grouped versus ungrouped icons in a computer interface. Human Factors, 42(4), 630-635.

UX Best Practice: Clearly differentiate navigation elements from one another.*

Users enter the job title or keyword that they want to search for and click the arrow pointed to the right to run the search.

UX Finding:• The “Save this Search” button is closer to the box where the job title is entered

than the arrow icon that runs the search.• Some participants incorrectly clicked on “Save and Search” to continue.

After clicking on the Save This Search button, one participant said: “Hmmm, create a new account and I actually don’t want to create a new account…Maybe I can just scroll down and have a look here.”

@romanocog #AAPOR

13

Issue: It is not clear how to get started and enter data on the form

1: You want to compare the average cost of a public university to the average cost of a private university. Using this site, approximately how much more is a 4-year private university likely to cost?2: You’re deciding between the University of Virginia Main Campus and Georgetown, and you’re interested in graduation rates. Which university graduates a higher percentage of students who enroll?

UX Best Practice: Ensure clickable parts of the form appear clickable.

Users must click “Add a School” to get started. They select a college or university and enter financial data.

UX Findings:• The “Add a school” button does not look

like an actionable button. It is not clear that this is how to start.

• The form did not look like it was clickable.• Users were unsure how to proceed once

they had successfully brought up two schools side-by-side.

10

17

2

1510

4Task

Accuracy

Public/Private Task1 Graduation Rate Task2

Success Fail n/a

@romanocog #AAPOR

14

Issue: The text on the home page is dense, and the form is not visible.

UX Best Practice: Ensure that the location of page elements do not create the illusion that users have reached the bottom of a page when they have not.1

Facilitate scanning by using clear headings, short phrases and sentences, and short paragraphs. 2

Users enter information into the form and are shown their cancer risk.

UX Findings:• Users do not read the dense text. • The form to enter information is below the fold,

and the layout creates the illusion that the bottom of the page has been reached.

“It seems like a lot of information. I would skip past it and use the tool.”

“I see blah, blah, blah. A lot of information and I don’t really care for it.”

“I didn’t see it because I didn’t scroll down. If I saw this on the first page, I would have been much happier.”

1: Ivory, M.Y., Sinha, R.R., & Hearst, M.A. (2000, June). Preliminary findings on quantitative measures for distinguishing highly rated information-centric web pages. Proceedings of the 6th Conference on Human Factors and the Web. Retrieved November 2005, from http://www.tri.sbc.com/hfweb/ivory/paper.html.2: Morkes, J. & Nielsen, J. (1997). Concise, SCANNABLE, and objective: How to write for the Web. Retrieved November 2005, from http://www.useit.com/papers/webwriting/writing.html.3: Rosenholtz, R., Li, Y., Mansfield, J. & Jin, Z. (2005).

@romanocog #AAPOR

15

See Bristol, Romano Bergstrom & Link, AAPOR 2014 for more about the UX across devices.

• Not necessarily the same UX across all devices

• Different issues occur with different modes• Match user expectations and survey

objectives• Conduct UX testing with real users

Assess the UX across devices.@romanocog #AAPOR

16

Thank you!• Twitter: @forsmarshgroup, • LinkedIn: http://www.linkedin.com/company/fors-marsh-

group • Blog: www.forsmarshgroup.com/index.php/blog

Jennifer Romano Bergstrom@romanocog

jbergstrom@forsmarshgroup.com

AAPOR 2014| Anaheim, CA