Heuristic Evaluation:

California DMV Website

Ryan Schabel

11/26/2013

1

Table of Contents 1

Introduction 2

Executive Summary 3

Inquiry Methods 5

- Survey 5

- Interview 7

Global Issues 9

- Error Prevention 9

- Main Menu and Submenus 9

- Navigation & Mapping 12

- Organization and Difficulty Finding the Right Questions 16

Local Issues 18

- Inefficient Use of the Home Page Real Estate 18

- Unnoticed “Important” Alerts 20

- Appointment Feature Needs More Visibility 21

- Making an Appointment 22

- Three Unnoticed Submenus 37

- The “How Do I…” Menu 40

- Finding a DMV Office Location 43

Conclusion 50

Appendix 52

Contact Info 53

2

Introduction

To establish the degree of a website’s usability, a heuristic evaluation should be conducted. A

heuristic evaluation generally requires research of the website, inquiry methods, user surveys,

and feedback reports, and the application of human factors concepts to the design. Thereafter,

suggestions can be put forth to be implemented into the redesign.

The stigma that is widely accepted by the general public is that a trip to the DMV is always

unpleasant. Improving the DMV website can help to make the experience of the DMV as a

whole more enjoyable. It is also important to keep in mind that many users of the California

DMV website do not visit the website frequently or are visiting the site for the first time.

Therefore, many users will not be aware of the location of the information they are searching

for until they become more familiar over multiple uses. With this in mind, it is important to

present information in a way which makes the user feel confident that their first attempts and

decisions are correct for navigating and using the websites features.

3

Executive Summary

DMV.CA.gov is a government website dedicated to providing information regarding the

Department of Motor Vehicles and laws and procedures involving road vehicles. All necessary

documents pertaining to drivers and vehicles should be easily accessible from any location of

the website. It is generally important to assume that the users of the DMV website will have a

very wide age range and language abilities between individuals. Anyone from the age of 16

years and older is eligible to obtain a driver’s license and English language ability is not required

to obtain a driver’s license or to own and operate a motor vehicle.

The heuristic evaluation process began by getting acquainted with the California DMV website

and attempting to explore the site as an average user would. Some of the features of the

website which were observed first were the use of menus, real estate, navigation between

different areas of the website, and finding features relevant to the goals a DMV would normally

allow. Some of these goals include finding information about vehicle registration, drivers

licensing, and finding a DMV office.

To establish a starting point for the evaluation, a survey was distributed to users which included

questions relating to the usability of the California DMV website. Interviews were also

conducted to investigate the usability from the perspective of the users. The survey was

presented after the participants were given four tasks to complete. Based on the tasks, eight

qualitative, multiple choice questions were given. Next, interviews were conducted with users

who had used the website on their own. The questions were open ended about the satisfaction

of the site, and what they thought could be improved about certain aspects of the site.

4

After becoming somewhat more familiar with the website, a comparison between similar

websites like the Illinois DMV (http://www.dmv.org/il-illinois/) and the Washington DMV

(http://www.dol.wa.gov/) were inspected. The major differences between websites, such as

real estate use, menu titles, and overall site interface brought features forward that may

contribute to a better user experience. Both the Illinois and Washington DMV websites had

usability similarities that differed from the California DMV, such as larger links and menu items,

the dedication of different levels of real estate for certain features, better use of organization

and grouping, and navigational tools to accommodate better navigation between pages.

Finally, after all inquiry methods were completed, the California DMV website was evaluated

based on human factors usability principles and heuristics. Some of the issues presented are

local and present in specific pages of the website while others are global and present across

many pages throughout the website.

5

Inquiry Methods

Survey

To gather information and perspective on user opinions of the California DMV website, a survey

was given to five participants. Before beginning the survey questions, four tasks were provided

for the participants to carry out. These four tasks included finding the closest DMV office to a

central location, finding information about registering a vehicle, finding information on how to

obtain a driver’s license, and finding information on how to transfer a car title. The questions

were related to task difficulty, frustrations, and general opinions of their experience.

The following is a brief summary of the survey results:

- 2 out of 5 users felt that the website was somewhat difficult to use

- 2 of the 5 participants reported not fully completing the tasks given

- 2 out of 5 participants responded with the answer stating it took more time than

expected

- 2 out of 5 participants found that the information was somewhat unclear.

- 2 out of 5 participants felt lost while navigating the website.

- 0 out of 5 participants found any difficulty returning to a page they had previously

found.

- How to obtain a driver’s license as a new driver, finding the closest DMV, or finding

information on how to transfer a car title was reported to be a very difficult task to

complete by at least one person.

6

- Finding California registration information was reported to be somewhat difficult.

The results of the survey indicate several usability issues which will be covered by the

heuristic evaluation. The full list of survey questions can be found in the appendix, Figure 1.

7

User Interview

An interview with 6 real-world users of the dmv.ca.org website were conducted to obtain more

expansive answers, compared to the survey feedback. All of the participants of the interviews

were between ages 21 and 24. Two of the six users recently moved to California in the past

three months. The following are important highlights from the responses of the interviews:

- “At first glance, I wasn’t able to find links for what I wanted”

- “It was hard to find where the closest DMV was. There didn’t seem to be an easily

accessible place to search for DMVs near my address”

- When you hover over the categories at the top of the page and the dropdown menu

comes down, it’s hard to keep your curser in the drop down box because it’s so skinny”

- I think the site is pretty s***** in some ways. The different types of registration (e.g.

new driver, from out of state, renewal) are very confusing. As someone coming in from

out of state I feel like the information needed to get a new license and re-register my

car is split into at least 3 different places. They have an "out of state" section but it isn't

really helpful.

- Make the most useful and popular information (how to get a license, nearest DMVs)

more prominent on the home page.

User feedback is very valuable for finding issues within an interface. Although users do not

always know the best solutions for certain problems, it is important to take all feedback into

consideration. Unfortunately, due to the time constraints and timing within the evaluation

process, both the interview and the survey do not cover all issues which will be mentioned in

8

the evaluation report. Ultimately, not all issues can be identified through user feedback and can

instead be brought forward through human factors principles.

9

Global Issues

A global issue is an issue which is present across several locations or aspects of a website and

violates a human factors principle or heuristic.

Main Menu and Submenus

Error Prevention – Eliminate error-prone conditions or check for them and present users

with a confirmation option before they commit to the action.

Above is an image of the home screen of the California DMV website with the cursor hovering

over the “Home” tab. When the tabs on this menu are scrolled over, a submenu appears

underneath all of the tabs horizontally.

10

If the cursor moves outside of the submenu, the submenu disappears and the cursor must

return to the parent “home” tab to restart the process. This aspect violates the Error

Prevention heuristic by forcing the user to be very accurate in their mouse movement.

For example, If someone wants to choose “Seniors” under the “Home” tab, if the cursor slips

out of the grey area, they will need to return to “Home” and hover past “Featured Links,”

“News Room,” “Industry,” and “Teens” in the very thin space given. This task would specifically

be difficult for a senior due to the required fine motor skills required to hover in a small

designated area of the screen.

Similarly, if a different tab is hovered over, the submenu for that tab will immediately replace

the last submenu, as shown by the Offices tab:

11

This issue was also mentioned specifically by a user in an interview: “When you hover over the

categories at the top of the page and the dropdown menu comes down, it’s hard to keep your

cursor in the drop down box because it’s so skinny.” It is clear that this menu design causes

difficulty for some users.

Redesign Recommendation: Resembling the Washington DMV website interface, listing the

submenu items vertically for each tab, instead of horizontally, would give the users a larger

area to move their cursors compared to the California DMV website, as shown:

12

Navigation & Mapping – Give the user a reference point to where they are in the

interface and have consistency between pages to allow for easy navigation.

As reported by the user survey results, 2 out of 5 participants reported difficulties while

navigating the California DMV website. A likely contributor to this issue is the absence of a

navigational tool to show the location of pages within the interface. For example, if a user clicks

on “Vehicle Registration” from the home page:

13

Then clicks “Changing Vehicle Ownership”

And finally clicks on “Requirements for Buying a Vehicle”

They will find themselves at the following page, titled “Buying or Selling a Vehicle Changing

Vehicle Ownership”:

14

The navigation issue surfaces when the users want to navigate to a page previously visited in

the interface. To quickly return to a previous page, the users must either find the page they

have in mind by using the back button or backspace, or they will need to return to the home

page and restart their navigation, both of which rely on short term memory. Additionally, every

new page visited feels like a standalone page, separated from the California DMV website as a

whole.

15

Redesign Recommendation: The Illinois DMV website has implemented a navigational tool for the

website which allows the user to move to one of the parent pages of each site they visit, like so:

In this screenshot, the user is in the “Title Transfers” page, but they can easily move back to the

“Registration & Titling,” “Illinois,” or “Home” page. Without this tool, the user is likely to feel lost

and have to repeat the navigational process if they want to find a page they are already aware of.

Implementing a navigational tool like the one used by the Illinois DMV website would require the

pages of the current California DMV website to be reorganized in a way which includes more

parent and child pages. The inclusion of a navigational tool would give the user a sense of location

within the website while simply providing a useful feature.

16

Organization and Difficulty Finding the Right Questions

Grouping – The grouping principle is used in interface design to position items that are similar

to each other in close proximity.

Throughout the California DMV website, there are pages with links which are intended to help

users find the answers to their questions. However, many of these links are in the form of a

long question, such as, “Where can I locate information regarding the Vehicle License Fee

Increase?” or “What are personalized and special interest license plates?” Both of these

questions may be useful if answered, but due to the poor grouping of these questions, finding a

relevant question becomes a difficult task. Here is one section of the FAQ page:

17

Although all the questions may be relevant to the section title of “Vehicle Registration and Title

Questions,” the links could be grouped into additional subgroups with titles based on the

subcategory they all fit into. For example, there are many questions regarding registration fees,

vehicle license fees, income taxes, and other monetary questions. However, all of the questions

that would fit into this monetary category are disorganized and separated from each other

throughout the list.

Redesign Recommendation: To follow the grouping heuristic, group questions with similar

subjects together while also providing more subheadings to make finding specific questions

easier for the user.

18

Local Issues

A local issue is an issue that is centralized at a location, usually only at one page of a website.

These issues can be the interface of a webpage or a tool or feature within a webpage.

Inefficient Use of the Home Page Real Estate

Real Estate refers to the actual visual space being utilized on the screen by the interface and its

separate features. Because there is a limited amount of space which can be viewed at one time

on a computer screen, the space on the screen must be consolidated appropriately to help

users achieve their goals as quickly as possible. On the home page, there is a large banner

which uses about half of the real estate of the screen.

19

This is very inefficient for providing relevant information to the users. Furthermore, even if the

information on the banner is relevant, the users are probably not going to stall and wait until

they see an image that seems similar enough to their goals.

Redesign Recommendation: An example of efficient real estate use and a better banner

design can be shown with the Illinois DMV homepage:

Pictured above is the first view a user will see of the Illinois DMV website. All of the

information displayed on the homepage is relevant to common user needs, such as the large

search bar, the main menu, and the side bar containing frequently used items. And although

there is still a banner with cycling images, the banner takes up less than half of the home

page real estate. The banner also provides titles for each cycling image, which can be

viewed at the user’s will.

20

Unnoticed “Important” Alerts

Efficient use of real estate is violated again by the “Important California DMV Alerts” section.

This section seems to be a critical part of the home page, but it will only be seen by those who

scroll to the second half of the page:

If these alerts are important, then why isn’t this information shown on a more visible part of

the real estate where the users can see it immediately?

Redesign Recommendation: Put the most important information on the real estate of the

screen which receives the most attention. Also, use a color which stands out from the

background to make the alert more salient, such as red instead of grey.

21

Appointment Feature Needs More Visibility

Because the DMV has an established reputation to be a less-than-enjoyable experience, giving

the appointment-making feature a more prominent location may improve the user experience.

Currently, the link to make an appointment is under “Online Services” outlined in red:

This feature isn’t only difficult to come across because of its location and size, but it is also

located in a submenu of a three part menu system, which is another issue on its own.

Redesign Recommendation: To improve the visibility of the appointment making feature,

there should be more real estate dedicated only to this feature.

22

Making an Appointment

Since many people are going to expect their visit to the DMV to be unpleasant, making any

adjustments to make their visit go smoothly would be beneficial. One of the reasons a visit to

the DMV can be unpleasant, but can be improved upon, is the long waiting time in line.

Therefore, having more people set up appointments for their visit would improve the

experience for all users and possibly the employees, as well. Here are the current steps to

making an appointment:

Begin at the Homepage:

23

Scroll to the middle of the page and click on “Make an Appointment before going to a DMV

Field Office”:

Choose the appointment type (Behind-the-wheel driving test, Office visit, View/Cancel

appointment):

24

Choose or find the DMV office you would like to visit:

If someone is planning on visiting a DMV office for the first time, just being given the town

name, such as San Jose, is not specific enough to know if it is the closest DMV to the user’s

address. In fact, the user probably has no idea where the office is located yet. Therefore, the

users are likely to use the “Map CA DMV Offices” link.

25

When the “Map CA DMV Offices” button is clicked, a new window appears with a map of

northern and southern California:

The town names in red are clickable and used to further narrow down the area of the map.

26

In this case, San Jose was selected and the window was refreshed to the following map:

In this map, only the red towns are clickable while the towns in black are not. However, the

map only shows one DMV office per town instead of all DMV offices within the map area.

27

Once a red town is clicked on, the window automatically closes and the user is left viewing the

appointment-setting screen with the selected town now chosen in the drop-down menu.

28

Although a DMV office has been chosen by the user, the address of the office has not been

provided, which violates the help and documentation heuristic as information should be easy to

search for and should focus on the user’s goals. The user is expected to continue setting the

appointment without knowing the location of the office.

Redesign Recommendation: Following the help and documentation heuristic, the map function

to find a DMV office should use a format similar to the “Find Your Local DMV Field Office” map.

This map provides all DMV locations on a map that can be moved by clicking and dragging. By

using this map, the user will immediately know where the office is located by the map location

and the address provided.

29

Once the location has been selected, the purpose for the appointment must be chosen and the

user must fill out his or her name and phone number.

Redesign Recommendation: The dropdown menu should have the address displayed next to

the town name. There is available real estate to display the full address next to the

dropdown menu once a location has been selected.

30

Then, “submit” at the bottom of the page must be clicked to continue the process.

Redesign Recommendations: A symbol, such as an asterisk, should be used to denote the

mandatory information needed so the form is properly completed on the first attempt. This would

follow the Error Prevention heuristic, preventing problems or errors from occuring in the first place.

31

After clicking “submit” the user will be brought to the next page of the appointment-setting

process.

This page displays relevant information, such as the unavailability of written exams after

4:30pm as well as a reminder stating what this appointment is not being made for. In this case,

the appointment is not for a behind-the-wheel test, but an appointment can be made for one

using the available link. The page also now tells the user the address of the DMV they are

Additionally, the “Submit” button violates the Consistency and Standards heuristic which

states users should not have to wonder whether different words mean the same thing.

Clicking on “submit” does not submit an appointment. Instead, the user is taken to the next

page and therefore, a word like “continue” would be a more appropriate word choice.

32

setting an appointment for and the first available appointment is automatically set as the

default appointment time.

Although this may inform the users of the soonest available time for an appointment, chances

are that they will want to choose specific times and days which are compatible with their

schedules. To pick a date, there is a calendar that can be used to choose a specific day as well as

a specific time.

33

Redesign Recommendation: Although the soonest available time is displayed, there is an

absense of cross-pollination, which is the maintaining of information across aspects of the

same website. To support the cross-pollination principle, the calendar that appears should

not allow the user to click on any dates before the soonest possible appointment time, as it

already has been stated by the website.

34

Once date and time are chosen, the user must check for availability. If the selected time is

available, a message in red states “an appointment for the date and/or time selected is

available” and the user may click on “Schedule This Appointment.”

Redesign Recommendation: To better follow an approprite use of affordances, which are

qualities of objects which suggest certain meanings, choosing an available appointment time

should be shown in green. It would not be surprising to find that many users assumed the time

they have selected is not available due to seeing the color of the text before reading the actual

message.

35

A more efficient way of presenting available dates and times for appointments would be to have

a calendar showing all available times within any given day, similar to a spreadsheet or Google

Calendar to visually aid the user and provide more useful information.

36

Another notable issue within the appointment-setting process is the session time-out. If the

user takes too much time on the appointment page, an error screen will appear stating “Session

timed out, your session has timed out due to inactivity.”

Redesign Recommendation: To help the user return to making an appointment, a link

should be provided to redirect them to a usable page.

37

Three Unnoticed Submenus

While becoming familiar with the website, there were three submenus which went unnoticed

as three separate information sources. It wasn’t until a few weeks into the process that the

three different menus were discovered. These menus include Online Services:

Media Center:

38

And Other Services:

If it took one person several weeks with many visits to realize this menu was here, then it will

likely never be used by many users as they will visit the site only a few times within a short

period. Additionally, displaying so much information in so little space creates real estate which

is too dense with text. Users are very unlikely to focus so much attention onto one small part of

the screen, especially when the users are very goal-driven and the density does not seem to

support their goals.

39

Redesign Recommendation: If the developers of the website feel that all of these features

are necessary to be shown on the first page, the grouping should be improved

dramatically. Grouping organizes objects with similar features into close proximity to one

another. For example, Important Info for Boat Owners, Tell Us How We’re Doing, and

Pedestrian Safety Info, and Office Closures have very little to do with each other and only

one of those links seems to have the importance of an “alert.”

The Latest Press Releases, Major Events Affecting Drivers, DMV Newsroom, and Subscribe

to Email Alerts all have the commonality of new and relevant information for the public.

Instead, all four of the links just mentioned are separated between three different

submenus. However, since most of these links can be found in the main menu, removing

them from this area of the site would decrease the redundancy in design as well as free

more real estate.

40

The “How Do I…” Menu

On the Home page of the California DMV website, there is a “How Do I…” scrolling menu which

is basically a Frequently Asked Questions section, as shown:

Similarly to the three submenus above it, grouping issues are present within this menu. These

items have been listed alphabetically, which makes finding the right question. As an example, if

a user is looking for an item about Teen Drivers, the user would have to begin reading each

item completely before determining whether or not it is a match. Finally, when the user gets to

the 8th item of the “Get information” section, they will find the “get information about teen

drivers” item.

Additionally, this menu also includes a redundancy component since there is also a “Frequently

Asked Questions” Section within the “Other Services” submenu. However, as it has already

been covered, the users are not likely to view the “Other Services” menu to find the link to the

FAQ page because of the density and lack of organization or grouping. Ironically, the FAQ

41

section incorporates some grouping principles, which could have been incorporated into the

“How Do I…” menu, like so:

42

Redesign Recommendation: As the FAQ page has already provided, the “How Do I…”

Section should also incorporate the same organization within the scroll menu, instead of

having the choices listed alphabetically. Furthermore, to provide more flexibility within the

website, while also reducing the redundancy, a link to the full FAQ page could be provided

next to the “How Do I…” menu title. This reduces the redundancy by using no additional

real estate for the same goal, while still allowing the user to choose which view to be

displayed.

43

Finding a DMV Office Location

The following is the process to find a DMV Office Location:

First, the user starts at the Home page, hovers over “Offices” and clicks on “Find Your Local

Office”:

44

Then, the user clicks DMV Field Offices Map, under the Interrupted Services and Closure

Information:

Redesign Recommendation: The salience of the office closures should be improved by using

red instead of blue for the outline. Also, the chat bubble has been changed to an

exclamation mark, which signifies urgency.

45

After clicking on the link for DMV Field Office Map, the user is brought to a page displaying a

map of California. The site is capable of using the internet connection to find the user’s

location. The map is based on Google Maps and displays all California DMV office locations:

The bullet points of information should be shortened to compensate for faster skimming. For

example, instead of, “All DMV Offices will be closed Thursday, November 28, 2013 and the

following day in observance of Thanksgiving Day,” a simpler version such as, “All DMV Offices

will be closed Thursday, November 28 and Friday, November 29 for the Thanksgiving

holiday.” This way, both Thursday and Friday listed, as some users may not notice “the

following day” is also closed while quickly skimming. Also, by changing “in observance of

Thanksgiving Day” to “for the Thanksgiving holiday,” the former suggests a single day while

the latter does not imply only one day and also uses ordinary language, following the

common sense principles.

46

Redesign Recommendation: This is not a critical issue, but it does violate the redundancy

and real estate issues. At the top of the page, “DMV Field Offices” is displayed at the top of

the page while “DMV Office Locations” is displayed again at the top of the map while also

making part of the map unusable, as shown:

Removing the top header on the map would resolve both of these issues.

47

The user can click on one of the locations to view more information about each office. This

information includes the address, wait times with and without appointments, and open hours

of the location, like so:

However, there is no way of making an appointment at an office without using the “make an

appointment” feature, which is completely separated from this page of the website.

48

Redesign Recommendation: The majority of users are going to be using the map feature to

plan a trip to a DMV office. Since wait times for appointments and non-appointments are

already listed in the details of the location, making an appointment should be accessible

from this page. A “make an appointment now” link under the wait times could be created

to follow visibility and grouping principles.

Another helpful feature would include a “Now open” or “closed” icon, which supports

giving the user more useful and relevant information to their goals, as supported by the

help and documentation heuristic.

49

If the “details” tab is clicked, closure alerts, services, and additional services are displayed:

Unfortunately, many users will not choose to click on the “details” tab after getting the

information they were searching for and the alert of the office closure will remain unseen.

Although this alert will likely be on the “Find Your Local Office” page, if the user is quickly

searching for the office map, it is reasonable to assume they may not stop to read the office

closures, especially with the current design.

Although finding a DMV office through the DMV Office Map is fairly straightforward,

redesigning the functionality with these issues in mind will have a significant positive impact on

the overall usability of the features and usability of the task in its entirety.

Redesign Recommendation: To improve the visibility of the office closure alert, the alert

should be displayed at the top of the first tab opened when any office has a planned closure,

change in open hours, or any other status change which could affect the user.

50

Conclusion

The objective of a successful website redesign is to make the experience of the website easy

and intuitive for the users. It is important to assume that differences between people will vary

greatly on dimensions of age, language ability, and experience of using technology in general. It

is widely known that a trip to the DMV is generally unpleasant and finding ways to make the

users of the DMV as enjoyable and stress free as possible. The users of the DMV are not likely

to be strongly familiar with the layout or the included information. Therefore, the website

should be designed with a user centered perspective and the interface should present itself in a

way that can be quickly learned within the first moments of being used.

Although the current design is somewhat aesthetically pleasing, there is much to be improved

upon. The first recommendation would be to rebuild the entire architecture of the website to

integrate better navigation and mapping. Secondly, the real estate of the main menu should be

reorganized to allow users to make more slight errors without consequences. Grouping is also a

top priority for the redesign. Throughout the entire site there are examples of lists which users

must sift through on their own to find any relevant information to help them reach their goals.

Finally, redundancy and clutter seems to be a common theme on the homepage. The main

menu is somewhat usable, but the other side menus are not useful due to their text density and

lack of grouping. The banner also needs to give up some of its real estate for more practical

purposes.

Having a usable appointment system is an excellent feature for a DMV, but the system could

use some improvement. Moreover, the availability of an appointment system should be made

51

much more clear as it is somewhat hidden in the website as it stands. Some of the messages

about appointments as well as office closures may go unnoticed due to low salience or

misinterpretation caused by inappropriate color schemes or location. The map function within

the appointment system was difficult for several users, although the separate map function for

finding an office location was much better. Therefore, if the separate map could replace the

current map within the appointment system, the process could be made much easier.

Although not a crucial issue, the information presented throughout the site could be reworded

to use closer to everyday language. Because about 90 percent of Californians either can speak

English or Spanish, the languages available on the website do not present a critical issue.

However, making the site easier to find the different language DMV handbooks or providing

more languages might benefit a small percent of the population.

52

Appendix

Figure 1

53

Contact Info:

Ryan Schabel Schabel12@gmail.com (224) 619-5016 San Jose State University