Evaluating myAT&T RedesignConner Drew

Types of Evaluation

1. Think Aloud Protocol

2. Cognitive Walkthrough

3. Heuristic Evaluation

Think Aloud Protocol

Think Aloud Protocol

Evaluate the usability of designs by encouraging a user to think out loud as they use your product.

• Understand the sequence of thoughts people have as they solve a problem.

• Studies found that there is no affect on thought sequences, as long as there is no introspection.

• Does not change the outcome of a task.

• “Please keep talking.”

Cognitive Walkthrough

Cognitive Walkthrough

Evaluate the learnability of a product, based on a theory of problem solving in unfamiliar situations.

• Intended to be used with the mentality of a first time user.

• Confirms the system could be used without any prior training.

• Task-based approach.

Cognitive Walkthrough

Tasks:

1. Compare your current usage to other data plans, then change your data plan accordingly.

2. Upgrade your current device to the new iPhone 7.

3. Suspend your phone because it was stolen while you were on vacation.

4. Process a insurance claim because you cracked your screen.

5. Make a payment on your current wireless bill and enroll in auto-pay.

6. Update your account’s password.

Cognitive Walkthrough

Questions:

1. Will the user try to achieve the right effect?

2. Will the user notice that the correct action is available?

3. Will the user associate the correct action with the effect that user is trying to achieve?

4. If the correct action is performed, will the user see that progress is being made towards a goal?

25 points

Unique Identifer Unique Screen ID Problem Description Severity (1-5, High) Frequency (1-5, Common) Proposed Solution

C-CD_1 Plan 2.A The “change data plan” CTA could be out of view when the user decides to take action.

3 1 Allow the user to select the data plan to change before the bottom of the list.

C-CD_2 Plan 3.A There is no title to section off data plan choices. 1 1 Add a title to the data plan section, or eliminate it all together and let the user choose the plan on the previous screen.

C-CD_3 Plan 3.1.A The phrasing “Update Plan” could be more clear 2 1 Change “Update Plan” to something more descriptive.

C-CD_4 Upgrade Home.A It may not be completely clear to navigate to devices before upgrading.

3 2 Add a shopping section or add an item to navigate to devices from account

C-CD_5 Upgrade 1.A “View Device Details” CTA may be in reference to hardware specs not available options in relation to the plan.

3 2 Combine Upgrade 1 and Upgrade 2 screens

C-CD_6 Upgrade 4.A The user may not know exactly all of the details when choosing a device.

2 1 Give a brief overview about what steps are involved when selecting a device, or give an info icon that displays a description about each section, especially the capacity.

C-CD_7 Upgrade 4.1.A The user may not understand what the grayed out capacity means before selection

1 1 Give a further description that the item is out of stock, or give an expected time frame for when the item will be back in stock.

C-CD_8 Upgrade 5.A The two Apple plans are irrelevant with AT&T 3 1 Remove the Apple options

C-CD_9 Suspend Home.A It may not be completely clear to navigate to devices before upgrading.

3 2 Make the option to suspend also live within the account settings

C-CD_10 Suspend 1.A “View Device Details” CTA may be in reference to hardware specs not available options in relation to the plan.

3 2 Combine Upgrade 1 and Upgrade 2 screens

C-CD_11 Warranty Home.A The user may not associate being able to submit an insurance swap from the device itself.

4 1 Educate the user before this incident occurs

C-CD_12 Warranty 1.A “View Device Details” CTA may be in reference to hardware specs not available options in relation to the plan.

3 2 Combine Upgrade 1 and Upgrade 2 screens

C-CD_13 Warranty 2.A “Create Claim” is a little ambiguous in regards to what the action is going to do.

3 1 Update CTA to say “Submit Insurance Claim” or “Replace Device”

C-CD_14 Warranty 3.1A Even though the radio selector is heavy and indicates a selection, it could be further communicated that this was selected.

1 3 Decrease the opacity of the options that were no selected to emphasize the selection

C-CD_15 Warranty 3.1B The “Continue” CTA doesn’t explain exactly what is is happening or will happen. It’s ambiguous.

2 1 Change “Continue” to “Review Insurance Claim”

C-CD_16 Warranty 4.A Even though is a review, there is not a confirmation that this is in fact what the user would like to do.

3 1 Add a notification alert between screens Warranty 4 and Warranty 5 to confirm the user does in fact want to submit an insurance claim.

C-CD_17 Warranty 5.A The messaging says the user’s “item” has shipped. This is a little ambiguous.

1 1 Explicitly call out what device is being shipped, such as iPhone 7.

C-CD_18 Payment Home.A The user might not immediately see the bill tab since you start on the usage tab.

2 1 Add a home feed option outlining the most important options

C-CD_19 Payment 2.1.A Even though the radio selector is heavy and indicates a selection, it could be further communicated that this was selected.

1 3 Even though the radio selector is heavy and indicates a selection, it could be further communicated that this was selected.

C-CD_20 Payment 2.1.B The user may only have the intention to pay their bill and may miss the AutoPay Enrollment, or not know exactly what it means

1 1 Add an info icon allowing more explanation to be displayed.

C-CD_21 Payment 3.A There isn't much indication that the payment was successful other than the amount being reduced to $0.

3 1 Add a notification banner at the top similar to the other success messages.

C-CD_22 Security 1.A “Log In Settings” indicates that it includes more than username, email, and password options.

2 1 Change word choice to include specifics, such as username, email, or password.

C-CD_23 Security 2.A User ID is arguably not needed since there is an email associated with the account for log in purposes.

1 1 Remove the user ID field.

C-CD_24 Security 3.A The user may not be familiar with why they need to enter their current password.

2 1 Add an info icon explaining why the current password is needed to update their password.

C-CD_25 Security 3.B The screen title says “Login Settings” even though the screen has switched to updating the password.

2 2 Update Login Settings to read “Update Password”

Unique Identifer Unique Screen ID Problem Description Severity (1-5, High) Frequency (1-5, Common) Proposed Solution

C-CD_1 Plan 2.A The “change data plan” CTA could be out of view when the user decides to take action.

3 1 Allow the user to select the data plan to change before the bottom of the list.

C-CD_2 Plan 3.A There is no title to section off data plan choices. 1 1 Add a title to the data plan section, or eliminate it all together and let the user choose the plan on the previous screen.

C-CD_3 Plan 3.1.A The phrasing “Update Plan” could be more clear 2 1 Change “Update Plan” to something more descriptive.

C-CD_4 Upgrade Home.A It may not be completely clear to navigate to devices before upgrading.

3 2 Add a shopping section or add an item to navigate to devices from account

C-CD_5 Upgrade 1.A “View Device Details” CTA may be in reference to hardware specs not available options in relation to the plan.

3 2 Combine Upgrade 1 and Upgrade 2 screens

C-CD_6 Upgrade 4.A The user may not know exactly all of the details when choosing a device.

2 1 Give a brief overview about what steps are involved when selecting a device, or give an info icon that displays a description about each section, especially the capacity.

C-CD_7 Upgrade 4.1.A The user may not understand what the grayed out capacity means before selection

1 1 Give a further description that the item is out of stock, or give an expected time frame for when the item will be back in stock.

C-CD_8 Upgrade 5.A The two Apple plans are irrelevant with AT&T 3 1 Remove the Apple options

C-CD_9 Suspend Home.A It may not be completely clear to navigate to devices before upgrading.

3 2 Make the option to suspend also live within the account settings

C-CD_10 Suspend 1.A “View Device Details” CTA may be in reference to hardware specs not available options in relation to the plan.

3 2 Combine Upgrade 1 and Upgrade 2 screens

C-CD_11 Warranty Home.A The user may not associate being able to submit an insurance swap from the device itself.

4 1 Educate the user before this incident occurs

C-CD_12 Warranty 1.A “View Device Details” CTA may be in reference to hardware specs not available options in relation to the plan.

3 2 Combine Upgrade 1 and Upgrade 2 screens

C-CD_13 Warranty 2.A “Create Claim” is a little ambiguous in regards to what the action is going to do.

3 1 Update CTA to say “Submit Insurance Claim” or “Replace Device”

C-CD_14 Warranty 3.1A Even though the radio selector is heavy and indicates a selection, it could be further communicated that this was selected.

1 3 Decrease the opacity of the options that were no selected to emphasize the selection

C-CD_15 Warranty 3.1B The “Continue” CTA doesn’t explain exactly what is is happening or will happen. It’s ambiguous.

2 1 Change “Continue” to “Review Insurance Claim”

C-CD_16 Warranty 4.A Even though is a review, there is not a confirmation that this is in fact what the user would like to do.

3 1 Add a notification alert between screens Warranty 4 and Warranty 5 to confirm the user does in fact want to submit an insurance claim.

C-CD_17 Warranty 5.A The messaging says the user’s “item” has shipped. This is a little ambiguous.

1 1 Explicitly call out what device is being shipped, such as iPhone 7.

C-CD_18 Payment Home.A The user might not immediately see the bill tab since you start on the usage tab.

2 1 Add a home feed option outlining the most important options

C-CD_19 Payment 2.1.A Even though the radio selector is heavy and indicates a selection, it could be further communicated that this was selected.

1 3 Even though the radio selector is heavy and indicates a selection, it could be further communicated that this was selected.

C-CD_20 Payment 2.1.B The user may only have the intention to pay their bill and may miss the AutoPay Enrollment, or not know exactly what it means

1 1 Add an info icon allowing more explanation to be displayed.

C-CD_21 Payment 3.A There isn't much indication that the payment was successful other than the amount being reduced to $0.

3 1 Add a notification banner at the top similar to the other success messages.

C-CD_22 Security 1.A “Log In Settings” indicates that it includes more than username, email, and password options.

2 1 Change word choice to include specifics, such as username, email, or password.

C-CD_23 Security 2.A User ID is arguably not needed since there is an email associated with the account for log in purposes.

1 1 Remove the user ID field.

C-CD_24 Security 3.A The user may not be familiar with why they need to enter their current password.

2 1 Add an info icon explaining why the current password is needed to update their password.

C-CD_25 Security 3.B The screen title says “Login Settings” even though the screen has switched to updating the password.

2 2 Update Login Settings to read “Update Password”

Heuristic Evaluation

Heuristic Evaluation

Compare an interface to an established list of heuristics, or best practices, to identify usability problems.35 points

1. Visibility of system status

2. Match between system and the real world

3. User control and freedom

4. Consistency and standards

5. Error prevention

6. Recognition rather than recall

7. Flexibility and efficiency of use

8. Aesthetic and minimalist design

9. Help users recognize, diagnose, and recover from errors

10.Help and documentation

25 points

Heuristic Evaluation

Unique Identifer Unique Screen ID Problem Description Evidence - Heuristic Violated Severity (1-5, High) Frequency (1-5, Common) Proposed Solution

H-CD_1 Plan Home.A The current billing cycle graph isn’t as clear as it could be

Recognition rather than recall 3 3 Update the graph into something more digestible, such as a circle graph focusing on the days left and the amount of the data used.

H-CD_2 Plan Home.B The user may get confused about what a down facing arrow means

Recognition rather than recall 2 5 Update the down arrow to be a right arrow and display a new screen with the information rather than expanding to show more.

H-CD_3 Plan 3.2.a The user may have more detailed questions about a plan change than what fits into the alert modal

Help and documentation 3 2 Present the modal within its own screen to allow more explanation to be provided. Also, create an info icon that hides the detailed information unless the user wants it.

H-CD_4 Upgrade 1.A There is a greyed out arrow shown when the first device in a a carousel is displayed, not really providing any value

Aesthetic and minimalist design 1 2 Remove the left arrow until the user actually swipes/taps to see the next device. It’s useless until then.

H-CD_5 Upgrade 1.B “View Device Details” may not match what people are used to associating with what is displayed next.

Match between system and real world 3 3 “View device options” may be a more accurate representation of functionality. Upgrade 1 and Upgrade 2 may also be combined into one screen, eliminating the needs for the CTA altogether.

H-CD_6 Upgrade 2.B The word “device” is used in two CTAs, and even though it’s in context of a device view, it’s not as accurate as it could be.

Match between system and real world 1 1 Update “device” to the device model/nickname since this information is known.

H-CD_7 Upgrade 3.A The down arrow for the filter options could be more descriptive

Recognition rather than recall 1 1 Use words such as “update filter” or “apply filter” would make the action more clear.

H-CD_8 Upgrade 3.B The arrows to indicate selection could be more clear Recognition rather than recall 1 1 Instead of using an arrow to indicate you can tap the device use a CTA that says “view more details”

H-CD_9 Upgrade 4.A If the user decides to stop the entire upgrade process, they have to tap multiple times to cancel the flow.

Aesthetic and minimalist design 2 2 Add a cancel button to the navigation in order to cancel the upgrade flow within one tap, returning to the device view.

H-CD_10 Upgrade 5.1.A There is an intended “review” capability on this screen, but there are still options being selected at this stage. It could feel very quick.

Help users recognize, diagnose and recover from errors

3 1 Add one more screen to review then confirm the upgrade order. Change CTAs to reflect current stage in the process.

H-CD_11 Upgrade 6.1 The user may skip over the track package option Recognition rather than recall 3 1 Include a larger CTA that calls our tracking the package more clearly.

H-CD_12 Suspend 2.A There is currently no explanation of what suspend means.

Help and documentation 3 1 Instead of using a modal, use an additional screen that provides more description of why you might want to suspend a device and the implications behind doing so.

H-CD_13 Suspend 3.A To unsuspend a device the user will tap a CTA that reads “reactivate”, but that potentially infers activating a new phone.

Match between system and real world 1 1 Change “reactivate” to “unsuspend” or something equivalent.

H-CD_14 Warranty 2.A “Create Claim” is a little ambiguous in regards to what the action is going to do.

Match between system and real world 2 1 Update CTA to say “Submit Insurance Claim” or “Replace Device”

H-CD_15 Warranty 3.1.A “Continue” CTA doesn’t describe exactly what continuing would do.

Match between system and real world 2 1 Update CTA to be more descriptive about the specific action

H-CD_16 Warranty 3.1.B The user may have forgotten about their plan needing a deductible and be confused as to why there is a charge.

Help and documentation 2 1 Include an info icon to allow the user to find out more information as to why there is a charge/deductible.

H-CD_17 Warranty 4.A There is no description about what the swap process will look like until after the claim is submitted. Even then it’s pretty vague.

Help and documentation 4 1 Provide an overview description about what will happen during the replacement process.

H-CD_18 Warranty 4.B The price is missing from the review screen Recognition rather than recall 3 1 Add the price of the warranty swap

H-CD_19 Warranty 5.A The user may not know why the track shipment CTA is greyed out. This is because the item hasn’t shipped yet.

User control and freedom 2 1 Add a description as to why the track shipment is greyed out or make the CTA active and allow the tracking system to explain that the device has not shipped yet.

H-CD_20 Payment 1.A “View previous activity” doesn't describe what kind of activity the user would be reviewing

Match between system and real world 1 1 Add a descriptor to describe billing activity

H-CD_21 Payment 2.2.A There is no description as to what exactly AutoPay is and why it’s useful.

Help and documentation 2 1 Add an info icon further explaining the conditions of AutoPay, or make the description inline, or a combination of the two.

H-CD_22 Payment 3.A There is no official confirmation that payment has been made other than the face the about due has decreased to $0.00

Consistency and standards 2 1 Add a success message at the top that the payment has been made successfully

H-CD_23 Security 1.A View devices could be more descriptive as to all of the options within the menu item

Recognition rather than recall 1 1 Update CTA to read Manage devices, which infers more functionality than simply viewing. This is to accommodate the multiple options you are able to work with within each device.

H-GB_1 Payment 1.A view entire bill is a strange wording, real world language 1 1 change the verbiage to something like view full statement or just view statement would be more indicative.

H-GB_2 Plan 3.1.A the total plan cost isn't listed. standards 2 1 Should have the total plan cost here so the user knows what their looking at per month.

H-GB_3 Plan 3.2.A it feels like there should be another screen after this for success.

error prevention/system status. 2 1 add another screen with bill total and confirmation that the plan was changed.

H-GB_4 Security 2.A too minimal almost. it feels to bare for a user with little knowledge to get around it without an issue

error prevention 2 1 Honestly, idk.

H-GB_5 Suspend 2.A the suspend device option could just change to reactivate or something like that instead of having the reactivate button appear up top

consistency 2 1 just pull reactivate down to the other buttons.

H-KW_1 Payment 1 there is no way for a user to access the homepage form “Bill”

Clearly Marked Exits 1 1 add a back button or home

H-KW_2 Payment 2.3 Customer cannot tell from message which card with be used for autopay

minimize the user’s memory load 3 1 add the last four digits to a card that is on file.

H-KW_3 Homepage Usage as the homepage seems not inline with user’s views

Speak to user’s language 2 5 Consider adding the same homescreen, but not under a tab

H-KW_4 ALL Screens No AT&T logos consistency If this was intentional never mind

H-KW_5 Suspend 2 The phrase “view device options” and the below list of actions seems redundent

prevent errors 1 1 consider renaming the “Device options” to more clearly relay what is behind it.

H-KW_6 Upgrade 3 there is a gray box in the upper right hand corner Simple and natural dialogue 1 1 Remove these or make their purpose more clear

Unique Identifer Unique Screen ID Problem Description Evidence - Heuristic Violated Severity (1-5, High) Frequency (1-5, Common) Proposed Solution

H-CD_1 Plan Home.A The current billing cycle graph isn’t as clear as it could be

Recognition rather than recall 3 3 Update the graph into something more digestible, such as a circle graph focusing on the days left and the amount of the data used.

H-CD_2 Plan Home.B The user may get confused about what a down facing arrow means

Recognition rather than recall 2 5 Update the down arrow to be a right arrow and display a new screen with the information rather than expanding to show more.

H-CD_3 Plan 3.2.a The user may have more detailed questions about a plan change than what fits into the alert modal

Help and documentation 3 2 Present the modal within its own screen to allow more explanation to be provided. Also, create an info icon that hides the detailed information unless the user wants it.

H-CD_4 Upgrade 1.A There is a greyed out arrow shown when the first device in a a carousel is displayed, not really providing any value

Aesthetic and minimalist design 1 2 Remove the left arrow until the user actually swipes/taps to see the next device. It’s useless until then.

H-CD_5 Upgrade 1.B “View Device Details” may not match what people are used to associating with what is displayed next.

Match between system and real world 3 3 “View device options” may be a more accurate representation of functionality. Upgrade 1 and Upgrade 2 may also be combined into one screen, eliminating the needs for the CTA altogether.

H-CD_6 Upgrade 2.B The word “device” is used in two CTAs, and even though it’s in context of a device view, it’s not as accurate as it could be.

Match between system and real world 1 1 Update “device” to the device model/nickname since this information is known.

H-CD_7 Upgrade 3.A The down arrow for the filter options could be more descriptive

Recognition rather than recall 1 1 Use words such as “update filter” or “apply filter” would make the action more clear.

H-CD_8 Upgrade 3.B The arrows to indicate selection could be more clear Recognition rather than recall 1 1 Instead of using an arrow to indicate you can tap the device use a CTA that says “view more details”

H-CD_9 Upgrade 4.A If the user decides to stop the entire upgrade process, they have to tap multiple times to cancel the flow.

Aesthetic and minimalist design 2 2 Add a cancel button to the navigation in order to cancel the upgrade flow within one tap, returning to the device view.

H-CD_10 Upgrade 5.1.A There is an intended “review” capability on this screen, but there are still options being selected at this stage. It could feel very quick.

Help users recognize, diagnose and recover from errors

3 1 Add one more screen to review then confirm the upgrade order. Change CTAs to reflect current stage in the process.

H-CD_11 Upgrade 6.1 The user may skip over the track package option Recognition rather than recall 3 1 Include a larger CTA that calls our tracking the package more clearly.

H-CD_12 Suspend 2.A There is currently no explanation of what suspend means.

Help and documentation 3 1 Instead of using a modal, use an additional screen that provides more description of why you might want to suspend a device and the implications behind doing so.

H-CD_13 Suspend 3.A To unsuspend a device the user will tap a CTA that reads “reactivate”, but that potentially infers activating a new phone.

Match between system and real world 1 1 Change “reactivate” to “unsuspend” or something equivalent.

H-CD_14 Warranty 2.A “Create Claim” is a little ambiguous in regards to what the action is going to do.

Match between system and real world 2 1 Update CTA to say “Submit Insurance Claim” or “Replace Device”

H-CD_15 Warranty 3.1.A “Continue” CTA doesn’t describe exactly what continuing would do.

Match between system and real world 2 1 Update CTA to be more descriptive about the specific action

H-CD_16 Warranty 3.1.B The user may have forgotten about their plan needing a deductible and be confused as to why there is a charge.

Help and documentation 2 1 Include an info icon to allow the user to find out more information as to why there is a charge/deductible.

H-CD_17 Warranty 4.A There is no description about what the swap process will look like until after the claim is submitted. Even then it’s pretty vague.

Help and documentation 4 1 Provide an overview description about what will happen during the replacement process.

H-CD_18 Warranty 4.B The price is missing from the review screen Recognition rather than recall 3 1 Add the price of the warranty swap

H-CD_19 Warranty 5.A The user may not know why the track shipment CTA is greyed out. This is because the item hasn’t shipped yet.

User control and freedom 2 1 Add a description as to why the track shipment is greyed out or make the CTA active and allow the tracking system to explain that the device has not shipped yet.

H-CD_20 Payment 1.A “View previous activity” doesn't describe what kind of activity the user would be reviewing

Match between system and real world 1 1 Add a descriptor to describe billing activity

H-CD_21 Payment 2.2.A There is no description as to what exactly AutoPay is and why it’s useful.

Help and documentation 2 1 Add an info icon further explaining the conditions of AutoPay, or make the description inline, or a combination of the two.

H-CD_22 Payment 3.A There is no official confirmation that payment has been made other than the face the about due has decreased to $0.00

Consistency and standards 2 1 Add a success message at the top that the payment has been made successfully

H-CD_23 Security 1.A View devices could be more descriptive as to all of the options within the menu item

Recognition rather than recall 1 1 Update CTA to read Manage devices, which infers more functionality than simply viewing. This is to accommodate the multiple options you are able to work with within each device.

H-GB_1 Payment 1.A view entire bill is a strange wording, real world language 1 1 change the verbiage to something like view full statement or just view statement would be more indicative.

H-GB_2 Plan 3.1.A the total plan cost isn't listed. standards 2 1 Should have the total plan cost here so the user knows what their looking at per month.

H-GB_3 Plan 3.2.A it feels like there should be another screen after this for success.

error prevention/system status. 2 1 add another screen with bill total and confirmation that the plan was changed.

H-GB_4 Security 2.A too minimal almost. it feels to bare for a user with little knowledge to get around it without an issue

error prevention 2 1 Honestly, idk.

H-GB_5 Suspend 2.A the suspend device option could just change to reactivate or something like that instead of having the reactivate button appear up top

consistency 2 1 just pull reactivate down to the other buttons.

H-KW_1 Payment 1 there is no way for a user to access the homepage form “Bill”

Clearly Marked Exits 1 1 add a back button or home

H-KW_2 Payment 2.3 Customer cannot tell from message which card with be used for autopay

minimize the user’s memory load 3 1 add the last four digits to a card that is on file.

H-KW_3 Homepage Usage as the homepage seems not inline with user’s views

Speak to user’s language 2 5 Consider adding the same homescreen, but not under a tab

H-KW_4 ALL Screens No AT&T logos consistency If this was intentional never mind

H-KW_5 Suspend 2 The phrase “view device options” and the below list of actions seems redundent

prevent errors 1 1 consider renaming the “Device options” to more clearly relay what is behind it.

H-KW_6 Upgrade 3 there is a gray box in the upper right hand corner Simple and natural dialogue 1 1 Remove these or make their purpose more clear

Improving my Designs

First Improvement

First Improvement

Users associated suspending or submitting an insurance claim more with account functionality, rather than device functionality.

Second Improvement

Second Improvement

Users associated suspending or submitting an insurance claim more with account functionality, rather than device functionality.

Second Improvement

Users associated suspending or submitting an insurance claim more with account functionality, rather than device functionality.

Third Improvement

Third Improvement

Users had a difficult time making sense of the status of the data being used during the billing cycle.

They were also trying to tap each individual data plan to update rather than the “Change Data Plan” CTA at the bottom of the screen.

Third Improvement

Users had a difficult time making sense of the status of the data being used during the billing cycle.

They were also trying to tap each individual data plan to update rather than the “Change Data Plan” CTA at the bottom of the screen.

Moving Forward

1. Further Iteration

2. Collaboration with developers

3. More validation testing

Thank You