prepare to readfast
v1.2 FAILURE.QualityFrog.com
Now that I have your attention,
v1.2 FAILURE.QualityFrog.com
You’re testing…
And an error message appears
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Did you expect it?
Or are you surprised?
v1.2 FAILURE.QualityFrog.com
How does it make you feel?
v1.2 FAILURE.QualityFrog.com
What do you do?
v1.2 FAILURE.QualityFrog.com
Not ready reading drive AAbort, Retry, Ignore, Fail?
v1.2 FAILURE.QualityFrog.com
Continue to the next test step…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Report a bug…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Resolve unknown conflict…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Find a file…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Eliminate something…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Request permission…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Try again…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Try something new…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Seek help…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Set a reference to an instance…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
lick OK…C
v1.2 FAILURE.QualityFrog.com
lick OK…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Save…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Enter a multiple of |ECL0…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Exit…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Reconnect…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Declare success…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Assign blame…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Laugh…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Restart…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Guess…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Gasp…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Request a raise…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
Swear…
v1.2 FAILURE.QualityFrog.com
v1.2 FAILURE.QualityFrog.com
So…
v1.2 FAILURE.QualityFrog.com
What do you do when you encounter an
error?
v1.2 FAILURE.QualityFrog.com
While you think about that, let me tell you what I do.
v1.2 FAILURE.QualityFrog.com
Oh, who am I?
v1.2 FAILURE.QualityFrog.com
I am Ben Simo
[email protected]
v1.2 FAILURE.QualityFrog.com
I use software
[email protected]
v1.2 FAILURE.QualityFrog.com
I test software
[email protected]
v1.2 FAILURE.QualityFrog.com
I question software
[email protected]
www.QuestioningSoftware.com
v1.2 FAILURE.QualityFrog.com
I code software
[email protected]
www.QuestioningSoftware.com
v1.2 FAILURE.QualityFrog.com
I code software that helps test software
[email protected]
www.QuestioningSoftware.com
v1.2 FAILURE.QualityFrog.com
I talk about software
[email protected]
www.QuestioningSoftware.com
v1.2 FAILURE.QualityFrog.com
I write about software
[email protected]
www.QuestioningSoftware.com
v1.2 FAILURE.QualityFrog.com
I think about software
[email protected]
www.QuestioningSoftware.com
v1.2 FAILURE.QualityFrog.com
I like software
v1.2 FAILURE.QualityFrog.com
I like softwarethat serves people
v1.2 FAILURE.QualityFrog.com
I like softwarethat helps people
v1.2 FAILURE.QualityFrog.com
I dislike bad software
v1.2 FAILURE.QualityFrog.com
I dislike bad softwarethat frustrates people
v1.2 FAILURE.QualityFrog.com
I dislike bad softwarethat confuses people
v1.2 FAILURE.QualityFrog.com
I dislike bad softwarethat angers people
v1.2 FAILURE.QualityFrog.com
I dislike bad softwarethat discombobulates people
v1.2 FAILURE.QualityFrog.com
I sometimes like big words that discombobulate people
v1.2 FAILURE.QualityFrog.com
But only for a moment
v1.2 FAILURE.QualityFrog.com
Discombobulate means toemotionally confuse
v1.2 FAILURE.QualityFrog.com
Discombobulate means tofrustrate
v1.2 FAILURE.QualityFrog.com
Discombobulate means tobewilder
v1.2 FAILURE.QualityFrog.com
Discombobulate means toconfound
v1.2 FAILURE.QualityFrog.com
Discombobulate means tobefuddle
v1.2 FAILURE.QualityFrog.com
Discombobulate means toperplex
v1.2 FAILURE.QualityFrog.com
Too many error messages discombobulate people
v1.2 FAILURE.QualityFrog.com
Error messages should help people,
not confuse them
v1.2 FAILURE.QualityFrog.com
You see,It’s not about the software
v1.2 FAILURE.QualityFrog.com
It’s about the people!
v1.2 FAILURE.QualityFrog.com
Testers often fail to test the error messages
v1.2 FAILURE.QualityFrog.com
They fail to think about how they affect people
v1.2 FAILURE.QualityFrog.com
So what do I do when I encounter error messages?
v1.2 FAILURE.QualityFrog.com
I test them
v1.2 FAILURE.QualityFrog.com
I test themusing the FAILURE
mnemonic
v1.2 FAILURE.QualityFrog.com
A mnemonicis a device
that aids memory recall
v1.2 FAILURE.QualityFrog.com
The FAILURE mnemonic is an acronym that helps me
recallguideword heuristics
v1.2 FAILURE.QualityFrog.com
Guideword heuristics are words
that guide thinkingthrough a problem
v1.2 FAILURE.QualityFrog.com
The FAILURE mnemonic’sguideword heuristics help
me think about testing error messages
v1.2 FAILURE.QualityFrog.com
The F reminds me to think about
Functional aspects of the errorFAILURE
v1.2 FAILURE.QualityFrog.com
I ask Functional questions like
Does the error detection
function as expected?
v1.2 FAILURE.QualityFrog.com
I ask Functional questions like
Does the error handlingfunction as expected?
v1.2 FAILURE.QualityFrog.com
I ask Functional questions like
Does the error reporting
function as expected?
v1.2 FAILURE.QualityFrog.com
I ask Functional questions like
Does the error dialog, window, or page
function as expected?
v1.2 FAILURE.QualityFrog.com
I ask Functional questions like
Does the software recover from the error?
v1.2 FAILURE.QualityFrog.com
I ask Functional questions like
Was there really an error?
v1.2 FAILURE.QualityFrog.com
I ask Functional questions like
Are other errors like this reported?
v1.2 FAILURE.QualityFrog.com
I ask Functional questions like
Do dialog buttons work?
v1.2 FAILURE.QualityFrog.com
I ask Functional questions like
Is error message information accurate?
v1.2 FAILURE.QualityFrog.com
I ask Functional questions like
Is it functional?
v1.2 FAILURE.QualityFrog.com
Only I don’t stop at Functional
v1.2 FAILURE.QualityFrog.com
A Functional error messageis not necessarily
an Appropriate error message
v1.2 FAILURE.QualityFrog.com
The A reminds me to askif the error reporting is
AppropriateFAILURE
v1.2 FAILURE.QualityFrog.com
I ask Appropriate questions like
Is the error reporting timely?
v1.2 FAILURE.QualityFrog.com
I ask Appropriate questions like
Is the error reporting accurate?
v1.2 FAILURE.QualityFrog.com
I ask Appropriate questions like
Are UI messages free of inappropriate technical
detail?
v1.2 FAILURE.QualityFrog.com
I ask Appropriate questions like
Is there a better way to handle or report the
error?
v1.2 FAILURE.QualityFrog.com
I ask Appropriate questions like
Is the error user friendly?
v1.2 FAILURE.QualityFrog.com
I ask Appropriate questions like
Are errors detected as early in the user
workflow as possible?
v1.2 FAILURE.QualityFrog.com
I ask Appropriate questions like
Are system resource problems detected
while there is still time to remedy the problem?
v1.2 FAILURE.QualityFrog.com
I ask Appropriate questions like
Is the situation communicated appropriately
for the intended audience?
v1.2 FAILURE.QualityFrog.com
I ask Appropriate questions like
Is a user allowed to waste time and effort only to be told that
their work cannot be applied?
v1.2 FAILURE.QualityFrog.com
The I reminds me to askabout the Impact of the error
FAILURE
v1.2 FAILURE.QualityFrog.com
I ask Impact questions like
Is the impact communicated?
v1.2 FAILURE.QualityFrog.com
I ask Impact questions like
Is the impact communicated
to the user?
v1.2 FAILURE.QualityFrog.com
I ask Impact questions like
Is the impact communicated
to others?
v1.2 FAILURE.QualityFrog.com
I ask Impact questions like
Will the user understand the impact?
v1.2 FAILURE.QualityFrog.com
I ask Impact questions like
Is any impact to the user’s data
communicated?
v1.2 FAILURE.QualityFrog.com
I ask Impact questions like
Does the message contain too much
information?
v1.2 FAILURE.QualityFrog.com
I ask Impact questions like
Was a transaction only partially completed?
v1.2 FAILURE.QualityFrog.com
The L reminds me to askabout the error Log
FAILURE
v1.2 FAILURE.QualityFrog.com
I ask Log questions like
Should the error be logged for support?
v1.2 FAILURE.QualityFrog.com
I ask Log questions like
Should the error be logged for system administrators?
v1.2 FAILURE.QualityFrog.com
I ask Log questions like
Should the error be logged for developers?
v1.2 FAILURE.QualityFrog.com
I ask Log questions like
Should the error be logged for technical
investigation?
v1.2 FAILURE.QualityFrog.com
I ask Log questions like
Will log information be available if the user
waits to report a problem?
v1.2 FAILURE.QualityFrog.com
I ask Log questions like
Are log messages standardized to support automated information
mining?
v1.2 FAILURE.QualityFrog.com
I ask Log questions like
Can I grep this error?
v1.2 FAILURE.QualityFrog.com
I ask Log questions like
Do logs contain enough information?
v1.2 FAILURE.QualityFrog.com
I ask Log questions like
Do logs contain useless information?
v1.2 FAILURE.QualityFrog.com
I ask Log questions like
Is this an error that should not be logged?
v1.2 FAILURE.QualityFrog.com
I ask Log questions like
Is there too much logging?
v1.2 FAILURE.QualityFrog.com
I ask Log questions like
Does excessive logging impact performance?
v1.2 FAILURE.QualityFrog.com
I ask Log questions like
Does excessive logging impact disk space?
v1.2 FAILURE.QualityFrog.com
I ask Log questions like
Does the logging help or complicate error
investigation?
v1.2 FAILURE.QualityFrog.com
The U reminds me to think about
the User InterfaceFAILURE
v1.2 FAILURE.QualityFrog.com
I ask UI questions like
Is the error reported to the user?
v1.2 FAILURE.QualityFrog.com
I ask UI questions like
Does this error need to be reported to the
user?
v1.2 FAILURE.QualityFrog.com
I ask UI questions like
What user activitycaused the failure?
v1.2 FAILURE.QualityFrog.com
I ask UI questions like
What user activitycaused the failure?
Is this communicated to the user?
v1.2 FAILURE.QualityFrog.com
I ask UI questions like
Who is the user?
v1.2 FAILURE.QualityFrog.com
I ask UI questions like
Are UI messages worded for the
intended audience?
v1.2 FAILURE.QualityFrog.com
I ask UI questions like
Is this message consistent with the
desired look and feel?
v1.2 FAILURE.QualityFrog.com
I ask UI questions like
Is the UI reporting consistent with
reporting for other errors?
v1.2 FAILURE.QualityFrog.com
I ask UI questions like
Is the UI reporting efficient?
v1.2 FAILURE.QualityFrog.com
I ask UI questions like
Does the user need to click away excessive
dialogs?
v1.2 FAILURE.QualityFrog.com
I ask UI questions like
Is this the best way to communicate the error?
v1.2 FAILURE.QualityFrog.com
I ask UI questions like
Could this error be handled without
involving the user?
v1.2 FAILURE.QualityFrog.com
The R helps me think abouterror Recovery
FAILURE
v1.2 FAILURE.QualityFrog.com
I ask Recovery questions like
Is the user told how to recover from the
impact of the error?
v1.2 FAILURE.QualityFrog.com
I ask Recovery questions like
Does the user need to contact support?
v1.2 FAILURE.QualityFrog.com
I ask Recovery questions like
Is support contact information provided?
v1.2 FAILURE.QualityFrog.com
I ask Recovery questions like
Does support know how to fix the problem?
v1.2 FAILURE.QualityFrog.com
I ask Recovery questions like
Does the software guide the user through the recovery process?
v1.2 FAILURE.QualityFrog.com
I ask Recovery questions like
Or is the user left to figure it out on their
own?
v1.2 FAILURE.QualityFrog.com
The E leads me to think about
the Emotions of usersFAILURE
v1.2 FAILURE.QualityFrog.com
I ask Emotions questions like
What emotions are likely to be raised by the error message?
v1.2 FAILURE.QualityFrog.com
I ask Emotions questions like
Does the error reporting evoke the
desired user emotions?
v1.2 FAILURE.QualityFrog.com
I ask Emotions questions like
Does this message enhance
user frustration?
v1.2 FAILURE.QualityFrog.com
I ask Emotions questions like
Does this message quiet
user frustration?
v1.2 FAILURE.QualityFrog.com
I ask Emotions questions like
Does this message encourage users to use
some other product?
v1.2 FAILURE.QualityFrog.com
I ask Emotions questions like
Does this message cause confusion?
v1.2 FAILURE.QualityFrog.com
Asking these types of questions can help turn FAILURE into success
v1.2 FAILURE.QualityFrog.com
The next time you encounter an error
message
v1.2 FAILURE.QualityFrog.com
The next time you encounter an error
messageTest it!
v1.2 FAILURE.QualityFrog.com
The next time you encounter an error
messageQuestion it!
v1.2 FAILURE.QualityFrog.com
Try the FAILURE mnemonic
Functional
Appropriate
Impact
Log
UI
Recovery
Emotions
v1.2 FAILURE.QualityFrog.com
And let me know
how it works for you
Functional
Appropriate
Impact
Log
UI
Recovery
Emotions
v1.2 Copyright © 2007 Quality Frog
Ben Simo
[email protected]
http://FAILURE.QualityFrog.com
Functional
Appropriate
Impact
Log
UI
Recovery
Emotions