building on the usability study: two explorations on how to better understand an interface
DESCRIPTION
In this presentation, we describe two separate studies that improved our ability to understand our users’ experience of our products at salesforce.com. The first study explored a methodology of combining expert and novice performance data to yield a measure of intuitiveness. The second study created a methodology that combines both verbal and nonverbal emotion scales to better understand the emotional effect our products have on our users. We present both these methods as expansions on the standard usability study and examples of ways to better understand your users within an industry environment.TRANSCRIPT
Building on the !Usability Study !
Two Explorations on How to!Better Understand an Interface
Anshu Agarwal Madhu Prabaker
Usability Analysts
HCI International 2009 San Diego, California
UserExperience
TraditionalUsabilityMeasure
???
UserExperience
TraditionalUsabilityMeasure MustbeValuable
&Practical
UserExperience
TraditionalUsabilityMeasure Intuitiveness
UserExperience
TraditionalUsabilityMeasure Intuitiveness
Emotion
UserExperience
TraditionalUsabilityMeasure Intuitiveness
Emotion
Measuring Intuitiveness
What does it mean to be Intuitive?
“Easytouseandunderstand”
“Atechnicalsystemis,inthecontextofacertaintask,intuitivelyusablewhiletheparticularuserisabletointeracteffectively,notconsciouslyusing
previousknowledge”‐[Naumannetal]
Project Goals"Measuring Intuitiveness
UserExperience
TraditionalUsabilityMeasure Intuitiveness
Measuring Novice Performance on an Interface
Measuring Expert Performance on an Interface
Combining Expert and Novice Performance
“…interacteffectively,notconsciouslyusingpreviousknowledge.”
Combining Expert and Novice Performance
Anintuitiveinterfacecanbethoughtofasonethat“achievesnovicetaskperformanceascloseaspossibletoexperttaskperformance”
Combining Expert and Novice Performance
Combining Expert and Novice Performance
Measuring Emotion
Project Goal"Understanding Emotional Impact
UserExperience
TraditionalUsabilityMeasure
Emotion
Existing Ways to Measure Emotion"The Common Tools
Physiological
Verbal
Non‐Verbal
Our Approach"Combine the Best from Both Worlds
Physiological
Verbal
Non‐Verbal
Our Verbal Measure"PAD Semantic Differential Scale
Pleasure,Arousal,andDominance(PAD)
SemanticDifferentialScale
[Mehrabian&Russell]
PADDimension BipolarPairs
Pleasure
Annoyed‐Pleased
Unsatisfied‐Satisfied
Despairing‐Hopefully
Tense‐Relaxed
Friendly‐Unfriendly
Arousal
Relaxed‐Stimulated
Calm‐Excited
Sleepy–WideAwake
Unaroused‐Aroused
Dominance
Controlled‐Controlling
Influenced‐Influential
Submissive‐Dominant
Guided‐Autonomous
Our Non-Verbal Measure"Emo-Card
ExcitedPleasant
CalmPleasantCalmUnpleasant
ExcitedUnpleasant
‐Pleasure
‐Arousal
+Pleasure
+Arousal
Validating the Technique"Empirical Study Design
22Participants(13male,9female)
InterfaceA InterfaceB
7Tasks(Randomized)
StandardMeasures EmotionalMeasures
TimeonTask Emocard
#ofErrors PADScale
Post‐TaskInterview
Validating the Technique"Empirical Study Results
Neither Time on Task nor Number of Errors were significantly different between Interface A and Interface B
InterfaceA InterfaceB
PAD Scale and Emocard showed some clear differences in how users experienced the interfaces
Validating the Technique"Empirical Study Results
Neither Time on Task nor Number of Errors were significantly different between Interface A and Interface B
InterfaceA InterfaceB
PAD Scale and Emocard showed some clear differences in how users experienced the CRM Systems
Validating the Technique"Empirical Study Results
Neither Time on Task nor Number of Errors were significantly different between Interface A and Interface B
InterfaceA InterfaceB
PAD Scale and Emocard showed some clear differences in how users experienced the interfaces
Conclusion
UserExperience
TraditionalUsabilityMeasure Intuitiveness
Emotion
Conclusion
UserExperience
TraditionalUsabilityMeasure
???Whatareyoumissingthat’s
Valuable&Easy
Agarwal, A. and Meyer, A. 2009. Beyond usability: evaluating emotional response as an integral part of the user experience. In Proceedings of the 27th international Conference Extended Abstracts on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009).
Anshu Agarwal [email protected]
Madhu Prabaker [email protected]