triggered think aloud protocol for eye tracking (chi 2011 presentation)
DESCRIPTION
Short presentation shared at CHI 2011. Describes a concurrent think-aloud protocol that uses eye tracking data to inform usability moderation in real time. The desire is to improve when and how probes are administered while minimally influencing task behavior. The original paper is in the CHI 2011 conference proceedings.TRANSCRIPT
Triggered Think-Aloud Protocol:Using Eye Tracking to Improve Usability Test Moderation
Beverly Freeman | PayPal | CHI 2011
We Think-Aloud
“Thinking aloud may be the single most valuable
usability engineering method.”
Concurrent vs. Retrospective Think-Aloud
CTA(concurrent think-aloud)
+ –Thoughts are captured while fresh
Task performance is uninterrupted
Talking may interfere with doing
Memory decay, fabrication
RTA(retrospective think-aloud)
Task performance
Verbalization of task performance
Gaze-Cued RTA
Gaze-Cued RTA
+ –ET video provides an additional memory aid
Participants may freak out!
Memory decay, fabrication
Task performance
Verbalization of task performance
Triggered Think-Aloud Protocol (TTA)
• Modified form of CTA
• Moderator uses real-time gaze data as an input to determine when and how to prompt
Gaze-Cued RTA:Gaze data Participant
TTA:Gaze data Moderator Participant
Triggered Think-Aloud Protocol (TTA)
Instructions to participants
Yes No“Share anything that would help me better understand your experience”
Triggered by observed behaviors (e.g. “How would you describe what you’re seeing?”)
“Continually think aloud”
Triggered by periods of silence
Moderator prompts
What We Want to Avoid
“Did you notice…?”
“You want me to keep talking? Um, I’m typing in my password now...”
“What are you looking at now?”
“Oh right! I should’ve noticed this thing over
here…”
Task performance
Verbalization of task performance
TTA: A Preliminary Case Study
• Refrain from probing– Avoid interfering with discovery
• Probe away! – Capture initial interpretation
Using TTA: Factors to Consider
1. Research goals– Formative vs. summative
2. Participant characteristics– Talkative vs. reticent participants
3. Experience characteristics– Static vs. dynamic content
– First-time encounter vs. habituated experience
– Diversity of user intents / strategies
4. ?
5. ?
6. ?
Next Steps for Us
• Refinement of TTA protocol– Initial participant instructions
– Moderator probes and prompts
• Experimental research– Investigate impact of various factors (think-aloud variation, participant
characteristics, experience characteristics, etc.) on behavior and verbal comments
Thank you!
Questions?