informal guerilla user study

“Informal Guerilla User Study”How to get quick and useful feedback on your app

ID for Devz at Hacker Dojo

Nilo Sarraf

January 2011

We see what we want to see!

• Assumptions– In research do NOT assume!– Researcher bias: “I know the results before

I even go into the lab!”

• Even top researchers are biased

• Humans are complex machines

Get Divorced!• Divorce your app

• Be objective

• Get stupid when doing research on your app!

Know Thy UserEven when they look totally alien to you

• Body language/body gestures

• Tone of voice

• Personality types

• Context/Environment

• Mood swings

The Observer Effect• Physics

– Changes that the act of observation will make on the phenomenon being observed

• Information Technology– The potential impact of the act of observing a

process output while the process is running• Psychology

– A form of reactivity, in which a researcher's cognitive bias causes them to subconsciously influence the participants of an experiment

How to Observe Users

• Observational techniques • Behavioral• Facial muscles• Body gestures• Voice pitch• Choice of words

QuickTime™ and a decompressor

are needed to see this picture.

Source:Elsevier 2010

http://comminfo.rutgers.edu/~tefko/Courses/Zadar/Doctoral%20studies/Lopatovska%20emotions%20in%20LIS%20IPM%202011%20artcle%20in%20press.pdf

Voice and Choice of Words“Know Thy User”

• High or low pitch

• Energetic or tired

• Fast or slow

• Pauses between words

• Choice of words

Personality Types(Psychology)

• Openness to Experience: the tendency to be imaginative, independent, and interested in variety vs. practical, conforming, and interested in routine

• Conscientiousness: the tendency to be organized, careful, and disciplined vs. disorganized, careless, and impulsive

• Extraversion: the tendency to be sociable, fun-loving, and affectionate vs. retiring, somber, and reserved

• Agreeableness: the tendency to be softhearted, trusting, and helpful vs. ruthless, suspicious, and uncooperative

• Neuroticism: the tendency to be calm, secure, and self-satisfied vs. anxious, insecure, and self-pitying

Personality Types “Know Thy User”

• Type "A”– Is a highly independent and driven personality, typically

representing the leaders in business

• Type "B" – Represents highly extroverted people who love the spotlight

• Type "C" – The antithesis of Type "B"; they are introverted detailists as

represented by such people as accountants, programmers, and engineers

• Type "D" Personality– Is best characterized as those people who resist any form of

change and prefer the tedium of routine, such as in clerical assignments

Type A

• Independent• Go-getters • Aggressive • Driven personality• Leaders in business

Type B

• Highly extroverted• Gregarious • Love the spotlight• Attractive

Type C

• The antithesis of Type "B”– Don’t like the spot light

• Introverted detailists• Accountants, • Programmers • Engineers

Type D

• Resist any form of change• Prefer the tedium of routine• Clerical assignments

Environment“Know Thy User”

• Familiar or unfamiliar location

• At the Café next door or at a usability lab

• Do you know the user personally?

Mood Swings“Know Thy User”

• Weather– Sunny or rainy

• Time of the day– Morning or evening

• Time of the week– Monday or Friday

Let’s Get Practical

• Informal User Study– aka Guerilla testing,

Dry run, and Hallway testing

• A quick and easy test• Should take no more

than 15 min

Before User Study

• Roughly speaking… – What is your objective?– What do you want to find out?– Who is your target audience? – Do you have paper or full-fledge

prototype?

During User Study

• It’s not you. It’s me!– We are not testing the users but the app– We are here to create a better app. So, we need

you!

• Reassuring the user • Make them feel ‘at ease’

– ‘At ease’ does not mean comfortable but just relaxed enough to give good insights

During User Study

• Listen - needs self-training

• Observe - needs self-training

• Have users talk aloud

• Do NOT prompt users

• Do NOT ask for design advise

• Have fixed questions/objectives that you want to find answers to

Take Notes

• Tape recording

• Take notes

Your Own Body Language

• Users can tell if you are very attached to your app

• Be curious– Show: I’m here to learn from you!

• Friendly ‘poker face’

• Show openness

‘Safe’ Questions

• “What do you think/believe this button means/does?”• “Is this what you expected to see?”• “If you were to click on this tab, what would you expect to

happen? What would you expect to see?”• “What do you think the app is trying to do/accomplish?”• “Who do you think may find this app

useful/beneficial/interesting/handy…?”• “What is your overall feeling/understanding of this app?”• “Where do you think this app would be used?”• “At this point, what would you do?”

‘Safe’ Questions

• “Taking a quick look, what do you think this app is about? What do you think it does?”

• “What would be the first you’d do?”• “Where would you go about exploring?”• “What peeks your interest at this point?”• “What is your overall feeling/understanding of

this app?”• “Any final comments you’d like to add?”

Pause

• Don’t be afraid to interrupt the user• Sometimes users go fast and all over the

place– Stop the user– So, let’s pause for a moment here

• What are you thinking?• How are you deciding your next steps?• What are you seeing?

After User Study

• Talk about your findings with– People in your industry– People in other industries– Researchers, if you know any

• Draw possible conclusions• Work on your app• Decide if you need any follow-up refined

studies

One Last Thing

• User study is NOT about testing a hypothesis

• User study is about uncovering new things that you did not know before