evaluating game usability - how game research will change the face of software applications
TRANSCRIPT
How game research will change the face of
software applications
Engineering Master in computational visualistics
Technology developer (EU FUGA project)
Game design researcher
Investigating quality of fun in gameplay
BTH Game and Media Arts Laboratory
Player experience and interaction studies
EU-funded project: “Fun of Gaming”
Game Usability
Consulting in psychophysiological studies
2008-06-30 2
1) Definition of game usability
2) Experimental player studies
3) Interaction and fun
4) Conclusion
We start with a general definition of
usability and games. Then we explore
the concept of game usability.
We get some hands-on insight into
psychophysiological study of game
experience and its implications.
Subsequently, we take a brief glance
at new interaction forms in games.
We summarize and conclude the
potentials of game usability studies.
2008-06-30 3
2008-06-30 4
Achieve goals with… (ISO 9241) Efficiency
Effectiveness
User satisfaction
Nielsen (1993) Learnability (easy-to-learn)
Efficiency (high user productivity)
Memorability (easy-to-remember)
Errors (low error rate, rectifiable)
Satisfaction (subjective fun of use)
2008-06-30 5
“The usability of a product is not an
attribute of the product alone, it is an
attribute of interaction with a product
in a context of use.”
2008-06-30 6
Empirical usability evaluation Performance tests
▪ Task effectiveness (speed, accuracy, errors)
Attitude surveys▪ User satisfaction and perception (questionnaires, interviews)
Common examples▪ Think-aloud protocol
▪ User-data collection (cf. performance test)
▪ Psychophysiological experiments (eye gaze, heart-rate, EMG, EEG)
▪ Surveys and questionnaires
▪ Interviews
2008-06-30 7* (Wiberg 2003)
Inspection methods
Heuristic evaluation (rules of thumb, expert review)
Cognitive walkthrough (learning of tasks)
Design walkthrough (scenarios for design elements)
Consistency inspection
Feature inspection
Theory-based reviews
Standard inspection
2008-06-30 8* (Wiberg 2003)
Usability forms basis of user experience
Can be broken down into measurable items
Time (tasks, completion, movement)
Errors
User Initiated Events (UIEs)
Evaluating user satisfaction is difficult
How do you measure “fun” (of use)?
2008-06-30 9
Kim et al. (2008) of Microsoft User Research
“The most complex systems imaginable for a
usability practitioner –
human interaction with
video games.”
2008-06-30 10
Structured activity of enjoyment
Goals
Rules
Challenge
Interactivity
Skill development
Learnable
Satisfaction
2008-06-30 11
Game studies deeply rooted in cultural studies
Aarseth, Mäyrä, Juul, Järvinen
Early research using
Reflections
Humanities approach
Qualitative evaluations (e.g. Consalvo)
Education (Jenkins, Squire, Prensky, Egenfeldt-Nielsen)
Few to none empirical studies (e.g. Yee)
2008-06-30 12
Empirical studies (using scientific method)
Surveys (Yee, Tychsen)
Psychophysiology (Ravaja, Salminen, Kivikangas)
All of the above (Lindley, Nacke)
Behavioral (IJsselsteijn et al.)
Communication sciences (Klimmt)
Researching “FUN”
Objective and subjective measures
2008-06-30 13
Study the activity of playing a game
Towards measuring fun
Combination of scientific approaches
Is the game
Learnable, fun to play, memorable, fail-safe
2008-06-30 14
2008-06-30 15
Formulate Theory
Design experiments
• Empirical validation
Create Stimuli
• For experiments
Conduct experiments
Construct validity
• Reliability
Analyze
• return to start
2008-06-30 16
Before designing experiments…
Compare two identical situations With one (or more) differential variables
Control conditions Cause present vs. cause absent
Randomization
Null hypothesis Falsification of hypothesis
2008-06-30 17
Game Design
Considerations
This Half-Life 2 level was designed
especially for increasing levels of
challenge, divided into three
different rooms, each containing
different opponent types and
attack patterns.
The player is limited to using the
crossbow and crowbar weapons,
making combat mechanics difficult
in themselves.
2008-06-30 18
Psychophysiological
Instrumentation
Solution
A custom instrumentation solution
was built that integrates in the
Half-Life 2 Hammer Editor.
The event code system directly
connects to the psycho-
physiological hardware (Stellmach
2008 - to be presented at
International conference on Fun
and Games).
2008-06-30 19
playerEEG
ECG
EMG
GSR Respiration
Eye Tracking
Interaction
Logging
Pre and post game
Surveys
2008-06-30 20
Electroencephalogram (EEG) Electrical brain activity measured with electrodes (cap)
Electromyography (EMG) Recording electric potential generated by muscles
Measure of valence (positive/negative emotion)
Galvanic skin response (GSR) Also electrodermal response (EDR), psychogalvanic reflex
(PGR), skin conductance response (SCR)
Electrical resistance of the skin
Measure of arousal (high/low)
2008-06-30 21
Eye position
Regions of interest (ROIs, Hotspots)
Eye movement
Sequences
Movement patterns (e.g. fixations, saccades)
Pupil dilation
Focus of interest
Special attention
2008-06-30 22
Psychophysiological
Game Experiments
Setup
The Game and Media Arts
Laboratory in Sweden conducts
psychophysiological research
experiments.
The figure shows the EEG cap worn,
while facial EMG electrodes are
being attached with adhesive tape.
2008-06-30 23
Galvanic Skin
Response Recording
Sites
Galvanic skin response was
recorded from palmar sites during
gameplay.
2008-06-30 24
Psychophysiological
Measurement Device
The Biosemi ActiveTwo System for
recording of physiological signals
(EEG, EMG, ECG, GSR)
2008-06-30 25
Experimental Game
Session
Experimental gaming session with
all measurement instrumentation
in place. The Tobii 1750 eye tracker
is used.
There is also a video recording of a
participant playing.
2008-06-30 26
Gaze Distribution on
Game Menu Screen
Gaze plots show the sequence and
length that eyes have rested on
certain positions.
2008-06-30 27
Gaze Hotspots
This figure shows areas of highest
visual interest in the game menu
screen.
2008-06-30 28
Data Analysis
Cross-correlation
Game events
Physical response
Survey results
2008-06-30 29
2008-06-30 30
2008-06-30 31
Physical movement
Musical expression
Haptic sensation
2008-06-30 32
Eye Tracker Game
Input
Using a Tobii Technology 1750 eye
tracker, we developed a game that
can be steered with eye input only
(Sasse 2008).
2008-06-30 33
Blob Gaze Video
The video shows an early in-game
view of the Blob “eye tracker only”
game.
The player controls a “Blob” with his
eyes and collects other blobs, which
grow over time. When two blobs
collide, the game ends.
The size and color sequence of
blobs influences the game score.
2008-06-30 34
Driving questions
Does interaction contribute to fun?
What forms of physical interaction trigger optimal
emotions?
Can interactions tested for games contribute to regular
software?
How can games be designed for people with disabilities?
Is fun not more than a measurable emotion?
2008-06-30 35
2008-06-30 36
Focus on user satisfaction
How to create fun
Instrumentation to record player behavior can
be used to analyze regular software users
2008-06-30 37
Fun of use is growing in importance
Boundaries blur between work and play
Modern applications share features of games
Self-explanatory
Easily accessible
Customization
2008-06-30 38
Studies of game experience are necessary
Great insight into game interaction feeds
back to general human-computer interaction
Metrics of game usability remain to be fully
explored
2008-06-30 39
Karat, J. (1997). User-centered software evaluation methodologies (Vol. 2).
Kim, J. H., Gunn, D. V., Schuh, E., Phillips, B., Pagulayan, R. J., & Wixon, D. (2008). Tracking real-time user
experience (TRUE): a comprehensive instrumentation solution for complex systems.
Nielsen, J. (1993). Usability Engineering: Morgan Kaufmann.
Sasse, D. (2008). A Framework for Psychophysiological Data Acquisition in Digital Games. Master’s
thesis (published in GameCareerGuide.com), Otto-von-Guericke-University, Magdeburg
Stellmach, S. (2007). A psychophysiological logging system for a digital game modification. Internship
Report (published in GameCareerGuide.com), Otto-von-Guericke-University, Magdeburg.
Wiberg, C. (2003). A Measure of Fun: Extending the scope of web usability. PhD Thesis, Umeå
University, Umeå, Sweden.
2008-06-30 40
2008-06-30 41
Lennart Nacke
Blekinge Institute of Technology
Game and Media Arts LaboratoryKarlshamn, Sweden
http://gamescience.bth.se
http://www.acagamic.com