Metrics and Computer Games

Edel Sherratt

The rise and rise of computer games

• Computer games are widely used• People who write games want those games to

be desirable• And want players to play and play again• Metrics can help answer questions about

games

Questions about Games

• Is a game usable?• Is it fun?• Will anybody play the game?• Will players keep on playing?• Will they buy a new edition?

What is special about games?

• Factors that are of interest in software design in general apply to games as well

• But the kinds of question that are likely to be of most interest concern usability and fun!

• A typical player is not obliged to play a game

Designing Games metrics – some considerations

• Purpose of the proposed measurement• Nature of the game being played: action, race,

strategy, local vs distributed, single player vs multiplayer

• Measurement methods• Analysing and presenting the measurement

data

Purpose of measurement

• Predict player experience and engagement• Analyse psychological implications of

gameplay• Predict the likely sales of a new game• Identify problem areas in a game

Measurement Methods

• Game usage is of particular concern• Methods from HCI (human computer interaction) – Feedback questionnaire– Interview– Video analyis– Instrumentation to capture aspects of interaction,

such as keystrokes or joystick moves– Instrumentation to capture physiological facts about

the player

Some useful papers• Mark Claypool, The effect of latency on user performance in Real-

Time Strategy games, Computer Networks 49, pp 52-70, Elsevier 2005

• Anders Tychsen, Crafting User Experience via Game Metrics Analysis, NORDICHI 2008, Lund, Sweden

• Anders Drachen, Alessandro Canossa, Towards Gameplay Analysis via Gameplay Metrics, MindTrek 2009, Sept-Oct 2009, ACM

• Alessandro Febretti, Franca Garzotto, Usability, Playability and Long-Term Engagement in Computer Games, CHM 2009, April 4-9, Boston USA, ACM

• Eduardo Jiménez Chapresto, Kenny Mitchell, Francisco José Serón, Capture and Analysis of Racing-Gameplay Metrics, IEEE Software Sept-Oct 2011

Claypool 2005

• Focus on interactive network games and impact of latency on user performance

• Genre: Real-time strategy

Claypool

• Methodology– Classify user interactions in real-time strategy

games and arrange to exercise each strategy– Determine metrics– Construct experimental environment– Perform experiments– Analyze results

Claypool

• Overall user performance not significantly affected for latencies from tenths of seconds to seconds

• Some correlation between latency and game exploration

• Possibly explained by the fact that real-time strategy emphasizes strategy over interaction

• And games are designed to use the network effectively

• Further study needed to consolidate results and to investigate impact on latency on long-term strategy

Tychsen (2008)Drachen and Canossa (2009)

• Two papers, focusing on combining analysis of data obtained by interviews and audio recordings with data obtained by instrumentation

• IT University of Copenhagen and a Danish games company, IO Interactive

• Emphasis on game experience• Genre: shooter

Tychsen

• Importance and difficulty of measuring player experience

• Different kinds of data captured by different techniques

• Instrumentation good for capturing ‘what’ and ‘how’

• Interview and questionnaires good for capturing ‘why’

Drachen and CanossaCase study 1 (of 2)

• Level analysis in Fragile Alliance• Analysing spacial patterns of player ‘death’• Combine with player behaviour in different

locations• Aim to locate trouble spots in a game• Most behaviour as anticipated by game

designers• Some differences

Drachen and CanossaCase study 2 (of 2)

• Perfect paths in Kane & Lynch• To what extent do players follow the path

envisaged by the game designers• Are there any levels that are too difficult?

Drachen and Canossa

• Instrumentation has a great deal to contribute to measuring user experience of computer games

• Special kinds of visualization are helpful for analysis of play-session data

Febretti and Garzotto (2009)

• Does usability affect long term user engagement in computer games, or is playability more influential?’

• Eight commercial long games were analysed• Genre: long games

Febretti and Garzotto

• Usability– Customizability– Controls; including consistency of controls with

user expectations for game genre– Appropriateness of game views– Interface– Menu

Febretti and Garzotto

• Playability– Concentration and immersion– Challenge– Player ability– Control– Objectives and feedback– Social interaction– Feedback

Febretti and Garzotto

• Conclusion: long term engagement depends more on playability than usability

• However, this seems to depend on users encountering usability defects infrequently

• Which was true of the high-quality games included in the study

• Hypothesis: longterm engagement depends on density of usability defects (DUD)

Chapresto and Mitchell; Serón (2011)

• Like Tychsen and Drachen & Canossa, considers combination of data obtained by surveys and interviews with objective data

• Also like these, considers new visualization techniques to study the data gathered

• Presents the Tractivity System for data logging

Chapresto and Mitchell; Serón

• Attributes measured– Powerplay triggering frequency– Wreckage frequency– Self-wreckage frequency– Player porgression– Race analysis

Chapresto and Mitchell; Serón

• For each attribute studied, problems were detected and corrected

• Looking to extend Tractivity to capture live data, develop yet more visualization techniques, and further improve the user experience