how do researchers choose commercial games for study?

How Do Researchers Choose Commercial Games for Study?How Do Researchers Choose

Commercial Games for Study?

Katrin Becker

2Futureplay 2009

© Becker 2009 How Researchers Choose Games

Background

• How does a commercially and critically successful modern video game support the learning that players must accomplish in order to succeed in the game (i.e. get to the end or win)?

• Study the masters.

• How to define ‘successful’ game?

3Futureplay 2009


In the good old days..

• Studied what we had

• Reported on the games we knew

• Wrote about interesting things we discovered about the games we were already playing

4Futureplay 2009


But now....

• Over 50,000 games – (http://www.gamestats.com/ reports almost 58,000)

• ~ 3000 titles released in 2008

• So much choice

• Begs the question...

5Futureplay 2009


Why this game?

6Futureplay 2009


Why Should We Care?

• Games Studies still building academic credibility • Speaks to generalizibility of studies• If you make a claim: where’s the evidence?!

7Futureplay 2009


What I did• Meta analysis• 2003-2006 and

2007-2008• Looked at papers

featuring 1 or a few games

• Looked at whether they explained and why this game was chosen.

8Futureplay 2009


What I DIDN’T

• Judge worthiness of game chosen

• Assume that no explanation = no thought

9Futureplay 2009


What I found outHow people used the game:

the mod or embellishment of an existing game Mod

more general discussion or study which uses a game as evidence, or to illustrate a point Example

people using the game, or some other aspect where the game is the tool Apparatus

a certain kind of game rather than people using the game Typical

any game or all games (the study looks at the game rather than people using the game) General

4%

4%

33%

40%

19%

2003

-200

6

3%

3%

8%

75%

8%

2007

-200

8

Of those papers examined

10Futureplay 2009


Does report have references to support claims about validity of

rationale for game choice. (i.e. the qualities necessary for a suitable

game for this study)?

2003-2006: 15% (yes, but only one gave more than a single reference)

2007-2008: 6% yes, 8% yes>1 = 14% total

11Futureplay 2009


Does report have references to support for claims that chosen game meets specified criteria?

2003-2006: 4% yes

2007-2008: 3% yes

12Futureplay 2009


Was there any mention of games not chosen or

description of unsuitable games?

2003-2006: 2% yes

2007-2008: 0% yes

13Futureplay 2009


Was there any mention of any structured methodology for the

game choice?

2003-2006: 2% yes

2007-2008: 0% yes

14Futureplay 2009


Some sample reasons...

• We’re already using it

• It is the most popular

• It is typical

• It has what I need for my study

• Has an in-game economy

• Have played it for a long time

• Typical of Japanese games

15Futureplay 2009


Some sample reasons...

• Embodies movement moreso than most other games

• Because of its story

• Narrative & open-ended

• A game with significant fantasy element

• Good game

• Prior work

16Futureplay 2009


Often the explanation is simply one that claims the game is of the sort needed for this study.

17Futureplay 2009


Nothing has changed

18Futureplay 2009


Who Cares?

• Speaks to– Credibility– Fitness for purpose– Generalizability

• Mario & Assassin’s Creed are both violent games.

– (yeah, right)

19Futureplay 2009


What is Needed?• Justify why THIS game

• Provide evidence supporting the claims we make about the qualities of the game that we have determined are necessary to our study

20Futureplay 2009


Questions / Discussion

how do researchers choose commercial games for study?

Education

game choice

successful game

gametypicalany game

game economy

games studies

mention of games

violent games

commercial games