top ten research findings in games, alicia sanchez, defense acquisition university
DESCRIPTION
Alicia Sanchez holds the title of “Games Czar” at the Defense Acquisition University. She will review the top ten most relevant research findings in the use of serious video games in training and education. While very little research on the use of video games in education and training makes headlines, there is great research in serious games, game components and implementations that have real impacts on the way we approach our game projects. Dr. Sanchez will aggregate the most important findings from 2012 and explain the science of video games for training and education! When: March 28, 2013, 9.00 am PT/noon ET/17.00 CETTRANSCRIPT
2012Dr. Alicia Sanchez
Defense Acquisition University
Top Ten Research Findings in Games
GROUND RULES
The Top 10 Research Findings for 2012 have been decided by me alone. All papers included have empirical results Papers selected not solely based on quality, but
on relevance to this particular conference In some cases I have not presented all of the
results that were found by these researchers I will make my presentation available I can’t make the research papers available Everything in this presentation has been
subject to my interpretation
Sanch
ez’s To
p 1
0 o
f 201
2
DENNER, J.; WERNER, L.; & ORTIZ, E. (2012). COMPUTER GAMES CREATED BY MIDDLE SCHOOL GIRLS: CAN THEY BE USED TO MEASURE UNDERSTANDING OF COMPUTER SCIENCE CONCEPTS? COMPUTERS & EDUCATION, 58(1), 240-249.
Computer Games Created by Middle School Girls
Stagecast Creator Software Voluntary after school program for computer
programming Each student created 1-5 games, each taking 4-6
weeks to complete (108 total games) 4 genres of games were built: maze, maze with
focused context, action, trivia Coding strategy employed to assess games
Participants – 59 girls in a voluntary after school program for computer programming, 72% Latina. Participants had no prior coding experience
10Sanch
ez’s To
p 1
0 o
f 201
2
DENNER, J.; WERNER, L.; & ORTIZ, E. (2012). COMPUTER GAMES CREATED BY MIDDLE SCHOOL GIRLS: CAN THEY BE USED TO MEASURE UNDERSTANDING OF COMPUTER SCIENCE CONCEPTS? COMPUTERS & EDUCATION, 58(1), 240-249.
Computer Games Created by Middle School Girls
Results: Games were coded and the following conclusions
were drawn: Programming games may be a promising approach to
engaging underrepresented students in the concepts and capabilities that will prepare them for computer science courses and careers
Students struggled with complex concepts Students did not leverage full capabilities of software
Discussion – Using game development as a learning tool can yield some results
10Sanch
ez’s To
p 1
0 o
f 201
2
WANG, L. & CHEN, M. (2012). THE EFFECTS OF LEARNING STYLE AND GENDER CONSCIOUSNESS ON NOVICE’S LEARNING FROM PLAYING EDUCATIONAL GAMES. KNOWLEDGE MANAGEMENT & E-LEARNING: AN INTERNATIONAL JOURNAL, 4(1), 63-77.
The effects of learning style and gender consciousness on novice learning in games
Custom Flash Development to learn programming through game-play Learning Styles
Reduced into diverging & converging via Kolb(not visual etc…..) Divergers – best at viewing concrete situations with multiple viewpoints Convergers – best at finding practical uses for ideas and theories
Gender Consciousness Gender role, traits, and equality
Participants – 122 eighth grade students (59 m, 63 f) Goals – examine the effects of learning style and gender
consciousness on: Comprehension of programming concepts Project performance Motivation
Sanch
ez’s To
p 1
0 o
f 201
2 9
WANG, L. & CHEN, M. (2012). THE EFFECTS OF LEARNING STYLE AND GENDER CONSCIOUSNESS ON NOVICE’S LEARNING FROM PLAYING EDUCATIONAL GAMES. KNOWLEDGE MANAGEMENT & E-LEARNING: AN INTERNATIONAL JOURNAL, 4(1), 63-77.
The effects of learning style and gender consciousness on novice learning in games
Results: The convergers comprehended the abstract programming principles
better than the divergers, but the gender consc did not have an effect Divergers with low gender consc outperformed high gender consc
divergers on project performance, but both gender consc groups of convergers performed equally
High gender consc convergers outperformed high gender consc divergers on project performance, but low gender consc convergers and divergers performed equally.
Intrinsic & extrinsic motivation was similar and positive for all four groups.
Discussion – Learning style impacted programming comprehension, and gender consc impacted project performance for divergers and convergers differently.
9Sanch
ez’s To
p 1
0 o
f 201
2
PANOUTSOPLOULOS, H. & SAMPSON, D.G. (2012). A STUDY ON EXPLOITING COMMERCIAL DIGITAL GAMES INTO SCHOOL CONTEXT. EDUCATIONAL TECHNOLOGY & SOCIETY, 15(1), 15-27.
A Study on Exploiting Commercial Digital Games into School Context
Sims 2 – Open for Business Researchers tested
Math objectives General Ed objectives Changes in attitudes about math teaching & learning
Participants: 56, 13-14 year olds in Athens, Greece Math scores pre intervention not significantly
different
8Sanch
ez’s To
p 1
0 o
f 201
2
PANOUTSOPLOULOS, H. & SAMPSON, D.G. (2012). A STUDY ON EXPLOITING COMMERCIAL DIGITAL GAMES INTO SCHOOL CONTEXT. EDUCATIONAL TECHNOLOGY & SOCIETY, 15(1), 15-27.
A Study on Exploiting Commercial Digital Games into School Context
Results Did not result in higher or lower math achievement Better achievement for games in general education No difference in attitudes towards math teaching and
learning Conclusion- Off the shelf games can do just as well,
if not better than standard curriculum for some objectives
8Sanch
ez’s To
p 1
0 o
f 201
2
GONZALEZ, C.; SANER, L. D.; & EISENBERG, L. Z. (2012). LEARNING TO STAND IN THE OTHER’S SHOES: A COMPUTER VIDEO GAME EXPERIENCE OF THE ISRAELI-PALENSTINIAN CONFLICT. SOCIAL SCIENCE COMPUTER VIEW, SAGE PUBLICATIONS.
Learning to Stand in the Other’s Shoes
Peacemaker COTS game that allows you to play multiple roles and make
decisions related to those roles within the Israeli-Palenstinian conflict
Researchers hypothesized: Practice in the game would reduce the effects of religious
views and political affiliations on decision making to reduce conflict
Participants: 42 undergrads ages 18-23 Game used in class activity twice in a semester
Randomly assigned to first role (Palestinian President or Israeli Prime Minister) all played both (Total of 4 plays)
Randomly assigned to Level Difficulty – Calm vs violent
7Sanch
ez’s To
p 1
0 o
f 201
2
GONZALEZ, C.; SANER, L. D.; & EISENBERG, L. Z. (2012). LEARNING TO STAND IN THE OTHER’S SHOES: A COMPUTER VIDEO GAME EXPERIENCE OF THE ISRAELI-PALENSTINIAN CONFLICT. SOCIAL SCIENCE COMPUTER VIEW, SAGE PUBLICATIONS.
Learning to Stand in the Other’s Shoes
Results Balance (peace) was the goal of the game and most
students did not achieve great balance scores Role played impacted time spent playing (Palestinian
role > time than Israeli role) Religion correlated with balance in the first session, but
not the second. Political affiliation correlated with balance in the first
session, but not the second. Conclusion – The game intervention mitigated the
initial religious and political views of its players.
7Sanch
ez’s To
p 1
0 o
f 201
2
CHEN, Z.; LIAO, C.C.Y.; CHENG, H.N.H.; YEH, C.Y.C.; & CHAN, T. (2012). INFLUENCES OF GAME QUESTS ON PUPILS’ ENJOYMENT AND GOAL-PURSUING IN MATH LEARNING. EDUCATIONAL TECHNOLOGY & SOCIETY, 15(2), 317-327.
Influences of Game Quests on Pupils in Math Learning
My-Pet-My-Quest Pet-nurturing game as previous research indicated
motivation, and researchers could implement an economic model of needs-consumption-work
Taiwanese students report relatively lower positive attitudes towards math, even though their math performance is high
Included a quest-delivery mechanism as has been used in role playing games to guide players to perform tasks – Quests provide three elements- objectives, learning tasks and
rewards Related to goal orientation
Researchers implemented a three tiered design process to include learning & quests within the game
6Sanch
ez’s To
p 1
0 o
f 201
2
CHEN, Z.; LIAO, C.C.Y.; CHENG, H.N.H.; YEH, C.Y.C.; & CHAN, T. (2012). INFLUENCES OF GAME QUESTS ON PUPILS’ ENJOYMENT AND GOAL-PURSUING IN MATH LEARNING. EDUCATIONAL TECHNOLOGY & SOCIETY, 15(2), 317-327.
Influences of Game Quests on Pupils in Math Learning
Participants: 53 Taiwanese 4th grade students – within subjects design
Question – What are the influences of game quests on student’s math learning in terms of perception of enjoyment and goal-pursuing
Findings Quests had positive impact
Quests influenced student’s perceptions including enjoyment, goal orientation & goal intensity
Quests elicited more active participation & promoted more enjoyable experiences
Conclusion – Quests whose goals align with learning objectives are favored by students
6Sanch
ez’s To
p 1
0 o
f 201
2
The Impact of Tutorials on Games of Varying Complexity
Games – Refraction, Hello Worlds & Foldit Puzzle games of varying complexity with Foldit being most
complicated by far Tutorials – Hypotheses
Games with tutorials will exhibit better player engagement and retention
Tutorials that present instructions in context will be more effective
Tutorials that restrict player freedom improve engagement and retention (by ensuring the player focuses and must complete the tutorial)
Having on demand access to help improves retention Online Data methodology – 8 types of tutorials, 3 games,
45,318 total subjects!
ANDERSEN, E.; O’ROURKE, E.; LIU, Y.; SNIDER, R.; LOWDERMILK, J.; TRUONG, D.; COOPER, S.; & POPOVIC, Z. (2012). THE IMPACT OF TUTORIALS ON GAMES OF VARYING COMPLEXITY. PAPER PRESENTED AT CHI’12, AUSTIN, TX.
5Sanch
ez’s To
p 1
0 o
f 201
2
The Impact of Tutorials on Games of Varying Complexity
Results Tutorials were only justified in Foldit (the most complex game) and
not in the other two games whose mechanics might be more easily discovered through experimentation
Tutorial context sensitivity improved player engagement in Foldit (the only game where a tutorial was justified) (players in Foldit with context sensitive tutorial played 40% more levels and 16% longer)
Tutorial freedom did not affect player behavior On-demand had a negative impact on engagement in Refraction
(only 31% used it) but increased engagement a small amount in Hello Worlds.
Discussion – The use of tutorials should be dependent on complexity of game. The functionality of those tutorials review these results!
ANDERSEN, E.; O’ROURKE, E.; LIU, Y.; SNIDER, R.; LOWDERMILK, J.; TRUONG, D.; COOPER, S.; & POPOVIC, Z. (2012). THE IMPACT OF TUTORIALS ON GAMES OF VARYING COMPLEXITY. PAPER PRESENTED AT CHI’12, AUSTIN, TX.
5Sanch
ez’s To
p 1
0 o
f 201
2
THOM, J.; MILLEN, D. R.; DIMICCO, J. (2012). REMOVING GAMIFICATION FROM AN ENTERPRISE SNS. PROCEEDINGS ACM CONFERENCE ON COMPUTERS SUPPORTING COLLABORATIVE WORK.
Removing Gamification from an Enterprise SNS
SNS based Gamification Authors implemented gamification constructs into a social networking
system of a large organization The gamification goal was to encourage content contribution
Points were awarded 5 points for photo or list 15 points for comments on profile pages, photos or lists
Badges were awarded 4 tiers of badges were based on accumulated points Badges were displayed on user’s profile page
Leaderboard Showing points
For an initial 6 month period half of the user’s had gamification, other half didn’t and didn’t know it existed. After 6 months everyone got it. 10 months after initial introduction, it was completely removed.
Participants – 3486 members who contributed at least one item of content during a four week analysis period
4Sanch
ez’s To
p 1
0 o
f 201
2
THOM, J.; MILLEN, D. R.; DIMICCO, J. (2012). REMOVING GAMIFICATION FROM AN ENTERPRISE SNS. PROCEEDINGS ACM CONFERENCE ON COMPUTERS SUPPORTING COLLABORATIVE WORK.
Removing Gamification from an Enterprise SNS
Results The gamification construct dramatically increased content
contribution initially, but then decayed. New users who could earn points added more content over the
short and long term, but the proportion of new users who contributed was the same for the gamification Vs non gamification site users
The removal of the site significantly impacted the contribution of data
Two main types of comments were observed, terse comments and target of interest comments. After the gamification construct was removed, the prevalence of the terse comments (hi!) subsided
Conclusion – Gamification does motivate some, but not all. If you’re going to implement, have a plan for removing!
4Sanch
ez’s To
p 1
0 o
f 201
2
MELUSO, A.; ZHENG, M.; SPIRES, H.A.; & LESTER, J. (2012). ENHANCING 5TH GRADERS’ SCIENCE CONTENT KNOWLEDGE AND SELF-EFFICACY THROUGH GAME-BASED LEARNING. COMPUTERS & EDUCATION, 59, 497-504.
Enhancing Science Content Knowledge and Self-Efficacy through Games
Crystal Island Online 3-D game funded by NSF designed to teach
science concepts focused on narrative-centered learning
Collaborative Vs Single player Effects of condition on science content learning and
science self-efficacy were evaluated Participants – 66 Fifth graders
3Sanch
ez’s To
p 1
0 o
f 201
2
MELUSO, A.; ZHENG, M.; SPIRES, H.A.; & LESTER, J. (2012). ENHANCING 5TH GRADERS’ SCIENCE CONTENT KNOWLEDGE AND SELF-EFFICACY THROUGH GAME-BASED LEARNING. COMPUTERS & EDUCATION, 59, 497-504.
Enhancing Science Content Knowledge and Self-Efficacy through Games
Results Students did not differ on science self-efficacy between
conditions, but all players demonstrated increases in science self-efficacy after playing Crystal Island
Students did not differ on science content knowledge between conditions, but all players made significant learning games from pre-post test assessments after playing Crystal island
In this case, cooperative did not have significant impacts over single player game-play
3Sanch
ez’s To
p 1
0 o
f 201
2
SMITH, P.A. (2012) COOPERATIVE VERSUS COMPETITIVE GOAL STRUCTURES IN LEARNING GAME. A DISSERTATION SUBMITTED TO THE UNIVERSITY OF CENTRAL FLORIDA.
Cooperative Vs Competitive Goal Structures in Learning Games
Acquisition Proposition Acquisition based diner dash style resource management
game designed to teach which documents were needed when Game modified to be cooperative, in which two players could
work together or competitive in which two players tried to get a higher score
3 Experiments, 2 versions of game vs text based content Coop Vs Comp (no instructions for comp) Coop Vs Comp (told to compete) Coop Vs Comp (Winner gets a $10 iTunes card)
Participants – 160 Undergraduates 18-22
2Sanch
ez’s To
p 1
0 o
f 201
2
SMITH, P.A. (2012) COOPERATIVE VERSUS COMPETITIVE GOAL STRUCTURES IN LEARNING GAME. A DISSERTATION SUBMITTED TO THE UNIVERSITY OF CENTRAL FLORIDA.
Cooperative Vs Competitive Goal Structures in Learning Games
Results All game players had significant learning outcomes In Exp 1 (no instruction to compete), the cooperative group
reported a larger increase in intrinsic motivation. Exp 2 & 3 saw overall increases in intrinsic motivation. Winners of all competitive games had higher self efficacy than
losers. Game winners who won $10 had higher self efficacy than
cooperators in Exp 3. Conclusion – The use of competition in games is a complex
variable that can have impacts on learning, motivation and self efficacy. If you want to use it, read this paper!
2Sanch
ez’s To
p 1
0 o
f 201
2
SHAREK, D. J. (2012). INVESTIGATING REAL-TIME PREDICTORS OF ENGAGEMENT: IMPLICATIONS FOR ADAPTIVE VIDEO GAMES AND ONLINE TRAINING. A DISSERTATION SUBMITTED TO NORTH CAROLINA STATE UNIVERSITY.
Investigating Real-Time Predictors of Engagement
Gridblocker Isometric tile-based puzzle game in which player must move a
block until the block ends up standing over a goal. Multiple levels of increasing complexity.
3 conditions Linear – Players get a harder level when they complete the
previous easier level Choice – Players choose whether the next level will be easier or
harder than the just completed level Adaptive – Uses an algorithm to determine the difficulty of the
upcoming level Goal – To test the impact of condition on engagement Participants – 340 people recruited through Amazon
1Sanch
ez’s To
p 1
0 o
f 201
2
SHAREK, D. J. (2012). INVESTIGATING REAL-TIME PREDICTORS OF ENGAGEMENT: IMPLICATIONS FOR ADAPTIVE VIDEO GAMES AND ONLINE TRAINING. A DISSERTATION SUBMITTED TO NORTH CAROLINA STATE UNIVERSITY.
Investigating Real-Time Predictors of Engagement
Initial results indicated that those in the adaptive or choice conditions did not report significantly: Higher engagement Higher personal affect Lower levels of cognitive load
Adaptive algorithm analysis: Choice players should have played easier levels less often than they
chose Those in adaptive condition played fewer levels yet achieved
greater difficulty than other conditions. No differences in length of time played.
Conclusion – Adaptive leveling when done appropriately can lead to exposure to more challenge (and choice can lead to players selecting less challenge)
1Sanch
ez’s To
p 1
0 o
f 201
2