Does Gamification Work? A literature Review of Empirical Studies on Gamification

Does Gamification Work?(Hamari, Koivisto & Sarsa, 2014)Danille Duijst 11266988MSc Information Studies Human Centered MultimediaTechnology Enhanced Learning 23/02/2017

Welcome everyone,

My name is Danille and Im presenting today the paper about Gamification from Hamari, Koivisto and Sarsa. 1

ContentGamification and related conceptsDoes gamification work? (Hamari, et al)Practical examplesQ&A

First I will explain the concept of Gamification and related concepts.Then I will describe how to paper looks at itThirdly, I will show you some examples of gamified applicationsAnd lastly I will answer the questions from you guys.

If you have any questions, you can ask them also during the presentation.2

Applying game mechanics and game design techniques to non-game activitiesGamification

Gamificationis the concept of applying game mechanics and game design techniques to engage and motivate people to achieve their goals.

Not to be confused with serious games where people learn by actually playing: Game based learning3

Quick questionGamification relates to what kind of motivation?

Before we start to dive deeper, who thinks gamification relates mostly to extrinsic motivation? Who thinks intrinsic? Both?

Extrinsic, because its external motivations.Intrinsic, because of the self determination theoryBoth

*Most students said extrinsic*Some said both

I asked this question to make everyone already think about gamification in a way that it could be intrinsically motivating.4

Four types of players(Bartle, 1996)

Killer: wants to win

Achiever: wants to book results

Socializer: wants to collaborate

Explorer: wants to understand

When we are talking about gamification, we are going into the field of game design. Bartle found in 1996 that there are four types of players, motivated by different things:

de Killer Gets motivated by winning from othersde Achiever Gets motivated by booking results and excelde Socializer Wants to collaborate with others de Explorer Wants to understand everything in detail

Disclaimer: It was based on MUDs Multi User Dungeon5

MDA Framework(Hunicke, LeBlanc & Zubek, 2004)

Another common principle of game design is the MDA framework: Rules, System and Fun

Game mechanics as the particular components of the game, at the level of data representation and algorithmsGame Dynamics as the bigger picture, how Mechanics and components are working togetherGame Aesthetics as How the user is experiencing it, how much fun it is

Useful because the MDA framework forces us to think about our rationale for considering some of these game elements

6

MDA - Mechanics & DynamicsMechanics:PointsLevelsChallengesVirtual GoodsLeaderboardsGiftingWin ConditionsDynamics:RewardRank (status)ProgressionIndividualityCompetitionAltruismNarrativeLead to

How are Mechanics and Dynamics related? Here are some examples:

Mechanics : data representation and algorithmsDynamics : big picture aspects, the system 7

MDA Aesthetics (8 types of fun)Sensation Game as sense-pleasureFantasy Game as make-believeNarrative Game as dramaChallenge Game as obstacle courseFellowship Game as social frameworkDiscovery Game as uncharted territoryExpression Game as self-discoverySubmission Game as pastimeWhich one, do you think, is most relevant for learning applications like Duolingo?

LeBlanc et al, describe the Aesthetics as 8 types of fun when playing a game.

Which game aesthetic do you guys think is mostly relevant for e-learning?

I think Challenge because you can present learning challenges as obstacles you have to overcome, and that can be fun.8

So, does gamification work?(Hamari, Koivisto & Sarsa, 2014)

To answer this question, we will dive a bit deeper in the paper of Hamari et al.9

24 empirical studiesHigh-level research question: Does gamification work?Literature Review:ResultsDependent/Independent variablesContexts of gamificationTypes of studiesStructure Paper

How is the paper build up?24 empirical studies were examinedQuestion Does gamification work? Literature Review coversThe resultsIndependent variables such as Motivational affordancesDependent variables such as the psychological/behavioral outcomesContexts of where the gamification is used And the types of studies that were conducted in the reviewed papers10

Conceptualizing GamificationMotivational AffordancePsychological OutcomesBehavioral OutcomesIndependent VariableDependent Variable & Independent VariableDependent Variable

In the paper from Hamari et al, they say that gamification has three parts and in this review they focused on:What motivational affordances the reviewed studies have implemented as dependent variables (I think this is a mistake in the paper and it should be independent variables, because the affordances influence the psychological outcomes)What psychological outcomes have been measured as possibly both independent and dependent variables (Dependent because the motivational affordances influence the psychological outcomes, and also independent because these outcomes influence the behavioral outcomes)What behavioral outcomes have been measured as dependent variables.

11

Motivational Affordances (in accordance with Webster and Watson, 2002)Leaderboards (10)Points (9)Achievements/Badges (9)Challenge (7)Levels (6)Story/Theme (6)Feedback (6)Clear Goals (4)Rewards (4)Progress (4)

Related to the game mechanics we saw before from the MDA framework, we saw in the paper of Hamari et al, that they created a list of motivational affordances of gamification.

The numbers represent the amount of times they found this affordance in the 24 papers they reviewed.12

Psychological and behavioral outcomesBehavioral outcomes (behavioral change)21 papers

Psychological outcomes (motivation, attitude, enjoyment)12 papers

Most of the studies examined behavioral outcomes:Behavioral 21 papers

And less examined outcome:Psychological 12 papers13

Research MethodsQuantitative (17)Qualitative (2)Mixed Methods (5)

The research methods being used in the examined papers were mainly done in a quantitative way (17 papers).5 papers used mixed methods (5) and 2 papers researched in a qualitative way.14

Contexts of gamificationEducation / Learning (most common)Negative outcomes (competition, evaluation difficulties, design features)Work (crowdsourcing systems)No explicit studies in Marketing context

Gamification of education or learning was the most common context for the implementations. Most of the papers gamifying work were conducted in crowdsourcing systems.

All studies in education were positive, but some negative outcomes were:Increased competitionEvaluation difficulties Design features

Marketing: gamification and purchase behaviour, is an interesting and potential successful field. In the end of this presentation, I will show you an example.15

Reported Results Most reported positive resultsOnly 2 studies found all of the test positiveEngagement by gamification depends on several factorsMotivation of usersNature of gamified systemNovelty effect (long term)Removing gamificationBoth qualitative/quantitative have positive and negative perceptions

Reported results from the reviewed studies are:Most reported positive results, but only for some motivational affordancesOnly 2 studies reported that the result was positive with all the motivational affordancesEngagement by gamification depends on factors like motivation of users or nature of gamified systemNovelty effect: some studies showed that gamification effects might not be long termRemoving gamification has a negative effect on users that are still engaged, because probably they feel frustrated about losing earned badges/points.16

So, does gamification work? But some conditions should be taken into account:Not all gamification elements are effectiveQualitative observations highlight important factorsThe role of the contextQualities of the usersUse thorough methodologies to research gamificationYes!

The big question is again: does gamification work?

Yes, but certain things should be taken into account.Not all the gamification elements (motivational affordances) are effective. When using them seperately could have a different effect than using them combined.Qualitative observations highlight important factors, therefore when using gamification, it is important to also do qualitative research, because in this way you discover essential things. Like in this research, where they found out that the role of the context plays an essential role, and the qualities of the users. Qualitites of users means that users interact with the gamified systems in a different manner and therefore the experiences can differ.Also the methodologies have to be more rigorous, because most examined papers used a small sample size, some lacked control groups, some could have done better statistics, timeframes of the experiments were short, there was lack of clarity in reporting the results.17

DiscussionCan we intrinsically motivate people by using gamification elements?

As we heard in the lecture on Tuesday, intrinsic motivation works mostly more effective than extrinsic motivation. Do you think it is possible to motivate people intrinsically by using gamification?

No, because gamification only exists of external rewards and therefore gamification can only motivate extrinsically.

Yes, according to Decis self determination theory: if an extrinsic motivator is found to be meaningful, pleasurable and consistent with a persons worldview, he/she can adopt it as though it were intrinsic.

18

Time for some practical examplesBased on Motivational Affordances

Practical example: CodecademyMotivational Affordances

Feedback (when submitting)Progress Clear goalsAchievements/BadgesPoints

Feedback, when you submit your answer you get immediately explanation if its good or wrong and why.Progress in measurement of daysClear goals, when you do a specific track, you see which steps you have to takeAchievements/badges, when finishing a special track, or receiving certain amount of points.Points you receive for finishing excersises.20

Practical example: Fitbit Motivational Affordances

Feedback (stars, praise)Progress (step count)Clear goals (dashboard)Achievements/BadgesLeaderboardsChallenge

Feedback: you receive stars for achieving amount of steps or praise after doing a workoutProgress: you see your steps, calories, etc. in a dashboard kind of styleClear goals: when the circle is not filled yet, you see you have that as a clear goal of the day, or you manually set goalsAchievements: you receive achievements when walking a lot for exampleLeaderboards: you can compare your step count with friends Challenge: you can challenge your friends, who walks most in the coming weekend.21

Practical example: DuolingoMotivational AffordancesPoints (XP)Challenges (time, social)Achievements/Badges (Topics)Levels (Leveling up by XP)LeaderboardsFeedback (on answers)Clear GoalsRewardsProgress

As we heard yesterday, Duolingos main selling point is gamification. They basically use all the motivational affordances mentioned in the paper of Hamari et al.

Clear goals, because the user can see which badges or net yet gold, and the skills/topics are divided in a clear way.Rewards are also in the form of Lingot pointsProgress in progress bars, after finishing an exercise but also when you are actually practicing a topic, you see how much time or questions are left.22

Practical example: DuolingoMotivational AffordancesTheme

Recently, Duolingo launched a chatbot (only Iphone) to learn languages in a conversational way. This related to the story affordance.

Its interesting to see that 1 application uses all motivational affordances proposed by Hamari et al.

23

Thanks for your attention

Any questions?

Relationship of gamification and purchase behaviour: it is in fact a very interesting field, and the last couple of years companies are using gamification more often to increase purchase behaviour

Do you agree with the Novelty effect: is the tendency of people to increase performance because their interest is increased. I think this could be true for some cases, but the loose of interest/performance could also happen because the gamification wasnt implemented in a good way, the context was wrong or the users motivation decreased. Like if they only gave a star for every answer questioned right, nothing else, its not really motivating in the long term.

Does gamification increase the quantity of the motivation value or it changes the quality of how one approaches the learning processThis is an interesting question, I think gamification can increase the value of motivation for some people and for others it changes the way they look at the learning process. For some people, it is very motivating to get a star, they need a lot of feedback to get motivated. But they arent really aware of that they are in a learning process. While some other people might not need this star to be motivated, because they are intrinsically already motivated enough to do this task, but they can appreciate and find it fun that the learning process is approached in a different/gamified way.

Another way to find if gamification is effective in a learning situation: I think with this study, they wanted to also find out other things than only effectiveness, in this case: They found different contexts of use, different outcomes (behavioral/psychological) and the research method different studies used, so thats interesting.

How would I test if gamification works? I would try to focus more on a specific context. I would have one group not using the gamified learning system (control group) and one group using the gamified system. If its for a learning context, I would probably do a before and after measuring of a grade. The problem is with such a study, that you can only conclude something about 1 context. So the conclusion would be gamification in e-learning works (or not). When you want to say something about the general concept of gamification, you have to compare different studies with different contexts.

24

Gamification & marketing

I thought this was an interesting example of using gamification in marketing, already in 2011(!). You collect bonbons to create your own magnum in the end. They collaborated with different brands and you can control your avatar through websites of these brands, and interact in a quite nice and funny way with these websites. They increased their marketing KPIs massively.25

ReferencesHamari, J., Koivisto, J., & Sarsa, H. (2014, January). Does gamification work?--a literature review of empirical studies on gamification. InSystem Sciences (HICSS), 2014 47th Hawaii International Conference on(pp. 3025-3034). IEEE.Bartle, R. (1996). Hearts, clubs, diamonds, spades: Players who suit MUDs.Journal of MUD research,1(1), 19.Hunicke, R., LeBlanc, M., & Zubek, R. (2004, July). MDA: A formal approach to game design and game research. InProceedings of the AAAI Workshop on Challenges in Game AI(Vol. 4, No. 1).www.duolingo.comwww.fitbit.comwww.codeacademy.com