How would you like your learning? The influence of reward

uncertainty on motivation for learning

in computer games.

Skevi Demetriou CPLiC

“A sine qua non of successful learning is motivation: a

motivated learner can’t be stopped.”

(Prensky, 2003)

“Video games are not the enemy, but the best

opportunity we have to engage our kids in real learning. ”

(Prensky, 2003)

Broader project• Interdisciplinary approach:

Education & Neuroscience.• Computer game studies on

Learning.• Game built for the purposes of this

project.• Classroom & Laboratory based

studies.• Child & Adult participants. • Mixed methods approach

(interviews, video recordings, drawings, Electro Dermal Activity measurements, statistical data, etc).

• Qualitative & Quantitative data.• Individually & Collaboratively

working participants.

BROAD AIM:To investigate the

potential link between prediction error (PE),

engagement and learning in laboratory experiments

involving adults, and then use this understanding to explore how children engage with learning games involving

chance-based uncertainty in more “real world” classroom contexts, seeking to interrelate

this understanding with the discourse and social

constructions associatedwith such games.

Theoretical frameworkNeuroscientific evidence to show that:

• Dopaminergic reward activity in the brain (mid brain areas) has been shown to vary with prediction error (Daw et al., 2006).

• The dopaminergic activation (ventral striatum, NAcc) depends on the magnitude of the prediction error (e.g. Fiorillo et al., 2003; Schultz, 2006).

• Neuroscientific evidence to show that dopaminergic activity in the brain aids memory formation and thus factual learning – Two notions:

Direct impact: Uncertain rewards may promote memory formation through the dopamine release in the brain area called the hippocampus (Adcock, 2006; Callan & Schweighofer, 2008)

Indirect impact: The uncertain reward - memory link is mediated by attention (i.e. Loftus, 1972, Muzzio et al., 2009).

Dopamine is a neurotransmitter produced in

the midbrain and transferred to cortical and subcortical

regions (Treber et al, 2005). It is released at the synapse

between 2 neurons and allows the transfer of impulse (information) . It is the

transmitter used in these specific parts of the brain (4

main dopaminergic pathways).

4B. How arediscourse and

constructions influenced by whether the artificial opponent is matched in terms of gaming skill

or academicability?

1. How does prediction error, in

learning games employing chance-based uncertainty,

influence memory in adults?

2. How might prediction error, in

learning games employing chance-based uncertainty,

influence emotional engagement as

measured by EDA?

3A. How doesprediction error, in a

computer-based learning game employing chance-based uncertainty and an

artificial competitor, influence children’s

memory?

3B. How is this learning influenced

by whether the artificial opponent is matched in terms of gaming skill or academic ability?

4A. What types ofdiscourse and constructions

are associated with competitive learning games involving chance-

based uncertainty and an artificial competitor in the classroom, and how

might these be interrelatedwith our biological

understanding? RESEARCH

QUESTIONS(RQs)

Quasi-ExperimentalStudy

Quasi-Experimental Study• Aim:

To explore how reward uncertainty and in particular, positive prediction error (PPE), is related to fundamental learning processes (i.e. orientation of attention, memory encoding and recall).

• In Specific:

To identify instances when material was learnt and look for a relationship between them and the size of the PE in whatever game event had just previously occurred.

Procedure - Methods

Paper basedComputer

basedPaper based

Pre testPlay

computer game

Post test

Paper

based• 30 questions of

the game• Identical to post

test• 5-10 mins

Computer based

• 30 questions• 60 trials• 30 mins

Paper

based• 30 questions of

the game• Identical to pre

test• 5-10 mins

Lab settings

Electro Dermal Activity (EDA) measuring unit.

2nd thing to do:Answer the Question.

LEARNING EVENT

1st thing to do:Choose a bandit (box)

for points.GAMING EVENT

Participant’s score.

State confidence level.

Guidelines on what to do each time. Progress in the

game.

The “4-Arm bandit”

computer game used

in this study.

The learning content was «Tribes and their traditions». It was drawn from the book «The Golden Bough», by Sir James George Frazer.

The questions appeared randomly from a pool of 15 from the total of 30 questions available. A question was replaced by a new one in the pool when answered correctly once. The same questions repeated until learned.

Goal:To get the highest

possible score. Prize for the final

winner.

Timeline of events in the game

(PE = Box’s_Score_Now – Box’s_Score_Last sampled)

PEENC

PEREC

Question presented

FCorrect answer revealed

T Successful learning

Questionpresented

Unsuccessful learning F

Definition of learning in the computer game.

Methodology• Sample: 16 adults (7 males and 9 females). All participants were

working individually.• Methods - Measures:

Pre and Post tests

Recordings of participants’ choices in the game (PPE & learning)

Video recording • Variables:

Two continuous pseudo dependent variables (they were an alleged cause rather than effect): Prediction error at encoding (PEENC) and recall (PEREC).

One pseudo independent variable: Learning (2 levels: successful/learning-SL and unsuccessful/non-learning-UL).

• Hypotheses:

1 (for RQ1): In a learning computer game employing chance-based uncertainty, for encoding, prediction errors would be higher prior to successful than for unsuccessful learning. (PEENC) Successful learning > (PEENC) Unsuccessful learning

2 (for RQ1): In a learning computer game employing chance-based uncertainty, for recall, prediction errors would be higher prior to successful than unsuccessful learning.

(PEREC) Successful learning > (PEREC) Unsuccessful learning

1. How does prediction error, in

learning games employing chance-based uncertainty,

influence memory in adults?

Post_testPre_test

Test

26

24

22

20

18

16

14

12

10

8

6

4

Sco

re

Participants' pre-test and post-test scores

Results & Discussion• On average participants scored significantly

higher in the post test (M = 19.25, SE = .78)

than in the pre test [M = 8.75, SE = .36,

t (15) = -16.11, p < .0005, r = .97].• In response to RQ1 and Hypotheses 1&2:

On average, for encoding, PE for successful learning (M = 17.41, SE = 1.35) was not significantly higher than PE for unsuccessful learning (M = 15.27, SE = 1.31) even though the trend was in the direction hypothesised [t(15) = 1.71, 1-tailed: p = .054, 2-tailed: p = 1.1].

Discussion: - This was expecting the possible attention effect of the PE to survive about 18-24 seconds, (i.e. from when participants had entered their question answer and indicated their confidence rating, to when they received feedback). However, literature suggests that dopaminergic reward effects are quite short-lived – only a few seconds (Bogacz et al., 2007).

- The encoding was also occurring in the negative context of being told their answer was wrong.

Results & Discussion• On average, for recall, PE for successful

learning (M = 20.10, SE = 1.36) was significantly higher than PE for unsuccessful learning

[M = 17.55, SE = 1.35, t(15) = 3.51, 1-tailed & 2-tailed: p < .005, r = .67].

• Positive prediction error (PPE) is linked to successful learning.

• Happy surprise is linked to successful learning.

Conclusions• Learning to be potentially influenced by positive prediction error (PPE).• In line with literature suggesting that positive prediction error triggers memory and factual learning

formation. PPE as a “memory enhancer”.• Reward uncertainty increases dopamine release in the brain and therefore attention and motivation to

engage (e.g. Fiorillo et al., 2003).

• This was examined in the context of an educational computer game. Potential for Education.• This study was followed up with studies using this computer game in classroom settings.

Statistically significant results. • This could help device educational computer games that could promote motivation and learning.

Design “learning provoking” situations. Neurocomputational modelling.• Gaming as a computer based activity, is known for its potential to promote learning even in formal

educational settings when embedded properly and with awareness of its benefits and constraints. Technology is not a “panacea”. It is a tool.

• This study was followed up with studies on uncertainty-involving not computer based gaming.

Very encouraging results. • Whether involving computers or not, the gaming element itself, obtains powerful dynamics in

enhancing individuals’ motivation and is potentially powerful in making learning derived from such instruction efficient (e.g. Randel et al., 1992; Whitehall, & McDonald, 1993; Ricci et al., 1996).

Playing is, above all, “a privileged learning experience” (Rosas et al., 2003).

Thank you

Skevi Demetriou CPLiC

www.neuroeducational.net