gamified learning activities in situ: lessons learnt with teachers and students

GAMIFIED LEARNING ACTIVITIES IN SITU:Lessons Learnt with Teachers and Students

Javier Melero, Davinia Hernández-Leo, Josep BlatInteractive Technologies Group

Universitat Pompeu Fabra

EEE Meeting – Leganés, 27-28 May 2013

OUTLINE


1. Introduction

2. A conceptual model for puzzle-based games design

3. Methodology

4. Evaluation

5. Results on teachers and students

6. Conclusions and future work

INTRODUCTION

3

• M-learning: Situated learning activities in physical spaces

• Benefits: Exploration skills and cooperation (Jeng et al. 2010)

• Previous Work: “QuesTInSitu” (Santos et al. 2011)

• Explorative and Spatial Skills

• Foster students’ motivation and self-assessment

Further research work involve improving students’ reflexion when performing situated learning activities


INTRODUCTION

4

How can we support more fruitful learning activities in situ that can be designed by teachers?

• Adopted approach: A metaphor based on puzzle games

• Educational Games: strengthen and support school achievement, cognitive abilities, motivation towards learning, reflection, attention and concentration.

• Puzzles: arrangement of a set of pieces into a single, well-fitting structure that interrelates them

• Benefits:

• Puzzle-based games can engage students in the subject topics, foster students’ problem solving, analytical and memory skills (Huang 2007; Bottino 2008).

• The nature of puzzle-based games seems relevant to consider as potential educational strategy to feasibly involve teachers as game designers (Huang 2007; Crawford 1982)

Research Question


INTRODUCTION

5

Main objectives:

• To propose a conceptual model, and its associated binding, for the design and computational representation of puzzle-based games

• To support the teachers the creation of in situ learning activities following the conceptual model

• Enactment with students

Objectives


PUZZLE-BASED GAMES

6

• Previous research studies about game design elements that have to take into account when designing educational games (Fisch, 2005; Jones, 1998; Kirriemuir et al., 2004; Malone, 1981; Sandford et al., 2005; Squire et al., 2003)

• A 4-dimension framework, considering the role of teachers, intended to evaluate the potential of using games- and simulation-based learning (de Freitas et al., 2006)

A Conceptual Model (1/2)

The proposed conceptual model considers:


PUZZLE-BASED GAMES

7

A Conceptual Model (2/2)

Graphical representation:


8

We explore how can we apply the puzzle-based game metaphor in the creation of question item-based activities

Considering students:

• Applying the puzzle-based game metaphor in situated learning activities enhance the students’ learning experience

• Aim: to engage students in reflecting on the correct solutions

• Similar to jigsaw puzzles: players could try to solve the different questions as many times as needed until reaching a correct solution

• Ways to find the correct solutions: reflecting on wrong past choices, consulting resources provided by the gamified application, discussing with other students, asking people, searching by the Internet, etc.

PUZZLE-BASED GAMESThe metaphor to design situated activities (1/2)


9

Considering teachers:

• Flexible metaphor for the design of gamified situated learning activities (independent of the subject matter)

• Elements considered in the metaphor to design the (assessment) in situ learning activities:

PUZZLE-BASED GAMESThe metaphor to design situated activities (2/2)

Board Geographical zones or museums rooms

Slots Questions designed for the gamified learning activity

Pieces Options associated to each question

Puzzle Groups of slots

Level Contains a puzzle. Levels as

Points Correct/Incorrect answers, consulting hints.

Bonus Extra points when all questions from a level have been correctly answered

Feedback Information associated to ranges of points

Hints Information of help to guide students to find the correct answer


10

• The teachers became co-participants in the design and analysis

• Involves social interactions with the teachers:

• Sharing ideas

• Looking at multiple aspects of the design and developing

• Involves different participants in the design (researchers and teachers

METHODOLOGYDesign-based Research Methodology (1/2)

Design

Enactment

Analysis

Re-design

• The methodology (Barab et al., 2004; Collins et al. 1992; DBRC, 2003) involves continuous cycles of:


11

Applying the design-based research methodology:

• Context: The puzzle-based game metaphor as an approach to create gamified in situ learning activities

• Participants: Teachers (designers) and students (end users) of secondary education

• Design: filling the templates containing the different key elements of the conceptual model

• Enactment: QuesTInSitu: The Game

• Analysis: Mixed evaluation method considering:

• Observations, tests, questionnaires, and log files (with students)

• Interviews and questionnaires (with teachers)

METHODOLOGYDesign-based Research Methodology (2/2)


http://youtu.be/BTSsXa_e-6M

12

EVALUATIONThree gamified situated learning activities (1/2)

MNAC Discovering Vic Discovering l’H

Teachers Involved

1 teacher 1 teacher 7 teachers

Purpose Activity associated to a subject, as a learning activity

Activity associated to a subject, as part of its formative assessment

Transversal activity in the school

Context Learning about different contemporary pictures of the MNAC

Learning about the city of Vic and its art history (unfamiliar city for most of the students)

Discovering and learning about the heritage of the city of l’Hospitalet (their own city)

Secondary teachers of different schools were involved in the design of their own gamified situated learning activities considering the puzzle-based game metaphor:


13

EVALUATIONThree gamified situated learning activities (2/2)

MNAC Discovering Vic Discovering l’Hospitalet

Number Levels 4 levels 4 levels 10 levels

Number Questions 20 questions 75 questions 55 questions

Points Correct Answers

50 points more 1 point more 250 points more

Points Incorrect Answers

10 points less 0.3 points less the first attempt, 0.5 point the second one, 1 point the third one.

100 points less

Number Hints 19 hints 25 hints 52 hints

Points Hints 50 points less 0.2 points less 100 points less

Extra Bonus 50 points more 1.5 points more when all the questions correctly answered at the first attempt, 0.75 points otherwise.

Proportional to the number of questions

Hints Content Short text about the context related to the question

Short text about the context related to the question

Suggestions rather than clues (ask people, read the information that appears next to the statue, etc.)

Levels Information Short sentence of the museums room

Short sentence of the geographical zone General information about the zone and particular information about the questions

Feedback Messages

Informal Formal Informal

Summary of the game design task for creating situated learning activities:


14

RESULTS


Educational benefits (metaphor)

Metaphor perceived as motivating, entertaining, educational

Metaphor allows consolidating knowledge and learning from mistakes

Metaphor is stimulating, encouraging, and motivating

Approach (answering as many times as needed)

Useful to help students to reflect but difficult for the teachers to know the students’ difficulties

The teacher totally agrees on the importance of the approach

Different ratings when asking about the importance of the approach*

Data gathering techniques: questionnaires (once the teachers finished their game design task) and interviews

Summary of results (according to the teachers’ opinions) [1/2]:

* A teacher pointed out that “I find the bonus, hints and punctuation more motivating and interesting than trying and trying to reach the correct answer or having a free hint. It is not bad, but I find these elements dispensable”


Teachers’ opinions (1/2)

15

RESULTSTeachers’ opinions (2/2)


Elements understanding

No problems understanding what the different elements mean

No problems understanding what the different elements mean

The “Level” element was the most problematic*. “Slot” and “pieces”, just at the beginning**

Hints Hints are useful to guide students, but not each question should have

Hints are important to guide students in case that they are lost or stuck

Hints are a good mechanism to advance in the game, but not all the questions should have

Bonus Important to keep students motivated

Important to keep students motivated

Important to keep students motivated

Punctuation Important since it may stimulate students to reflect on their decisions when selecting an answer

Highlights the importance of designing adapted punctuation depending on the number of wrong attempts

Good approach to allow students self-reflecting on their performance. 5 out of the 7 teachers found very important the adapted punctuation depending on the number of wrong attempts.

Feedback Important to students to reflect on their actions

The teacher did not find very important the feedback

Feedback are perceived as motivating and necessary

Summary of results (according to the teachers’ opinions) [2/2]:

* 6 out of the 7 teachers quite or totally agreed that they had difficulties understanding what a level means

** During the discussion of a meeting, some teachers argued that at first the metaphor is quite abstract, and it is needed to recall and interpret the meanings of each element [Observer-1]; but once the elements were understood, it was easy [Observer-2]


16

RESULTS


Following the metaphor

16/36 students don’t like to be forced to answer the questions until reaching the correct solution

52/63 students followed the metaphor

25/56 students fully followed the metaphor

18/36 students preferred having only one chance to solve the questions

45/63 students preferred this approach to only having one chance

32/56 students preferred this approach to only having one chance

28/36 students did not try to do it better in next attempts

53/63 students think more about the possible solutions and pay more attention

36/56 students totally agree that think better the solutions and pay more attention

Data gathering techniques: questionnaires (once the students finished the activity), observations (from researchers during the activity), log files (from the app.)

Summary of results (according to the students) [1/4]:


Students' Opinions (1/4)

17

RESULTS


Punctuation 29/36 students didn’t find appropriate and (30/36) motivating the amount of points when solving correctly the questions

47/63 students found appropriate and (47/63) motivating the amount of points when solving correctly the questions

45/56 students found (30/56) appropriate and motivating the amount of points when solving correctly the questions

14/36 students didn’t find appropriate and (14/36) discourage the amount of points when solving correctly the questions



Extra bonus 19/36 students didn’t find appropriate the amount of points

37/63 students found appropriate the amount of points

42/56 students found appropriate the amount of points

17/36 students didn’t find it motivating

46/63 students found it motivating





18

RESULTS


Hints No data gathered 9/14 groups of students accessed to the hints (max. 2 hints)

7/14 groups of students accessed to the hints (max. 5 hints)

0/18 Students found hints useful because (0/15) help them not to be stuck



3/16 students found appropriate the amount of points subtracted



2/20 students carefully accessed because of the points*



* Strategies followed by the students were: looking the information of the surroundings, asking people, searching by the Internet, all the members agreeing on the answers, and divide the questions among the members of the group [data from questionnaires and observations]




19

RESULTSStudents' Opinions (4/4)


Information’s levels

4/36 students found it helpful



17/36 students it found unnecessary

17/63 students found it unnecessary


Feedback 7/36 students it found motivating



16/36 students it found unnecessary





19

RESULTSComparing Students’ Performance


Particular case: students from the MNAC experiment using the gamified application vs. not using the gamified application:

• Post-test containing 8 random questions already answered in the app.

• Comparison of correct answer: gamified application vs. not gamified application

20

The conclusions, considering the teachers’ opinions, are:

• The metaphor allows the design of gamified learning activities in situ depending on the teacher’s purpose

• Concerns about having the possibility of answering several times the questions are mainly about knowing what the most problematic questions are to the students

• Problems understanding the “level” element

• The teachers can design their own punctuation depending on the activity’s purpose

• Different approaches followed to design hints: textual information vs. clues

• Results suggest that not all the questions should have associated hints

• Extra bonus valued as a good approach to motivate students

• Levels’ information design depends on the purpose: concrete vs. generalize

CONCLUSIONS AND FUTURE WORKConclusions (1/2)


21

CONCLUSIONS AND FUTURE WORKConclusions (2/2)

The conclusions, considering the students, are:

• Not to force students to answer all the questions for a given level. Like in jigsaw puzzles, one could leave it without finishing it

• Punctuation is not the most important information when situated activities are as a part of the students formative assessment

• Does not make sense having bonus when forcing the students to answer all the questions.

• Hints are not consulted because students follow other strategies to find the answers

• Feedbacks are necessary and motivating

• When interpreting these results, we should also consider contextual aspects (relation teachers-students, students age, socioeconomics factors, sense of freedom, etc.), and the teachers’ decisions for the game design


22

Future work includes:

• Evaluating the students’ performance on “Discovering l’Hospitalet”

• Evaluating a gamified design for “Discovering Sant Sadurni” in which 7 teachers have been involved

• Evaluating an authoring tool with teachers (excepted end of June)

CONCLUSIONS AND FUTURE WORKFuture Work


THANK YOU!!


Questions? Suggestions? Doubts?

gamified learning activities in situ: lessons learnt with teachers and students

Education

puzzlebased game metaphor

puzzlebased gamesthe

puzzlebased games design3

gamified learning activities

outlineeee meeting legans

puzzle games educational

nature of puzzle

fruitful learning activities