MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
GaLAGame and Learning Alliance
The European Network of Excellence on Serious Games
SIG3.5 – SGs for Humanities & Heritageuser studies
Michela Mortara, Chiara Eva Catalano, CNR – IMATI
Jeffrey Earp, CNR-ITD
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Overview
Icura [ongoing, 50%]
• Playing History – The Plague [started, 20%]
Playing History – The Plague [started, 10%]
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
USER STUDY #1: ICURA
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Game description
• Educational objective: cultural awareness - Japanese etiquette
• Genre: adventure• Environment: 3D • Platform: PC• Intended educational context: informal (at home)• Developer: TU-Wien• Duration: less than 1 h• Target: secondary school and higher• Previous studies: evaluation done by the developers at
the Wien university on a group of 20 students. Reports on papers and PhD. thesis
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Previous evaluation
• Pre-questionnaire (demographic data & game habits)• Pre-test with 12 questions ( on language basics, behaviour
& etiquette, culture and society)• Game session• Post-test (same 12 questions)• Final questionnaire about user satisfaction
Observations:• On average from +5 to +10 correct answers btw pre and
post– Selective attention bias
• Learning gain linked to the way the content is communicated (information agent vs quest)
• Lacking estimate of retention
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Our setting
• Same phases but different questions in the pre-test to avoid the selective bias
• Higher participant number (CNR, RWTH, ORT, DIBE)• Open answers to assess higher level in Bloom’s
taxonomy (post questionnaire) e.g.:• What have you learnt from the game and how (through what mechanics)? • Did the game increase your interest towards Japan/ Japanese culture?
Would you read more about Japan? Would you visit it?• How would you summarise Japanese culture in 3 adjectives?• What can you deduce about Japanese culture from the facts you learnt in
the game? • How would you compare Japanese and Western culture?
• Plan to repeat the post test after a few months to evaluate retention
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Current status
• CNR session– 10 university students– Colleagues&friends
• RWTH session– 17 university students
• ORT session– 12 people– students
• DIBE session
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Partial outcomes
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Partial outcomes
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Partial outcomes
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
PLAYING HISTORY – THE PLAGUE
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
The game
• Educational objective: history, historical reconstruction • Genre: adventure• Environment: 3D • Platform: browser (Unity Web player)• Intended educational context: formal (at school)• Developer: Serious Games Interactive• Duration: nearly 3 hours• Target: 8-13• Previous studies: evaluation done by a master student
with two primary school classes in Denmark. Report in her Master thesis (in Danish).
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
The experience: a Joint Research Action
• JRA involving CNR-IMATI (SIG3.5), CNR-ITD (TC2.8, T6.3), SGI and DIS (Deledda International School) in Genova
• Objectives:– explore the suitability and effectiveness of serious games in the
field of humanities and cultural heritage in schools;– evaluate the educational efficacy of the game and the level of
student engagement and enjoyment triggered by the learning activities;
– investigate critical aspects of SG deployment in formal ed./school contexts, especially teacher’s role
– examine cultural/educational issues related to game’s adoption in Italy (game produced in northern Europe and set in medieval Italy)
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
The context
• School: English-speaking school in Genoa following International Baccalaureate & IB Middle Years programmes
• Classes: 2 classes Year 7 (11-12 year olds). Approx. 35 pupils
• Deployment settings: computer lab, classroom, home access
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Research framework
• Approach: qualitative, design based research – DBR Collective
• Tools:
– question matrix for gathering teachers’ GBL/ICT experience & attitudes, plus their expectations, intentions and goals for the experience
– pedagogical planning tool (teachers document intentions/goals of intervention and plan learning activity sequence)
– researchers’ observation sheets for monitoring classroom activities
– pupils’ pre/post tests (in teacher’s resource pack from SGI)
– mind map recording pupils’ collective pre/post knowledge/awareness
– teachers’ monitored focus group
– teacher questionnaire
– reflection sheet for recording researchers’ attitudes, intentions, goals and interventions in the experience
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Activity Phases 1 & 2…
1. Teacher preparation
– introduction to the experiment, the game and its teaching resource pack
– teachers’ group discussion of matrix questions (researcher compiled)
– tech. training session (game’s teacher functions; ITD pedagogical planner)
– teachers consolidate pedagogical plans
2. Deployment
– pupil’s individual pre-test & collective brainstorming of pre-knowledge (making collective mind map)
– class briefing: game presentation + play
– gameplay sessions in pairs (plus possible home play)
– pupil summative test & debriefing (re-examining collective mind map)
= total 4/5 school periods with researcher monitoring & support
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
… Activity phases 3 - 6
3. Teacher debriefing
– focus group (inc. re-examination of matrix questions and pre-deployment responses, and of pedagogical plan/s)
– final questionnaire
4. Analysis of data gathered from phases 1-3
5. Follow up
– Repeat of post test to evaluate pupil retention
6. Final data analysis and reporting
MadridApril 15-17, 2013
European Commission Information Society and
Media
General Assembly
Current status
• Series of organisational meetings held involving CNR, DIS & SGI
• Activity phase 1 commenced– introduction to teachers– discussion / compiled of teachers’ of question matrix
Issues with running game on DIS network have delayed commencement of deployment sessions with pupils• Solution: DIS agreed to move students to CNR (2
sessions in May)