gaming learning analytics real colegio complutense harvard
TRANSCRIPT
Gaming Learning Analytics: using data for improving serious games applicabilityReal Colegio Complutense at Harvard
Baltasar Fernandez-Manjon [email protected] , @BaltaFMe-UCM Research Group , www.e-ucm.es
Cambridge, 07/04/2016
Cátedra Telefónica-Complutense Educación Digital y Juegos Serios
2
Serious Games: definition and use• Any use of digital games with purposes other
than entertainment (Michael & Chen, 2006)• Serious Games have probed to be educationally
effective in several domains– Medicine, military, business, corporate training
• But still is a low adoption of Serious Games in mainstream education
• Serious Games considered usually as a complementary content– Mainly used for motivational purposes– No actual impact on the final mark
Data in Education: LA and GLA• Learning Analytics Improving education based
on data analysis– Data driven– Evidence-based education
• Gaming Learning Analytics is a specific case when all interaction data is used in serious games for improving the learning process supported by the games– Educational games not as “black boxes”
4
What happens when new technology is used in education?Serious Games….
Educational games in the classroom: game
• The Radix Endeavor– https://www.radixendeavor
.org• MMOG with Science,
Technology, Engeneering and Math (STEM) topics– Many different quests
• Adapted to middle and high school curriculum
• Facilities for teachers– Class configuration– Quest assignation– Tested in actual school
settings
Educational games in the classroom: settings
• Master students in the e-learning class– 13 students, 8
nationalities• No previous info about the
game– If stuck ask for help or
watch the help videos• One hour free play
– Who perform “better” at 30 minutes and at the end
• Publish screen captures in the class Moodle course forum
Educational games in the classroom: outcome• Students were really engaged for the whole hour
– They enjoyed the experience– Consider the game a commercial quality game – No one asked for help
• All of them published the screenshots in the Moodle class forum
• Educational outcome– Only 4 of the students complete more than 2 quests – Most of the students just spend the time exploring the
environment but not doing anything “useful” or learning anything
• Actual data can be contradictory with simple observation
8
Other experiments ..
• Experiment: Gamified experience in the classroomGoals– First to complete the game– Maximum “points” in the gamebut– Initially students were not informed that
Learning Analytics was in place
After some playing time LA data was showed to the students
9
Using data interaction in the classroom
GLA simplify SG deployment in the classroom‣ Teacher knows what is happening in the
class‣ Teacher “keep the control”
Behavior change when students know that LA was in place
• More oriented to success and less exploration or “playing”
Serious games at school (USA)
Takeuchi, L. M., & Vaala, S. (2014). Level up learning: A national survey on teaching with digital games. New York: The Joan Ganz Cooney Center at Sesame Workshop
• 62% teachers have used games to simulate and supplement learning.
• (Harris Poll, 2014 –Pearson-, 1000 teachers K12, USA)
11
How teacher selects SG
Uses of Gaming Learning Analytics in educational games
• Game testing – game analytics– It is the game realiable?– How many students finish the game?– Average time to complete the game?
• Game deployment in the class – tools for teachers– Real-time information for supporting the teacher– “Stealth” student evaluation– Knowing what is happening when the game is
deployed in the class• Different uses for different stakeholders
– Teachers, students, administrators, – Game developpers, researchers
13
Can we generalize and simplify the adoption of Gaming Learning Analytics?RAGE Project….
ProsocialLearn
RAGENo-one-
left-behind 3d-
Tune-In
BEACONING
XX XX
XXXX
XX
XX
XX
XXXX
XX
XX
EU H2020 serious games projects
H2020 RAGE in a nutshell
• RAGE will deliver advanced technology and know-how to support the European Applied Games industry build-up and job creation– February 2015, 4 years, 20 partners, 9M €
• Creating a new serious games ecosystem by making available a set of reusable technology components for developing advance serious games easier, faster and more cost-effectively
• Open source gaming learning analytics framework – provide all the required services (e.g. game tracker,
learning analytics server, visualization of analytics information)
– easy inclusion of gaming learning analytics techniques in the new games
Realising an Applied Gaming Eco-System
RAGE: creating the Gaming Learning Analytics infrastructure
H2020 RAGE project will simplify the process of Serious Games creation with ready to use assets• Game trackers (Unity3D, javascript)• LA server infrastructure and services
• Basic analysis and visualization of LA data• Extensible architecture
• Standards support (e.g. xAPI)• Open code (github)Using interaction data for assessment
Realising an Applied Gaming Eco-System
17
RAGE GLA Open Architecture
Clients AA Applications
Games
Analytics Frontend
AAFrontend
Play
ers
Devs
, Stu
dent
s,
Teac
hers
.Ad
min
s
UsersRolesResourcesPermissionsApplications
Auth
entic
ation
Auth
oriza
tion
JWTJSON WEB
TOKEN
GamesSessionsResults
TopologyAnalytics Backend
GamesSessions
Collector
Results
Application 1
Application N
Kafk
a Q
ueueLRS
We reuse APEREO Open LRS
18
New Reference Models for GLA
Implications of GLA in the design of the game
RAGE, serious games and xAPI• Define interactions model to assess serious
games• Create a xAPI vocabulary to represent the model–Repurpose existing verbs and activities–Create verbs and activities where needed–Create extensions where needed
• Define analysis for the model– Simple metrics processing statements
(serious game agnostic)–More advanced analysis based on serious
game goals
H2020 Beaconing project
• BEACONING stands for ‘Breaking Educational Barriers with Contextualised, Pervasive and Gameful Learning’ • Started in january 2016, 15 partners, 9 countries, 6M
• Global goal is learning ‘anytime anywhere’• Exploitation of technologies for contextual pervasive games and use of
gamification techniques• Problem based approach to learning• Enriching the Gaming Learning Analytics data model with
the contextual, geolocalized and accessibility information• Large pilots in real settings with content providers
– Formal and informal learning across virtual and physical spaces• GLA is a key element in the games and pilots
evaluation• Using RAGE infrastructure and extending it for these
new requirements and applications
21
New models for specific uses: games for people with cognitive dissabilities
22
Cytopathology Challenge
GOALS• Educational application for an Introduction to Cytopathology
course in cooperation with Harvard Med School and MGH• Train cytologists both in Harvard U and in limited-resource
countries CONSTRAINTS• Work in Low-cost Android tablets (50$)• Work in web browsers • No always wifi-connection to play• Limited budget
23
Cytopathology Challenge
2.Send traces
1.Play Cytopathology Challenge
3.Analyze data
4.Recover results
5.Show analysis to evaluate the students
Scientific Goals: from formal pre-post evaluation to GLA?• Off-line learning analytics• Formal evaluation of
games is very complex and expensive– Pre-test– Post-test
• Very few games have been formally probed to be effective
• Similar results with Learning Analytics than with pre-post test?
24
Conclusions
• GLA in Serious Games has a great potential from the application and research perspective
• Still complex to implement GLA in SG– Increases the (already high) cost of the games– New specifications could simplify the task (e.g.
xAPI)• New open frameworks produced by EU projects and
new standards specifications could greatly simplify GLA implementation and adoption
• And this is just the beginning (BYOD, VR, new interactions devices..)– Games are driving the disruption ….
26
Thank you!•¿Questions?• Mail: [email protected]• Twitter: @BaltaFM
• GScholar: https://scholar.google.es/citations?user=eNJxjcwAAAAJ&hl=en&oi=ao • ResearchGate: www.researchgate.net/profile/Baltasar_Fernandez-Manjon• Slideshare: http://www.slideshare.net/BaltasarFernandezManjon