push notifications, digital badges & leaderboards: evaluating the impact of quitch mobile...
Post on 12-Apr-2017
55 Views
Preview:
TRANSCRIPT
CRICOS 00111D TOID 3059
Push notifications, digital badges &
leaderboards: Evaluating the impact of
Quitch mobile application on learning
Dr Grainne Oates FLB, Senior Lecturer, Accounting/FinanceDr Katya Pechenkina LTU, Research FellowDan Laurence LTU (formerly), Learning DesignerDr Daniel Eldridge FSET, Lecturer in ChemistryProf Dan Hunter FBL, Foundation Dean, Swinburne Law
Presented at Transforming Learning Conference Swinburne University of Technology13-14 September 2016Melbourne Australia http://transformconference.com/ #SwinTLC
Outline
• Mobile apps for learning
• Quitch: Rationale & elements
• Quitch: Development process
• Measuring impact
• Findings
• Further steps
Mobile apps for learning
• Challenges: low retention & high failure rates
• Students likely to access content via mobile
phone (or a similar device) not desktop computer
• Push notifications
• Student expect immediate constructive feedback
• Self-testing has capacity to improve outcomes
• Multitude of mobile apps - rarely tailored to
lecture content
Quitch: Rationale & elements
• Engagement loops
• Regular ‘push-notifications’
• Dynamic timed quiz
• Points & badges
• Leaderboards
• Progression trees
• Videos & forums
A solution should:
• Be easy for educators to use
• Have real time analytics for early intervention &
adaptive learning
Quitch: Development process
Quitch: Development process
Quitch: Development process
Quitch: Analytics
Possibilities:
• Reengage ‘at risk’ students
• Test if lecture content is comprehended
• Adapt course material dynamically
• Provide regular personalised feedback
Measuring impact• Is there a relationship between introducing a
mobile app into the classroom and student
retention rates?
• Is there a relationship between the use of
the app and academic performance?
• Is student performance within the app
related to academic performance?
• Do students respond positively to the
introduction of the app?
Measuring impact: Methods• Two cohorts taking part in the Quitch trial
(S2, 2015)
Accounting (462 enrolled, 265 app users)
Science (249 enrolled, 129 app users)
• Total sample: N=711, with 55% using the
app (394) + prior cohorts for comparison
• Data collected (with student consent):
Final grades compared to the pre-app cohort
App uptake & engagement analytics
Small-scale qualitative online survey (N=8)
Student feedback from end-of-year surveys
Findings
Is there a relationship between introducing a
mobile app into the classroom and student
retention rates?
• The app introduction was associated with a
positive improvement in student retention
(calculated at 12.23%) compared to the pre-
app semester
Findings
Is there a relationship between the use of the
app and academic performance?
• App users averaged a percentage mark of
65.19% compared to students who did not
use the app (who averaged a percentage
mark of 58.16%).
• App users on average achieved marks that
were 7.03% higher than students who did not
use the app
Findings
Is student performance within the app related to
academic performance?
• For app user there was a positive correlation
of .40 between performing well on the app
and achieving higher academic grades
Findings
Do students respond positively to the introduction of the
app?
The app “reinforced the content” & “kept [students]
involved and invested” by appealing to their competitive
nature
Leaderboards & badges: allowed students to see how they
placed compared to their peers. However: it is “dangerous
to start ranking people” & some students felt “exposed”
Immediate feedback was desired in addition to right/wrong
answer indicator
Further steps
• Correlation ≠ causation
• In-depth studies of student &
educator experiences needed
• Implications for universities,
educators & students
• Tailored content, timed
content-aligned quizzes,
immediate feedback, rankings,
flexibility
Thank youPresented by Dr Katya PechenkinaResearch FellowThe Office of the Senior DVC & ProvostLearning TransformationsEmail: epechenkina@swin.edu.auTwitter: @katya_pechenk
top related