Push notifications, digital badges & leaderboards: Evaluating the impact of Quitch mobile application on learning

Download Push notifications, digital badges & leaderboards: Evaluating the impact of Quitch mobile application on learning

Post on 12-Apr-2017

53 views

Category:

Technology

2 download

TRANSCRIPT

  • CRICOS 00111D TOID 3059

    Push notifications, digital badges &

    leaderboards: Evaluating the impact of

    Quitch mobile application on learning

    Dr Grainne Oates FLB, Senior Lecturer, Accounting/FinanceDr Katya Pechenkina LTU, Research FellowDan Laurence LTU (formerly), Learning DesignerDr Daniel Eldridge FSET, Lecturer in ChemistryProf Dan Hunter FBL, Foundation Dean, Swinburne Law

    Presented at Transforming Learning Conference Swinburne University of Technology13-14 September 2016Melbourne Australia http://transformconference.com/ #SwinTLC

  • Outline

    Mobile apps for learning

    Quitch: Rationale & elements

    Quitch: Development process

    Measuring impact

    Findings

    Further steps

  • Mobile apps for learning

    Challenges: low retention & high failure rates

    Students likely to access content via mobile

    phone (or a similar device) not desktop computer

    Push notifications

    Student expect immediate constructive feedback

    Self-testing has capacity to improve outcomes

    Multitude of mobile apps - rarely tailored to

    lecture content

  • Quitch: Rationale & elements

    Engagement loops

    Regular push-notifications

    Dynamic timed quiz

    Points & badges

    Leaderboards

    Progression trees

    Videos & forums

    A solution should:

    Be easy for educators to use

    Have real time analytics for early intervention &

    adaptive learning

  • Quitch: Development process

  • Quitch: Development process

  • Quitch: Development process

  • Quitch: Analytics

    Possibilities:

    Reengage at risk students

    Test if lecture content is comprehended

    Adapt course material dynamically

    Provide regular personalised feedback

  • Measuring impact Is there a relationship between introducing a

    mobile app into the classroom and student

    retention rates?

    Is there a relationship between the use of

    the app and academic performance?

    Is student performance within the app

    related to academic performance?

    Do students respond positively to the

    introduction of the app?

  • Measuring impact: Methods Two cohorts taking part in the Quitch trial

    (S2, 2015)

    Accounting (462 enrolled, 265 app users)

    Science (249 enrolled, 129 app users)

    Total sample: N=711, with 55% using the

    app (394) + prior cohorts for comparison

    Data collected (with student consent):

    Final grades compared to the pre-app cohort

    App uptake & engagement analytics

    Small-scale qualitative online survey (N=8)

    Student feedback from end-of-year surveys

  • Findings

    Is there a relationship between introducing a

    mobile app into the classroom and student

    retention rates?

    The app introduction was associated with a

    positive improvement in student retention

    (calculated at 12.23%) compared to the pre-

    app semester

  • Findings

    Is there a relationship between the use of the

    app and academic performance?

    App users averaged a percentage mark of

    65.19% compared to students who did not

    use the app (who averaged a percentage

    mark of 58.16%).

    App users on average achieved marks that

    were 7.03% higher than students who did not

    use the app

  • Findings

    Is student performance within the app related to

    academic performance?

    For app user there was a positive correlation

    of .40 between performing well on the app

    and achieving higher academic grades

  • Findings

    Do students respond positively to the introduction of the

    app?

    The app reinforced the content & kept [students]

    involved and invested by appealing to their competitive

    nature

    Leaderboards & badges: allowed students to see how they

    placed compared to their peers. However: it is dangerous

    to start ranking people & some students felt exposed

    Immediate feedback was desired in addition to right/wrong

    answer indicator

  • Further steps

    Correlation causation

    In-depth studies of student &

    educator experiences needed

    Implications for universities,

    educators & students

    Tailored content, timed

    content-aligned quizzes,

    immediate feedback, rankings,

    flexibility

  • Thank youPresented by Dr Katya PechenkinaResearch FellowThe Office of the Senior DVC & ProvostLearning TransformationsEmail: epechenkina@swin.edu.auTwitter: @katya_pechenk