Presented by Menna Brown

Gamification and Adherence to Web basedGamification and Adherence to Web-based Mental Health Interventions: A Systematic ReviewReviewTheme: Protecting and Improving the Public’s Health

AuthorsAuthorsMenna Brown, Noelle O’Neill, Hugo van Woerden, Parisa Eslambolchilar, Matt Jones, Ann John

OverviewOverview

• Context • GamificationGamification• Methods• Results• Limitations • Conclusions

ContextContext Promoting well-being while improving and managing mental healthconditions is a worldwide priority (WHO)p y ( )

Common mental health disorders (CMD) and poor well-being havei ifi t i i l d i di id l tsignificant economic, social, and individual cost

Web-based psychological treatments to improve mental health and well-Web based psychological treatments to improve mental health and wellbeing outcomes are now well established and widely accepted.

However, adherence is a critical issue.

GamificationGamification“the use of game design elements in non-the use of game design elements in non-game contexts” Detarding et al 2011

Goal settingChallengeLevelsLevelsPointsProgressR dRewards Badges/trophiesGame leadersStory/themeFeedback

Fitbit• App and wearable device that

‘t k t d h l

PokemonGo• Get up, adventure outside and

t h ild P ké d ti‘tracks your steps and helps you reach your goals’.

• Compete (leader boards) with f

catch wild Pokémon – advocating behaviour change

• Interactive real world mapfriends and other users to try and go the furthest distance

• Track personal achievements and

• Catch as many Pokémon as you can to earn Experience Points (XP) and increase your Trainer

progress• Publish your information to your

social media channels

level• Earn medals for distance travelled • Unlock medals

Aim and MethodsAim and Methods Undertake a systematic review of peer reviewed randomised control trials(RCT’s) designed to manage CMDs or wellbeing which incorporated(RCT s), designed to manage CMDs or wellbeing, which incorporatedgamification features.

PROSPERO registration (CRD42015017689)

Seven electronic databases were searched: Medline, PsychINFO, CINAHL,Business source complete, Cochrane central, INSPEC,ACM digital library.

Search terms combined: “Web-based,” “intervention,” “CMD/wellbeing,” and “adherence”

Title/abstract followed by full text review by three independent reviewersData extraction –piloted data extraction form Risk of bias assessment carried out using Cochrane tool

Inclusion CriteriaInclusion Criteria• The intervention must include one or more gamification features;• Designed to manage any CMD or improve well-being (includingDesigned to manage any CMD or improve well being (including

physical conditions that report CMD/well-being outcome);• The intervention was delivered via the Web (Internet);• The intervention was designed to be accessed on more than one

occasion;• RCT study design; andRCT study design; and• The study must have reported at least one measure of attrition,

adherence, engagement, dropout, or other term referring to such.

ResultsResults 61 RCTs included = 82 interventions47 different intervention programmes identified

The majority managed depression using cognitivebehaviour therapy (CBT)

Eight of ten gamification features reviewed were in use.‘Levels’ and ‘Game leaders’ were not used

The majority utilised only one gamification feature(n=58) with a maximum of three features, 19 used twoGF d 5 d 3 GFGFs and 5 used 3 GFs

The most commonly utilised feature was ‘story / theme

Data analysisData analysis SPSS 22 and Review Manager 5.3Adherence to study protocol: primary outcome measure

A percentage score for adherence to each intervention was calculated toA percentage score for adherence to each intervention was calculated to allow comparison across interventions. This was the percentage of those completing post-assessment by the

b f ti i t i iti ll d i d (t i t ti t i l )number of participants initially randomized (to an intervention trial arm)

Used because limited data were available on total completion rate ofUsed because limited data were available on total completion rate of interventions

AdherenceAdherence • Overall adherence to study protocol ranged between 3.3%

and 100% (mean 71 7% SD 20 3%)and 100% (mean 71.7%, SD 20.3%)

• Mean adherence to intervention was lower overall than• Mean adherence to intervention was lower overall thanmean adherence to control when control was inactive71.7% vs 78.2%)% %)

• Mean adherence to interventions excluded (noMean adherence to interventions excluded (nogamification feature) was 75.2% (SD 19.6%) with a rangeof 5.3% to 100%.

Gamification and adherenceGamification and adherence Mean adherence % in studies using only one

72 375.9

276.380

90

g ygamification feature (n=58)

72.3

53.5

72

50

60

70

80

30

40

50

0

10

20

Goal setting (n=6) SD Progress (n=10) SD Feedback (n=3) SD Rewards (n=6) SD Story/theme (n=33)Goal setting (n=6) SD 22.8

Progress (n=10) SD 31.2

Feedback (n=3) SD 24

Rewards (n=6) SD 13.3

Story/theme (n=33) SD 17%

Mean adherence in studies using one, two or

80

Mean adherence in studies using one, two or three gamification features

78.2

76

78

71.570

72

74

70.5

66

68

70

66One feature (n=58) SD 21.6 Two features (n=19) SD17.9 Three features (n=5) SD 12.3

Mean adherence rate (%) by condition

60708090

100

2030405060

01020

Examination of Additional Intervention CharacteristicsExamination of Additional Intervention Characteristics

No significant statistical differences observed:

• Delivery format (sequential / free navigation) (P=.197)• Automated / guided (P=.05)• Intended duration (P=.14)• Modules (P=.80)

T t l b f i t ti f t (P 08)• Total number of interactive features (P=.08)

Standard multiple regression indicated that the independent variablesStandard multiple regression indicated that the independent variables only explained 10.3% (P=.22) of the variance in adherence.

LimitationsLimitations• Use of adherence to study protocol as the outcome measure (less insightful)

rather than completion data.• Only included RCTs • Varied reporting complicated initial identification of studies for inclusion• Interventions using gamification features in conditions other than depression• Interventions using gamification features in conditions other than depression

were small in number, which limited opportunity to explore the influence of gamification features on adherence across health conditions.

• Use of gamification features: Goal setting and feedback were aligned with• Use of gamification features: Goal setting and feedback were aligned with established strategies used in therapeutic treatment of CMD and their role is well defined in terms of supporting and encouraging behavior change.

• Adherence also may be influenced by additional factors that could not be• Adherence also may be influenced by additional factors that could not be assessed in this review

• Publication bias

ConclusionsConclusionsGaming features have been implemented in the design of interventions tomanage CMDs and well-being

No evidence that specific gamification features improved higher adherence to theintervention program as measured by adherence to protocol.intervention program as measured by adherence to protocol.

No evidence to suggest interventions incorporating additional gamificationfeatures had any statistically significant influence on adherence However nofeatures had any statistically significant influence on adherence. However, nostudies explicitly examined the role of gamification on program adherence orengagement.

Interventions intended to last 10 weeks or longer and those incorporating3GFs had higher mean adherence

Further researchFurther researchFurther research should look to examine whether application of specificgamification features influences adherence to protocol and completiong p prate:

C i t ti lt i ifi ti f t ( b• Compare same intervention altering gamification features (number,combination and type)

• Look across different health context s (different conditions)( )• Alternative therapies (ACT)• Navigation style• Assessment of participants motivation

ReferencesReferences

• Deterding S, Dixon D, Khaled R, and Nacke L. From game design elements to gamefulness: defining gamification. in Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments 2011 ACM doi:10 1145/2181037 2181040Future Media Environments 2011 ACM. doi:10.1145/2181037.2181040

• Cugelman B. Gamification: What It Is and Why It Matters to Digital Health Behavior Change Developers. JMIR Serious Games 2013 PMID:25658754

• Hamari K, Koivisto J, and Sarsa H. Does gamification work? A literature review of empirical studies ifi ti i 47th h ii I t ti l f t i 2014 6 9 H ii USAon gamification. in 47th hawaii International conference on system science 2014 6-9. Hawaii, USA.

doi 10.1109/HICSS.2014.377• Kelders S, Bohlmeijer E, and Gemert-Pijnen J. Participants, usage, and use patterns of a web-based

intervention for the prevention of depression within a randomized controlled trial. Journal of Medical Internet research 2013 15, e172. PMID:23963284

• Cochrane, Cochrane Handbook for Systematic Reviews of Interventions, ed. Higgins J and Green S. 2008 England: Wiley-Blackwell. ISBN 978-0-470-69951-5

Risk of BiasRisk of Bias