no use reasoning with adolescents? a randomised controlled trial comparing persuasive messages

03.05.2023 1Twitter: @heinonmattiFacebook: Heinon Matti

SMS reminders to increase accelerometer

wear-timeA within-trial RCT comparing persuasive messages

Reg. no: DRKS00007721


Background

Twitter: @heinonmattiFacebook: Heinon Mattimattiheino.wordpress.com 03.05.2023 3

• Physical activity recommendations based on self-reported activity levels → problems:• Remembering past activity• Reporting “what the researcher wants to hear”

• Solution: objective measurement devices• New problem: need to wear it for most of the study

period!‒ E.g. if you only wear the activity device when exercising,

looks like 100% of your day was spent working out!

The ”why”

?

Twitter: @heinonmattiFacebook: Heinon Mattimattiheino.wordpress.com

The ”why”

• Let’s Move It ‒ A school-based multilevel intervention to

increase physical activity and decrease sitting among youth*

‒ Ongoing since 2012‒ RCT phase from 2015 to 2017

‒ Ca. 16–19 year-old vocational school students

‒ Waist-worn accelerometers used

*Hankonen, Heino et al. (in preparation). Randomised controlled feasibility study of a school-based multi-level intervention to increase physical activity and decrease sedentary behaviour among older adolescents.


The Pervasive Problem: non-wear in accelerometry

• Let’s Move It Feasibility study• Students fell short of accelerometer wear

targets (>10hrs of data on >4 days)‒ Qualitative work: non-wear attributed to

forgetting

How do we ensure adequate accelerometer wear times?


Methods


The setup

• Within-trial RCT during internal pilot study of the main trial

• Participants wear the accelerometer for seven consecutive days during Let’s Move It baseline data collection

• Fight forgetting with (SMS) reminders• Next question: What kind of reminders?

Could an old copy machine

help here?


Langer, Blank and Chanowitz (1978) Mixed success w/ replication: Folkes (1987);Key, Edlund, Sagarin and Bizer (2009)

”Harnessing the power of ’Because’”…?


Message types: an example

”Reason”, day 3:”Succinct”, day 3:

“Hello! Because the study wouldn't succeed without your help, please remember to put on the motion measurement device again and wear it until you go to sleep (except in the shower etc.) - thanks!”

[emphasis added]

“Hello! Please remember to put on the motion measurement device again and wear it until you go to sleep (except in the shower etc.) - thanks!”


Consenting in Let’s Move It accelerometryN=375

Opting in SMS n=276

Randomised to ”Reason” n=138

Randomised to ”Succinct” n=135

Opting out n=95

Participants:


• Does providing a reason via SMS influence accelerometer wear time:

‒ Total wear time‒ Number of days >10 hours of data accumulated (0-7)

Research questions


• ”Probability of observed (or rarer) data, if null hypothesis is true” • (also assumes randomisation, stopping rules etc…)

What’s a ”p-value” again?

When p is high (eg. p=0.32), no conclusions can be drawn!

(Dienes, 2014)

Reactions upon discovering this can vary.


• remember, p-value: Probability of data, given H0• A Bayes factor BF01:

A better question?”Which is more probable, null or alternative?

0 …∞1 31/3

Very roughly:

BF01:

Data favor alternative

Insufficient data

Data favor null

A great explanation: http://alexanderetz.com/2015/11/01/evidence-vs-conclusions/

When < BF < 10, evidence quite weak

http://alexanderetz.com/2015/11/01/evidence-vs-conclusions/

http://alexanderetz.com/2015/11/01/evidence-vs-conclusions/


Results


Total wear time


Total weartime differences

Bayesian ANOVA gives us BF01=29.03

1:1 (50%) prior odds become 1:29, p(effect)=3%

10:1 (91%) prior odds become 10:29, p(effect)=26%


Valid wear day percentages

Χ2(7) = 7.893, p = 0.342

BF01 = 1.27BF01 = 7.09


Measurement days of >10 hours of valid data

Horizontal lines represent means, with shaded 95% Bayesian Highest Density Intervals (HDIs)


Discussion


Conclusions

• Why didn’t reasons help?• Hidden moderators blah blah?• SMS format hinders the effect? (Why?)• Wearing the device a question of capability, not

motivation?• No use reasoning with adolescents?

‒ E.g. university students more compliant

• A case of an undead theory? ‒ Ferguson, C. J., & Heene, M. (2012). A Vast Graveyard of Undead

Theories.


Conclusions

• Why didn’t reminders help?• Operationalisation failure?

‒ Messages claimed to have been read but no objective log data

• Self selection? ‒ Unlikely, as opting in was mostly determined by recruitment

prompt

• Non-wear not due to remembering?‒ Although they said it was and thought the reminder helped

immensely!


The Lakatos principle:

Meehl, P. E. (1990). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it. Psychological Inquiry, 1(2), 108–141.

“Accepting the neo-Popperian view that it is inadvisable to persist in defending a theory against apparent falsifications […] the rationale for defending by non-ad hoc adjustments lies in the theory having accumulated credit by strong successes, having lots of money in the bank.“ – Paul Meehl

- Does the “power of because” lean on Monopoly money?


Thank you!

@heinonmatti

mattiheino.wordpress.com

Questions?

Comments?

Ideas?

https://linkedin.com/in/heinonmatti

no use reasoning with adolescents? a randomised controlled trial comparing persuasive messages

Health & Medicine