Critical Review of Intervention Studies Worksheet CAIS NCCMT On-Line Journal Club Oct 13, 2016 / Nov 17, 2016 Article: Stockwell, M. S., Westhoff, C., Kharbanda, E. O., Vargas, C. Y., Camargo, S., Vawdrey, D. K., & Castaño, P. M. (2014). Influenza Vaccine Text Message Reminders for Urban, Low-Income Pregnant Women: A Randomized Controlled Trial. American Journal of Public Health, 104(Suppl 1), e7–e12. http://doi.org/10.2105/AJPH.2013.301620

SAMPLE Comments

I. Are the Results Valid?

1. Did the trial address a clearly focused issue?

Yes P – low income, urban obstetric patients I – 5 weekly text messages re vaccines + 2 appointment reminder text messages, in addition to phone calls as in control C – usual phone reminders re appointments O – influenza vaccination

2. Was the assignment of patients to treatments randomized?

Yes- 1:1 allocation , randomization through SPSS, concealed to researcher

3. Were patients, health workers and study personnel ‘blinded’?

No - researchers blinded. Not possible to blind patients or health care personnel.

4. Were the groups similar at the start of the trial?

See Table 1 Yes – except for gestational age, which was accounted for in the adjusted analysis.

5. Aside from the experimental intervention were the groups treated equally?

Can’t tell. No note of other treatment, but cannot always assume no other difference in treatment.

6. Were all of the patients who entered the trial properly accounted for?

Yes – Figure 1. Accounts for all dropouts. However, no specific mention of Intention to Treat analysis

II. What are the results?

7. How large was the treatment effect?

See Table 2 – main results – absolute difference and relative risk, and adjusted Odds ratios. No statistically significant differences in Relative Risk (CIs all cross 1); no statistically significant difference in Absolute Risk (CIs all cross 0), Some small differences in adjusted Odds Ratios. Some interesting findings in sub analyses (table 3) – that women at 28-33 weeks GA at beginning of trial more likely to be vaccinated.

Critical Review of Intervention Studies Worksheet CAIS NCCMT On-Line Journal Club Oct 13, 2016 / Nov 17, 2016 Article: Stockwell, M. S., Westhoff, C., Kharbanda, E. O., Vargas, C. Y., Camargo, S., Vawdrey, D. K., & Castaño, P. M. (2014). Influenza Vaccine Text Message Reminders for Urban, Low-Income Pregnant Women: A Randomized Controlled Trial. American Journal of Public Health, 104(Suppl 1), e7–e12. http://doi.org/10.2105/AJPH.2013.301620

Based on Critical Skills Appraisal Programme. (2013). 11 questions to help you make sense of a trial. Retrieved from http://media.wix.com/ugd/dded87_40b9ff0bf53840478331915a8ed8b2fb.pdf

8. How precise was the treatment effect?

For statistically significant Odds Ratios – eg AORb By Dec 31, AOR (95% CI) is 1.30 (1.003, 1.69). Not very precise (wide confidence interval). Also, at lower end of CI (1.003) effect is very small

III. Will the results help locally?

9. Can the results be applied in your context?

Maybe – if technology already exists or does not cost too much to implement.

10. Were all clinically important outcomes considered?

Were participants previously vaccinated?

11. Are the benefits worth the

harms and costs?

Would look to see if there is a review, other trials, or wait until more evidence.