Automatic classification of published clinical articles using metadata instead of content

Centre for Health InformaticsAustralian Institute of Health Innovation

Adam G. Dunn, Guy Tsafnat, Enrico Coiera

2

Our aim is to examine the clinical evidence for drugs to identify

(plus measure & find indicators for)the biases that make drugs look safe

and effective when they aren’t.

3

4

Prasad’s definitions of evidence reversals: 1. “We use the term reversal to signify the phenomenon of a new

trial—superior to predecessors because of better design, increased power, or more appropriate controls—contradicting current clinical practice… either less effective than previously thought or harmful”

2. 40% of articles in NEJM 2001-2010 testing the standard of care were defined as a reversal, where tested interventions were found to be worse than prior standards.

• Prasad et al. Reversals of established medical practices: Evidence to abandon ship. JAMA. 2012;307(1):37-38.• Prasad et al. The frequency of medical reversal. Archives of Internal Medicine. 2011;171(18):1675-1676.• Prasad et al. A decade of reversal: an analysis of 146 contradicted medical practices. Mayo Clinic Proceedings. 2013;88(8):790-

798.

5

Jefferson 2014: “…We believe these findings provide reason to question the stockpiling of

oseltamivir, its inclusion on the WHO list of essential drugs, and its use in clinical

practice as an anti-influenza drug.”Wang 2012: “The benefit of oseltamivir and zanamivir in preventing the transmission of

influenza in households is modest and based on weak evidence.”

Muthuri 2014: “encourage early initiation of neuraminidase inhibitor treatment in outpatients who are appreciably unwell with suspected or confirmed influenza, or at increased risk of complications, including those with influenza A H3N2 or influenza B.”Beck 2013: “NAIs should be deployed during a future pandemic for either post-exposure prophylaxis or treatment depending on national policy considerations and logistics.”

Neuraminidase inhibitors

6

Neuraminidase inhibitors

• 152 systematic and narrative reviews (2005-2013)• 510 authors (407 unique researchers)• 10,086 citations (4,574 unique documents)

7

8

Co-authorship network

k = 11X = 73 20 31 28accuracy = 0.6645precision = 0.7019recall = 0.7849f1-score = 0.7411---------------------------------

9

Citation similarity network

k = 17X = 83 10 34 25accuracy = 0.7105precision = 0.7094recall = 0.8925f1-score = 0.7905---------------------------------

10

Industry affiliations

11

Resistance & safety topics

12

Thank you

• AG Dunn, D Arachi, J Hudgins, G Tsafnat, E Coiera, FT Bourgeois (2014) Financial Conflicts of Interest and Conclusions About Neuraminidase Inhibitors for Influenza, Annals of Internal Medicine, [accepted].

• X Zhou, Y Wang, G Tsafnat, E Coiera, FT Bourgeois, AG Dunn (2014) Citations alone were enough to predict favourable conclusions in reviews of neuraminidase inhibitors. Journal of Clinical Epidemiology [accepted].

• AG Dunn, E Coiera (2014) Should comparative effectiveness research ignore industry-funded data? Journal of Comparative Effectiveness Research, [accepted].

• K Robinson, AG Dunn, G Tsafnat, P Glasziou (2014) Citation networks of related trials are often disconnected: implications for bidirectional citation searches, Journal of Clinical Epidemiology, 67 (7): 793-799. doi:10.1016/j.jclinepi.2013.11.015.

• AG Dunn, FT Bourgeois, E Coiera (2013) Industry Influence in Evidence Production, Journal of Epidemiology & Community Health, 67:537-538. doi:10.1136/jech-2013-202344.

• AG Dunn, B Gallego, E Coiera (2012) Industry influenced evidence production in collaborative research communities: A network analysis, Journal of Clinical Epidemiology, 65(5): 535-543. doi:10.1016/j.jclinepi.2011.10.010.