using online webmd reviews to examine drug performance

Can social media tell us which drug is better?

Assessment of Web-Based Consumer Reviews as a Resource for Drug PerformanceJ Med Internet Res 2015;17(8):e211

Can we use online reviews?

Easy to understand & access

Large numbers– 6% have shared drug

experiences online– Millions of health experiences

online

Limited Access to Drug Comparisons

Publications1. Funded by drug companies2. Shown to be biased3. Hard to critique based on the way it’s written

FDA Drug Trials4. Studies are funded by drug

companies themselves5. Biased self-reporting

An online WebMD Drug Review

Compare drugs that treat the same condition– Controls for the heterogeneous populations– Assume similar populations being treated

Agreeing with the LiteratureCondition: Joint damage causing pain

Nabumetone 2.6

Meloxicam 3.1

Rofecoxib

4.6

“The overall 6-month effectiveness rates <…>59.8% for nabumetone, 67.6% for meloxicam.”Clin Drug Investig. 2004;24(2):89-101

“Median time to a good or excellent PGART was significantly shorter with rofecoxib (52 hours) than nabumetone (100 hours, P = 0.001)”

J Clin Rheumatol. 2006 Feb;12(1):17-25

427 drug pairs with significant and substantially different ratingsFirst Drug

(online rating)Second Drug (online rating)

Deduced online trend

Amlodipine (2.5) Felodipine (3.2) Amlodipine < Felodipine Amlodipine (2.5) Telmisartan (3.1) Amlodipine < Telmisartan

Search scientific literature

Search terms: Condition, Drug A vs. Drug Be.g. Hypertension, Amlodipine vs. Felodipine

See whether literature agrees with online trend

Online ratings supported 62-68% when rely on literature evidence

0.5 0.750000000000002 10

102030405060708090

100

62 (48/77)

64 (28/44)

68 (15/22)

* 71.4 (55/77)* 75 (33/44)* 81.8 (18/22)

Scientific LiteratureScientific Literature+FDA Labels

Difference between drug ratings

% v

alid

ated

with

scie

ntific

su

ppor

t

Where is there more support for online ratings?

FDA label carries warnings

FDA Warnings

Amiodarone had the LOWEST online ratings. Many complain about side effects.

Amiodarone#Reviews/total

#reviewsComplaints

13.9% (23/166) “kill, died, death, dead”

But the FDA does not recommend amiodarone to treat atrial fibrillation.

Scientific Publication says “Amiodarone is effective compared to flecainide” Arch Intern Med. 1995 Sep 25;155(17):1885-91

Online data supported 71-82% when FDA labels added as support

0.5 0.750000000000002 10

102030405060708090

100

*62 (48/77)

*64 (28/44)

68 (15/22)

* 71 (55/77)

* 75 (33/44)

*82 (18/22)

Scientific LiteratureScientific Literature+FDA Labels

Difference between drug ratings

% v

alid

ated

with

scie

ntific

su

ppor

t

*significant

Validating Results

• 71% of online comparisons agree with literature and FDA labels

• 82% for comparisons with at least 1 point rating difference

Reviewer biases and insights

Reviewers give higher ratings to:

– Addictive drugs– Alternative treatments

Addictive Drugs Rate Higher

Condition: Muscle spasmCarisoprodol and cyclobenzaprine not significantly different

Clin Ther. 2004 Sep;26(9):1355-67.Drug Rating % reviews with the word

“addict”Carisoprodol 4.35 9.2

cyclobenzaprine 3.33 0

It's a great medication, but can easily become dangerously addicting.

so far this has been the best medication to help give me almost complete relief. Just be careful using it, it is addictive.

Alternative Choice of Treatment(the road less travelled)

Second-line treatments rate higher.

Is it truly a better drug?

Or a preselected population who are happy to have found this drug because standard therapy did not work?

Example: Underactive Thyroid

Thyroxine (synthetic thyroid hormone)“official” approved drugOnline rating 2.22

Natural thyroid extractOnline rating 3.92

When the Generic is Better than the Brand-Name

• Pro-Air brand rated 2 stars lower than generic• “Green”, environmentally-friendly design, cost more $

Online reviewers complained about “inhaler”Generic had a counter with number of doses remaining, Pro-Air did not

Pro-Air later released new design

Conclusion

• Online drug reviews are useful, with caveatsReviewers give higher ratings to– Addictive drugs– Alternative treatments

• Gain new insights like bad inhaler design