Stay Our of My Forum!Evaluating Firm Involvement

in Online Rating Communities Neveen Awad

Wayne State University, Detroit, MI

Hila EtzionUniversity of Michigan, Ann Arbor, MI

2006 Awad Etzion

Growth of online word of mouth• A growing number of consumers are

contributing content to online product review forums, discussion boards, weblogs (blogs), video sharing communities, etc.

• An even larger number of consumers reference online word of mouth

(Source: Pew Internet 2006)

2006 Awad Etzion

Amazon.com’s bet• Amazon.com eliminated its entire budget

for television and general-purpose print advertising

“ Word of mouth is important because on the Web you can reach so many more people beyond your circle of friends”

Bill Curry, Amazon.com Spokesperson

2006 Awad Etzion

Example

2006 Awad Etzion

Comparison of Reviews Example

2006 Awad Etzion

Should firms be involved with online word of mouth?“ Some retailers are struggling with how they

should handle a flood of submissions, and in particular, negative reviews that could make it difficult to sell a product.

Many sites simply use automated filters to check reviews for profanity and then publish a majority of them. Others like Newegg, have employees closely vet each submission and reject a greater percentage of reviews.”

(WSJ, 2005).

2006 Awad Etzion

Lots of questions… few concrete answers

• Are online reviews and ratings related to online sales?

• What are the consequences of firm intervention activities on the information value of online forums?

• How do consumers assess information provided through online reviews?

2006 Awad Etzion

Literature Review: Effect of Online Word of Mouth on Sales

• Senecal and Nantel (2004) Chevalier and Mayzlin (2006)Online product reviews influence consumer purchase decisions

• Godes and Mayzlin (2004) Dispersion of conversations among different Usenet groups is

significantly related to Nielsen (viewership) ratings of TV shows, but volume is not

• Liu (2006), Duan et al. (2005) The volume of Yahoo! Movies discussions has a significant impact on motion picture box office revenues, but not the valence

• Dellarocas, Awad, and Zhang, (2005)Early volume of online movie reviews is a proxy of early sales; valence is a significant predictor of word of mouth and rate of decay of external publicity

2006 Awad Etzion

Literature Review: Online Reviews – Bias ?• Li and Hitt (2004)

Books ratings decline over time, showing a positive bias in reviews written by early buyers.

• Dellarocas (2003) Under certain conditions manipulation of online ratings

can increase the informativeness of the forum

• Chen and Xie (2004) It is not always beneficial for the seller to support a review system

• Mayzlin, (2006)If third-party signals are sufficiently noisy, consumers listen to promotional chat strategically posted by firms

2006 Awad Etzion

Research Questions• How online reviews effect the shape of the

demand functions for imperfect substitutes?

• Do consumers reference different metrics of online word of mouth depending on the nature and number of reviews?

• Does firm filtering of the online reviews effect the impact of these reviews on online transactions?

• Should online retailers filter bad reviews?

2006 Awad Etzion

The Model• Retailer selling 2 imperfect substitutes.

111222 )()( DcpDcp

211211 ),( dpbpappD 122212 ),( dpbpappD

• Retailer selling 2 imperfect substitutes and online review system

Ti the total number of reviews for product i

pi price of product i to consumers

ci cost of product i (for the retailer)

Ri review information for product i

the seller’s profit

)()(),( 121

22

21

212212 MM EM

TTTEM

TTTdpbpaRRD

)()(),( 221

21

21

121211 MM EM

TTTEM

TTTdpbpaRRD

wi = (Ti)/(Ti+Tj) EM the a-priori expected value

of the summary statistic M.

Assumption:

2006 Awad Etzion

)()(),( 112212212 MM EMwEMwdpbpaRRD

)()(),( 221121211 MM EMwEMwdpbpaRRD

The Model, Ratings {-1, 0, 1}Retailer selling 2 imperfect substitutes and online review system

M Summary Statistic Value Value when B =0

EM EM f

S Average rating (G-B)/(G+B+N) G/(G+N) 0 1/2

Pg Fraction of good ratings G/(G+B+N) G/(G+N) 1/3 1/2

Pb Fraction of baa ratings B/(G+B+N) 0 1/3 1/2

Assumption: consumers expect each rating to be submitted with the same probability

Gi – Number of good reviews (1) for product iNi– number of neutral reviews (0) for product iB i– number of Bad reviews (-1) for product i

2006 Awad Etzion

Should the seller filter?

),()0,0( 21 BBMMM

),()0,0( 21, BBM

fM

NffM

p1=100 , p2=90, =5 , =2.5,

G1=15 , G2=20 , B1=3 , N1=5 , N2=3,

a=1000 , b=1 , d=0.2 , c1=c2=80

2 4 6 8 10 12 14B2

-10

-5

5

10

Avg

Avg w. Bias

B2

Fraction of G’s

Fraction of G’s w. bias

2006 Awad Etzion

2 4 6 8 10 12 14B2

-2

2

4

6

Should the seller filter?

),()0,0( 21 BBMMM

),()0,0( 21, BBM

fM

NffM p1=90 , p2=100, =3, =3,

Avg

Avg w. Bias

B2

Fraction of G’s

G1=15 , G2=20 , B1=3 , N1=5 , N2=3,

a=1000 , b=1 , d=0.2 , c1=c2=80

Fraction of G’s w. Bias

2006 Awad Etzion

How Consumers choose M?)()( 2211},{ MMMSPgM EMhEMhCMax )()( 2211},{

fM

fMMSPgM EMhEMhCMax

2 4 6 8 10 12 14B1

0.05

0.1

0.15

C

2 4 6 8 10 12 14B1

0.05

0.1

0.15

0.2

0.25

C

G1=30, G2=20, B2=0,N1=1, N2=1

2 4 6 8 10 12 14B1

0.5

0.6

0.7

0.8

0.9

C

2 4 6 8 10 12 14B1

0.1

0.2

0.3

0.4

C

G1=50, G2=1, B2=0, N1=0, N2=1. (RHS – suspect filtering, LHS- don’t)

If CM is increasing in B2, then CM favors product 1. If CM is decreasing in B2 – then clearly it prefers product 2

Pg favors 2 S favors 1M=S Pg &S

favor 1M=S

Pg &S favor 2M=Pg

Red= average

Blue= Percentage of good

2006 Awad Etzion

Hypothesis 1: Biased Filtering

• When consumers become aware of biased reviews, their usage of online review information will change (White, 1999; Mayzlin 2006).

• Hypothesis 1: When firms implement an intervention strategy aimed at filtering out negative reviews, percentage of positive reviews will be significant and positively associated with online transactional amount.

2006 Awad Etzion

Hypothesis 2: Noise Reduction• Information overload is one of the biggest

issues online (Berghel, 1997)• Firm manipulation of online forums can either

increase or decrease the informational value of the forum (Dellarocas, 2004).

• Hypothesis 2: When firms implement an intervention strategy aimed at “noise reduction”, valence will be significantly associated with online sales.

2006 Awad Etzion

Data• Collected from a large online retailer

• Dates range from April 16th, 1999 to February 2nd, 2006

• The firm changed its reviews filtering method

• The user reviews data consisted of an optional text review of product together with an integer numerical rating that ranged from 5 (best) to 1 (worst). ~ All of the reviews are first market as pending ~ As the team goes through the reviews, they either

approve the reviews or reject them.

April 16th, 99 March 3rd, 05 Feb 2nd, 06

2006 Awad Etzion

Biased Filtering

4.66 0.04 1.838 0.05 2.70% 1.428 77.70% 1.101

Noise Reduction 4.57 0.055 2.641 0.101 3.85% 1.942 73.56% 1.078

% Change -2% 38% 44% 102% 42% 36% -5.33% -2%

Num

ber o

f 1s

Num

ber o

f 5s Pe

rcen

tage

of

5s

Sale

s

Perc

enta

ge

of 1

s

Rat

ing

Varia

nce

of

Rat

ing

Num

ber o

f R

evie

ws

Summary Statistics

2006 Awad Etzion

Data Summary  Biased Filtering Noise Reduction

Number of Products 631,316 556,880 Number of Subcategories  2,590 2,590 Number of Departments  108 144 

 Number of products w/ reviews 48,003 92,828

2006 Awad Etzion

Transactions Per Month

0

500000

1000000

1500000

2000000

2500000

3000000

1999

/04

1999

/08

1999

/12

2000

/04

2000

/08

2000

/12

2001

/04

2001

/08

2001

/12

2002

/04

2002

/08

2002

/12

2003

/04

2003

/08

2003

/12

2004

/04

2004

/08

2004

/12

2005

/04

2005

/08

2005

/12

2006 Awad Etzion

Model Dependent variable: Log (Online Sales)

ln _

ln ( ( ))

1 2 3_ _1 _ _ 554 6

ln7 8 9( _ _ )10 ( ( ))

Volume Valence Category Valence

Variance percentage

ReviewLength ProductPrice

salesi j k

rev percentage rev

Density

price category dummy i j k

2006 Awad Etzion

Column 1 Column 2 Column 3Before After Before After Before After(b) (b) (b) (b) (b) (b)pr > t pr > t pr > t pr > t pr > t pr > t

-0.0191*** -0.0116** -0.0203*** -0.0130*** -0.0204** -0.0135***0.002 0.034 0.001 0 0.001 0

0.00328 0.0115*** 0.0447 0.0511**0.113 0.001 0.209 0.009-0.2222*** 0.036** -0.2553*** 0.0120*0 0.033 0 0.01-0.1908*** -0.0353** -0.2092*** -0.0256*0 0.039 0 0.036

-0.0191* -0.00624*

0.093 0.089

0.1184 -0.00670.209 0.8470.0376* 0.00830.055 0.319

No of Obs 4457 7887 4457 7887 4457 7887adj Rsq 0.0039 0.0021 0.0103 0.0486 0.0127 0.0504

ln(Product_Price)

Valence

ln(Volume)

No Reviews DummyImperfect_ Substitutes_ValencePercentage Review_n_1Percentage Review_n_5

2006 Awad Etzion

Results: Hypothesis 1: Bias Filtering

Hypothesis 1: When firms implement an intervention strategy aimed at filtering out all negative reviews, positive reviews will be significant and positively associated with online transactional amount.

Supported

2006 Awad Etzion

Results: Hypothesis 2: Noise Reduction

Hypothesis 2: When firms implement an intervention strategy aimed at “noise reduction”, the review valence will be significant and positively associated with online transactional amount.

Supported

2006 Awad Etzion

Conclusion• Before the change the percentage of ‘1’

ratings were not significantly associated with number of purchases, but the percentage of ‘5’ were

• After the filtering strategy change; the average valence does significantly affect the number of purchases per product.

• Firms should filter reviews in certain situations

2006 Awad Etzion

Questions

2006 Awad Etzion

Online Word of Mouth- Bias?Books ratings decline over time, showing a positive bias in reviews written by early buyers.

Li and Hitt (2004)

Under certain conditions manipulation of online forums can increase the informativeness of forums

Dellarocas,(2004)

Impact of online reviews on online retailer depends on product, information, and consumers.

Chen and Xie (2004)

If third-party signals are sufficiently noisy, consumers listen to promotional chat strategically posted by firms

Mayzlin, (2006)

2006 Awad Etzion

Effect of Online Word of Mouth on SalesOnline product reviews influence consumer purchase decisions

Senecal and Nantel (2004) Chevalier and Mayzlin, (2006)

Dispersion of conversations among different Usenet groups is significantly related to Nielsen (viewership) ratings of TV shows, but volume is not

Godes and Mayzlin (2004)

The volume of Yahoo! Movies discussions has a significant impact on motion picture box office revenues, but not the valence

Liu (2006), Duan et al. (2005)

Early volume of online movie reviews is a proxy of early sales; valence is a significant predictor of word of mouth and rate of decay of external publicity

Dellarocas, Awad, and Zhang, (2005)

2006 Awad Etzion

Should the seller filter if consumer does not expect biasWhen M=average rating ),()0,0( 21 BBMMM

If products have same margins,

1. When (p1-c1)=(p2-c2)

then M 0 with equality when =.Reviews only transfer demand from one product to another• When =:

If B1(2G1+N1+N2) > B2(2G2+N1+N2) then M 0 iff (p2-c2) ≤ (p1-c1).

If B1(2G1+N1+N2) < B2(2G2+N1+N2) then M 0 iff (p2-c2) >( p1-c1).

))()(())()(())()(())()((

221112112221

221121112212

cpkcpGBcpLcpGBcpLcpGBcpkcpGB

Where L= 2G1+G2+N1+N2 and k=2G2+G1+N1+N2

2006 Awad Etzion

Data• Collected from a large online retailer• Dates range from April 16th, 1999 to February 2nd, 2006

• The firm changed its reviews filtering method on March 3rd 2005~ First period: April 16th, 1999 through March 3rd 2005~ Second period: March 4th of 2005 through February

2nd, 2006.

• The user reviews data consisted of an optional text review of product together with an integer numerical rating that ranged from 5 (best) to 1 (worst). ~ All of the reviews are first market as pending ~ As the team goes through the reviews, they either

approve the reviews or reject them.