modeling choice with limited data - celect at strata + hadoop world 2015

Bringing Science to the Art of Retail SM

Modeling Choice with Limited DataVivek Farias, CTO @ CelectRobert N. Noyce Professor @ MIT

©2015 Celect, Inc. All Rights Reserved.2

An Important Question


An Important Question

Comparisons are eas(ier) …

… but also, comparisons are everywhere.


Comparisons Are Everywhere

Traditional Reviews: Peter prefers Tamarind to Cafe Boulud

Peter G.

Peter G.’s Profile

About Peter G.

Peter G.

©2015 Celect, Inc. All Rights Reserved.


5

Click Stream Data



6

Click Stream Data: Vinay is more in the mood for The Expendables 3 vs. Cosmos or The Bank Job or ... ?



(Brick and Mortar) Retail Data



8

(Brick and Mortar) Retail Data: I prefer Neutrogena to Dove, and Aveeno, and …


Two Applications• Helping a department store build “hyper-local”

experiences‒ Fully data driven approach to modeling choice

• Helping a top 10 US Retail website personalize to intent‒ A new view of what personalization online means

9

“Harvesting and Learning From Comparisons”


A Mental Picture Of A Choice Model

10

Representative Customer


A Generic Choice Model

11

30% 30% 7% 7% 20% 6%



12

= 30+30 = 60%

30% 30% 7% 7% 20% 6%



13

30% 30% 7% 7% 20% 6%



14

= 30+30+20 = 80%

30% 30% 7% 7% 20% 6%



15

$200

$300

$100

30% 30% 7% 7% 20% 6%



16

$100 $100$200 $200 $300 $30030% 30% 7% 7% 20% 6%



17

30% 30% 7% 7% 20% 6%$100 $100$200 $200 $300 $300 $200 $300



18

$200 $300$200 $200 $300 $30030% 30% 7% 7% 20% 6%


Revenue Estimation

19

?

Challenge: How to deal with sparse data & a high dim model? Traditional Approach: Assume comes from a parametric family

Generic Model Limited Data

Predicted Rev Rate


Revenue Estimation

20

Challenge: How to deal with sparse data & a high dim model? Our Approach: Assume that comes from a sparse model

Generic Model Limited Data

?Predicted Rev Rate


Our Approach

21

“Simple” “Conservative”

What do we know?


Case Study: Choice Modeling + Assortments• Mid-Size Retailer (~$4B in revenue. Close to 300 stores)• Task

‒ Optimize assortment using our approach to learn choice patterns at individual stores

• Data‒ In-store inventory + transactions over time‒ Online browse/ transactions

• Controlled experiment at 10 representative stores (Q3 14)

22

23

Case Study: Choice Modeling + Assortments


Test Stores

Control Stores

08/03 - 09/27

06/15 - 07/26

Test Period

24



Test Stores

Control Stores

08/03 - 09/27

06/15 - 07/26Trailing 6

Weeks

25



5.71% -2.91%

-.54% -2.51%

Test Stores

Control Stores

08/03 - 09/27

06/15 - 07/26

Test Stores Net Relative5.71 - -0.54 = 6.25%

26



5.71% -2.91%

-.54% -2.51%

Test Stores

Control Stores

08/03 - 09/27

06/15 - 07/26

Control Stores Net Relative

-2.91 - -2.51 = -0.40%

27


5.71% -2.91%

-.54% -2.51%

Test Stores

Control Stores

08/03 - 09/27

06/15 - 07/26

Adjusted Net Relative6.25 - -0.40 = +6.65%



Case Study Summary• Celect modifiers were applied to a limited portion of each

test store• Adjusting against control stores and for trailing 6-wk

trends:‒ Celect modified departments yielded an incremental 6.65%

increase in revenue‒ We predicted 6.43%, so our performance is predictable

• Test stores in total yielded an incremental 2.98% increase in revenue potentially due to positive spillover effects of a better assorted store


Personalization

29

“If you liked this then you will also like these other things”


Personalization

30


• What does it mean to personalize online?• Explosion of ad-hoc heuristics -- tailor made to application• All have a ‘nearest neighbors’ flavor• How does choice play a role?


Personalization

31


• What does it mean to personalize online?• Explosion of ad-hoc heuristics -- tailor made to

application• All have a ‘nearest neighbors’ flavor• Our Approach: Build a choice model, and optimize

assuming choice


Implementation at a Gigantic Retailer• Retailer (~$36B in revenue. ~ 10 Monthly Uniques)• Task

‒ Personalized offerings for search and checkout‒ < 5 ms RTT tolerance

• Data‒ Online browse/ transactions‒ CRM

• Live on traffic

32


Implementation at a Gigantic RetailerCategory Incumbents Celect Increase

Appliances $4.38 $5.57 27.17%

Clothing $0.64 $0.77 20.31%Lawn & Garden $3.26 $4.50 38.04%

Tools $1.36 $1.56 14.71%

Toys $0.41 $0.67 63.41%

Outdoor Living $4.49 $5.03 12.03%

Total $2.86 $3.43 19.93%

20% increase in monetizing site

traffic.

Distinct incumbents in each category


In Summary• Understanding choice is valuable• We can model choice by learning from atomic comparisons

‒ Comparisons are everywhere• Data-Driven Choice modeling can re-shape offline retail

functions‒ Hyper-Localized Buying, Planning and Allocation

• Data-Driven Choice modeling can re-shape personalization‒ Understanding choice beyond nearest-neighbor algorithms

34

35

A Top Strategic Initiative"Finally, our localization initiative is important to our growth plan as we look to offer increasingly local-centric assortments. We are testing new software called Celect, a predictive modeling tool that allows us to leverage customers' omnichannel purchase and browse behavior to identify merchandise localization opportunities. We believe that localization can allow us to further increase market share in our low-volume doors.”

Kathy Bufano, CEO, Bon-Ton StoresBONT Earnings Call, May 21, 2015



Thank you!

Live Webcast – October 15th at 1pm ETwww.celect.com/screaming-data-webcast

Visit blog.celect.com

Follow us on Twitter! @celect

http://www.celect.com/screaming-data-webcast

http://blog.celect.com/

https://www.twitter.com/celect

https://www.twitter.com/celect

modeling choice with limited data - celect at strata + hadoop world 2015

Technology