recovering sku-level preferences and price sensitivities ... · 1imagine that tide currently has...
TRANSCRIPT
Recovering SKU-level Preferences and Price
Sensitivities from Market Share Models Estimated on Item Aggregates
David R. Bell, Andre Bonfrer and Pradeep K. Chintagunta1
August 7, 2003
1The Wharton School, University of Pennsylvania, 3730 Walnut Street, Philadelphia PA 19104 (email:
[email protected]); Singapore Management University, 469 Bukit Timah Road, Singapore 259756
(email: [email protected]); and Graduate School of Business, University of Chicago, 1101 E 58th
Street, Chicago IL 60637 (email: [email protected]), respectively. The authors would
like to thank the Wharton-SMU Research Center for generous support, and seminar participants at the
Marketing Science Conference for helpful comments and suggestions.
Recovering SKU-level Preferences and Price
Sensitivities from Market Share Models Estimated on Item Aggregates
Abstract
Market share models such as the logit and MCI have been used extensively to studythe effects of marketing activities on aggregate shares of brands. Many product cate-gories contain a large number of stock keeping units (SKUs) and consist of assortmentsthat change over time (due to the prevalence of introductions, deletions and stock-outs).To aid in decisions such as product deletions (for example), marketers are interested inmeasuring preferences and sensitivities to marketing activities at the SKU level. Directestimation of these parameters may be hampered by the large number of coefficientsto be estimated and by the possibility that few observations are available for certainSKUs. Fortunately, even though most product categories contain a large and variablenumber of individual alternatives, they can also be defined by a much smaller stableset of attributes (e.g., brand, size, flavor, function, etc.). Instead of estimating thepreference for each SKU, researchers have estimated the preferences of attributes thatcomprise the SKU and then computed the implied preference for the SKU.In this paper, we propose an alternative approach to obtaining SKU-level prefer-
ences. We distinguish an attribute-level model in which the unit of analysis is themarket share for an alternative created by aggregation (e.g., “Colgate toothpaste”)from a true SKU-level model and develop an analytical relationship between para-meters obtained from these two models. We show that the researcher can recoverSKU-level parameters via calculation from estimated attribute-level parameters, cir-cumventing the need for direct estimation of the more complex true SKU-level model.So in a category with 168 SKUs involving 10 distinct brands, we show that instead ofestimating 168 preference parameters (when we have an “outside” alternative in addi-tion to the 168 “inside” ones), we need to estimate only 10 preference parameters fromwhich we can compute the 168 parameters as long as share and marketing mix data areavailable at the SKU level. Our market share models are calibrated using ninety-eightweeks of data for 168 SKUs in the toothpaste category. Estimation on holdout samplesdemonstrates superior predictive performance compared with other available methods.Implications for the derivation of SKU-level elasticities and forecasts of market shareand price sensitivity for new items introduced to the category are discussed.
Key Words: Discrete Choice, Market Share Models, SKU
1
1 Introduction
A typical retail assortment for a given product category consists of several hundred items
(often referred to as Stock Keeping Units (SKUs) or Universal Product Codes (UPCs)). The
presence of so many items poses considerable challenges to modelers of market response and
to retail managers trying to optimize the product offerings. While it may be desirable or
important to study behaviors across the full set of alternatives, this would likely result in a
model that is non-parsimonious and computationally intensive. If the goal of the researcher is
to uncover true SKU-level parameters, then two options are available. The first is to estimate
a model with a complete set of fixed effects, one for each item. This option is difficult
to implement with aggregated data of the type usually seen in marketing studies, since
the modeler quickly loses degrees of freedom with this approach. Furthermore, additional
problems may be encountered when there exists considerable volatility in the underlying
choice set due to entry and exit of SKUs in the category (Bucklin and Gupta, 1999). A
less serious problem is that the computational requirements for such a non-parsimonious
model can be quite significant, especially if one is constructing multiple models over many
categories, or running simulations that require multiple estimation of these models.
The second option to estimating SKU-level market share response models is to use a
pure characteristics based approach of the type introduced to the panel data literature by
Fader and Hardie (1996) (hereafter FH). This method avoids the ad hoc aggregation of SKUs
into choice composites – an approach still prevalent in individual-level choice models that
populate the marketing literature. FH propose consumer utility for a particular item should
be represented as an additive combination of utilities for the attribute levels that define the
alternative, rather than as a single fixed effect. An important property of this model is that
the number of parameters required does not increase with the number of alternatives, but
rather only with the number of additional levels of underlying attributes. This procedure not
only produces superior model fits in sample, but also allows researchers to create forecasts
for new items (provided the individual elements that make up the new alternative are present
in the market).1
While this second approach has been implemented on individual panel data, Bucklin and
1Imagine that Tide currently has 16oz and 32oz sizes, but no 64oz offering. Provided some competitor
offers a 64oz size, it is possible to develop a forecast for the new product “Tide, 64oz”.
2
Gupta (1999) note that managers have a preference for market share (or “aggregate”) data
which is more readily available and thought to be more reliable (less subject to sampling
biases). Moreover, academic researchers in marketing are showing increasing levels of interest
in market share models of the type advanced in the economics literature (e.g., Nevo 2001).
The model by FH represents a pioneering effort at modeling an entire product assortment
based on identifiable (to the researcher) product characteristics. Ho and Chong (2003) extend
this idea and develop a model where consumers “reinforce” chosen and non-chosen options in
their patterns of item selection in a manner consistent with an experience-weighted attraction
theory of consumer learning (Camerer and Ho 1999). Both models have a good deal to offer
market share-based approaches, yet these ideas have not diffused into that literature. In
this paper, we seek to bridge that gap and develop a model that is grounded in theory
and in the data actually used by retailers. We go beyond these previous empirical studies
and provide formal analytic relationships between the parameters estimated from simple
“composite” demand models and the true SKU-level fixed effects which can be derived from
them, circumventing the need for direct estimation.
To gain an appreciation of the substantive nature of the task at hand, consider the
following data from the toothpaste category (used in our empirical application). Table 1
(Panel A) shows that there are 168 unique items in the category. Furthermeore, over a period
of almost two years of observation many items are added, dropped, or stocked out. Table 1
(Panel B) indicates that each unique item can be further described as a combination of levels
over the following five attributes: brand, flavor, form, function and size. A “full assortment”
reflecting SKUs across all possible combinations would yield 10 × 14 × 3 × 7 × 3 = 11, 760items.
––––––––––––––
[ Table 1 about here ]
––––––––––––––
How then, is the market structured according to the product characteristics? Table 1
generates several interesting empirical observations, some of which (to our knowledge) appear
to have been overlooked in the estimation of market share models, yet seem highly germane
to model building2
2While we focus on toothpaste for our empirical analysis, the general empirical patterns shown in Table
1 can also be seen in many other categories (e.g., cereal, soft drinks, yogurt, etc.).
3
• The choice set faced by consumers changes dramatically over time (Panel A).• There are many more possible product locations (11,760) than locations actually oc-cupied by products (168) (Panel B).
• There is heterogeneity across attributes in the extent to which the owners of brands“fill in” the product line. Sixty-five percent of brand-size combinations are occupiedsuggesting that manufacturers decide on average to offer most sizes, whereas onlytwenty-one percent of brand-flavor combinations are occupied, implying that SKUsare more likely to be “unique” on the dimension of flavor (Panel B).
The implications are immediate. A standard market share model estimated on the true
alternatives dictates a very large number of fixed effects and estimation on the correct unit
of analysis (the SKU) could be complicated by time-dependent variation in the composition
of the choice set. Moreover, the location of items in the attribute space is clearly non-
random, which implies there is something to be learned from studying consumer sensitivity
to changes in attribute levels.
––––––––––––––
[ Table 2 about here ]
––––––––––––––
Table 2 reports for each attribute a Herfindahl index obtained from attribute-level shares
and compares it to a “competitive” value of 1/n where n is the number of levels. Consistent
with the data in Table 1, flavor and function are attributes with greater degrees of concentra-
tion (suggesting these attributes offer more unique value for the brands) whereas form and
size have index values close to that for the benchmark competitive level (suggesting these
are parity attributes).
Overview and Contribution. We derive a market share model that takes the fundamen-
tal properties of the data shown in Tables 1 and 2 into account, and provide the following
contributions to the extant literature on development and estimation of market share mod-
els. First, we show how one can recover underlying SKU-level fixed effects from a much
more parsimonious share regression on attribute-level shares.3 For example, assume that
the researcher is provided with the fixed effects arising from a market share model specified
3The attribute-level model is generated by aggregation in which the unit of analysis is the market share
of a level of a particular attribute (e.g., brand, size, etc.). In a brand-level model, for example, one defines
4
on brand-level data. These fixed effects represent the mean utilities of, for example, the
Colgate, Crest, Arm & Hammer brands. We show how these parameter estimates can be
used to generate fixed effect coefficients for each of the individual items under the respective
brands, such as mint-flavored, gel-form Colgate 6OZ toothpaste with tartar control. This is
possible for any number of items (SKUs) and works for any attribute.
Second, marketing mix coefficients for an underlying item-level model can be recovered
from a second-stage regression of transformed shares on appropriately mean-centered prices.
Third, the attribute-level coefficients can be used to relate item-level response measures
(e.g., elasticities) to higher order attribute-level response measures.4 An important practical
benefit is that the retailer gains insight into which attributes show the greatest sensitivity
and therefore suggests ways in which the assortment could be changed. We are also able to
provide forecasts for new items that may be introduced to the assortment and outperform
current approaches on predictive performance. More significantly, we can also forecast the
price sensitivity of these items, based on their product characteristics. Thus retailers and
manufacturers can adopt this procedure to improve decisions in category assortment planning
and new product design.
The remainder of the paper is organized as follows. The next section briefly reviews
the relevant market share modeling literature and develops the specification and demand
equations for our approach. In particular, we explicate the analytical relationship between
an SKU-level model and an attribute-level model. Next, we provide a description of the data
used for the empirical application and further motivate our study. The empirical results are
presented in a separate section and the paper concludes with the discussion of the empirical
findings which includes a comparison with an alternative “pure characteristics” modeling
procedure.
market share for the Colgate brand by adding up the shares of all items that have this brand name (and
similarly for Crest, etc.). In a size-level model one computes share for all items under the 4OZ size, 6OZ
size, etc.4This provides some determination of the extent to which combinations of attributes produce interactive
effects. For example, the “baking soda” flavor applied to the brand “Arm and Hammer” may deliver market
share beyond that predicted by an additive model.
5
2 Background and Model
Increasingly, researchers in marketing are turning their attention to the analysis of market
share data obtained from retail settings. First — as noted earlier — retail managers are more
comfortable and familiar with market share data than with the panel level data often favored
by marketing academics (Bucklin and Gupta 1999). Store level data is also less subject
to sampling biases and especially useful for understanding “micro-marketing” and retail
tailoring (e.g., Montgomery 1997). Moreover, market share modeling has a long tradition
dating back to early work by Cooper and Nakanishi (1974; 1982) on the MCI model and
forecasting models of Brodie and deKluyver (1982).5
In many instances it may be reasonable to aggregate data to the level of brand (man-
ufacturer) and hold the set of brands constant over other dimensions of the dataset (e.g.,
time and or/markets), however if the goal is to better understand interrelationships among
items (e.g., for a product assortment decision) it is necessary to take a more disaggregate
approach. It is important not to be constrained methodologically by techniques that require
ex ante aggregation or constancy of this (aggregated) choice set over other dimensions. The
question of how items can be derived from attributes has been the subject of successful study
in the panel data literature (see FH 1996; Ho and Chong 2003) and this work provides a
starting point for our methods. FH stress that consumers choose items, not ad hoc brand-
level composites defined by researchers, and that retailers and manufacturers also require
understanding of item-level behavior. This is especially true in complex categories where
items are markedly different in their observed characteristics (e.g., flavor, form, function,
size, etc.). In the toothpaste category on which our model is calibrated, we observe consid-
erable variation in items on the shelf at any point in time, but no variation over time in the
number of brands which remains constant at ten.6 Figure 1 shows the time variation in the
5For a more detailed review see Leeflang, Wittink, Wedel and Naert (2000, p. 171-178).6Although variation in the number of levels may occur, it is relatively rare compared with the variations
in the items available in a choice set. One instance where this occurs is with the introduction of very new
products, involving a new level of a given attribute. For example the recent introduction of “Vanilla Coke”,
where the vanilla flavor does not exist in any other soda beverages.
6
number of items (previously summarized in Table 1).
––––––––––––––
[ Figure 1 about here ]
––––––––––––––
2.1 Model
We start with a basic market share model commonly found in the literature and describe the
relationship among fixed effect parameters estimated at different levels of aggregation. In
particular, we show how fixed effect parameter estimates for SKU j – the most disaggregate
unit of analysis – obtained from market share equations for all j = 1, . . . J items are related
to fixed effect estimates for attribute level l of attribute a estimated from market share
equations for the l = 1, . . . La levels of that attribute (where La is much smaller than J –
see Table 1). For ease of exposition we focus on the “brand” attribute as the higher level
composite.7 To continue this example, assume Brand A is composed of three unique items
of differing sizes (e.g. Small, Medium, Large), whereas Brand B is composed of only two
unique items of differing sizes (e.g. Small, Large). We first aim to show how the item-
level fixed effect for item j (e.g., β1 for j = 1) estimated from a five-equation system is
related to the brand-level fixed effect for Brand A (i.e., βA) estimated from a two-equation
system. We do this initially within the context of models that have fixed effects only (i.e.,
no covariates). For each model specification fixed effect parameters are identified through
use of data collected over some other dimension (e.g., time or market). In our empirical
application, observations for each item and composite are collected over time t = 1, . . . , T .
After first showing how to relate and recover fixed effects from models without covariates we
extend the analysis to the case of exogenous regressors. We again show how the fixed effects
and the true price (or marketing mix) coefficient that would arise from an item-level model
can be calculated without recourse to direct estimation, but by using parameter estimates
from the attribute-level model.
7The choice of attribute is immaterial. The researcher may choose any product attribute as the basis for
aggregation.
7
2.2 Fixed Effects for Items and Composites (Brands)
We begin by showing how the parameters from two models: (1) the disaggregate item-level
share model, and (2) the brand composite model are related when each model contains only
fixed effects. The goal is to show how item-level fixed effects may be recovered analytically
when one estimates only the attribute-level model. As noted earlier, we focus on brand as
the attribute over which aggregation takes place. This is for the purpose of exposition only
and we do so without loss of generality. Subsequently, we extend the analysis to the inclusion
of covariates, illustrating our approach using price as the variable of interest.
Fixed Effects: Attribute-Level Parameters from Item-Level Parameters
Following the standard approach in the literature we assume that the market share for item
j at time t can be expressed as
sjt =expµjt
1 + Jk expµkt
,(1)
where µjt is the item-specific utility relative to an outside good. In general, µjt can be a
function of both fixed effects for item j and time-dependent covariates and their associated
response parameters. Similarly, the share of the so-called outside good is given by
s0t =1
1 + Jk expµkt
.(2)
Parameter estimation is facilitated by a linearizing transformation (e.g., Besanko, Gupta
and Jain 1998)
log(sjt)− log(s0t) = µjt.(3)
Specifically, the parameters of the µjt component can then be estimated using OLS under
the assumption that the model also contains a structural error term jt that now takes on
the role of the error in the regression equation. Item-specific intercepts βj (excluding an
intercept for the outside good) can be estimated in a straightforward manner. Assume that
the market share for a particular brand b is simply the sum of the market shares for the first
8
two items. For brand b = 1 let
S1t = s1t + s2t(4)
=expµ1t + expµ2t1 + J
k expµkt.(5)
Clearly, the brand (attribute) share specification and the item share specification use the
same outside good value, which we can fix as
S0t = s0t =1
1 + Jk expµkt
.(6)
Applying the log transformation to the brand-level model of equation (5) we obtain the
brand-specific mean utility value for brand b (shared by items 1 and 2)
µbt = log(S1t)− log(S0t) = log(expµ1t + expµ2t).(7)
The key question of interest is: Can we recover the item-specific parameters from the brand
(attribute) share equations? Before addressing this however, let us first determine whether in
fact the reverse is possible: Can we recover the brand-specific parameters from the item-share
equations? The following calculation recovers the brand-specific intercepts for the attribute
share model using only the parameters and error terms from the item-specific model as inputs
(recall that in this example brand b is only available for items 1 and 2)
µbt = βb + bt = log(exp β1 exp 1t + exp β2 exp 2t)(8)
E[µbt] =1
T
T
t=1
βb + bt(9)
=1
T
T
t=1
log(expβ1 exp 1t + exp β2 exp 2t)(10)
= βb.
To confirm that this relation holds we obtain β1 and β2 and βb directly from OLS estimation
on the item-level and brand-level market share equations, respectively. We also compute
estimates of OLS residuals exp 1t and exp 2t from the item-level models. Using a simulated
dataset we computed these values for the right hand side of equation (10) and were able to
9
perfectly recover (to four decimal places) the brand-level intercepts βb estimated from the
brand share model, clearly demonstrating this link (details are given in Appendix A). Note
that this procedure explicitly requires both the parameter estimates βj and the residuals jt
to recover the brand-level parameters.
Fixed Effects: Item-Level Parameters from Attribute-Level Parameters
The next challenge is to recover item-specific intercepts via calculation, using only estimates
from the attribute-share model. For ease of exposition and without loss of generality we will
focus on the brand attribute and assume all Lb levels of this attribute are represented for
all periods of the data.8 Moreover, we note that the following information is available after
estimation of the attribute-share models
1. The estimated attribute-model intercepts for the brand levels, i.e., βb b = 1, . . . Lb.
2. The design matrix of the items (i.e., the description of each item in terms of a combi-nation of single unique levels for each attribute).
3. The error components of the data estimated from the attribute share models.
4. The outside good value.
5. The market share of each of the items, and the attribute share for each attribute level.
As we show below our method requires only 1 and 5 above in order to recover the item-level
fixed effects by calculation. Following our earlier analysis we have
expµbt = expµ1t + expµ2t.(11)
Let D = 1 + Jj=1 expµjt denote the denominator of the brand and item share models and
note that D can be rewritten as
D =expβb exp bt
Sbt.(12)
In general, exponentiating the mean utility for any product (whether the outside good, a
single item or a composite like a brand), and dividing it by the associated market share will
8This is much less stringent than requiring that all items are available in each and every period — something
that is clearly not true in practice (cf. Table 1 and Figure 1).
10
yield D. Next, using the definition of item share sjt given in equation (1) we recognize that
the product of the exponentiated (unobserved) item-level intercept and error component for
item j under brand b is equal to the product of D and the item-level share
exp βj exp jt = D · sjt.(13)
Taking the log of both sides we find that the item-level intercept is related to the brand-level
intercept as follows
βj + jt = βb + bt − log Sbt + log sjt.(14)
To obtain βj by calculation we take the average over t which yields
βj =1
T
T
t=1
[βb + bt − log Sbt + log sjt]
= βb +1
T
T
t=1
logsjtSbt.(15)
The last line of equation (15) is simply the sum of the estimated brand intercept and the
average log share of the item relative to the share of the brand. Because the denominator is
weakly greater than the numerator, the log sum will always be negative. Alternatively, we
could rewrite the expression as
βj = βb − 1
T
T
t=1
logSbtsjt.
Using simulated data the item intercepts calculated in this manner (see Appendix A) repro-
duce the directly estimated item intercepts almost exactly.9
2.3 Adding Covariate Effects
We now demonstrate how to recover covariate effects (e.g., price, feature and display). While
we focus on price for ease of exposition, our analysis is easily extended to any number of
9Calculated parameters are equal to estimated parameters to more than ten decimal places for all esti-
mates. We also tested the robustness of this relation given the presence of interactions among the attributes
(e.g. Colgate makes a better tartar control formulation than does Crest) and found that equation (15) is
still valid in this case.
11
covariates. The introduction of covariates complicates matters starting with recovery by
calculation of the item-level fixed effects βj, as the mean utilities for brand composite b and
item j now become
µbt = βb + γbPbt + bt and(16)
µjt = βj + γpjt + jt,(17)
where Pbt =1
Lb
Lb
i=1
sjtSbtpjt
is the item-share weighted price for the brand composite. First, we note how these changes
will affect the derivation that lead to equation (15) and the recovery of fixed-effects when
both the item-level and attribute-level models do not contain covariates. The parameters γ
and γb are the (true) item and brand-level price response parameters, respectively. That is,
γ is the parameter that would be obtained if one were to estimate an item-specific market
share model. We now ask the following questions. What is the relationship between βj and
βb when both models include price covariates? What is the relationship between γb and
γ? How do we recover the true γ from the estimate of the attribute-level counterpart γb?
Following the logic that lead to equation (12) we can still write, for item j
exp βj exp γpjt exp jt = D · sjt,(18)
however in this instance D is given by
D =exp βb exp γbPbt exp bt
Sbt.(19)
After taking logs and rearranging terms we find that
βj + γpjt + jt = βb + γbPbt + bt + log sjt − logSbt soβj = βb + γbPbt − γpjt − jt + log sjt − log Sbt.(20)
As before, the fixed effect values can be computed by taking averages over time
βj = βbestimated
+ γbestimated
¯Pbobserved
− γ
unknown
pj
observed
+1
Tlog sjt
observed
− 1Tlog Sbt
observed
,(21)
12
with the double bar indicating the average of the share-weighted price Pb taken over time
t = 1, . . . , T . We have been careful to distinguish what can be observed in the data, what can
be estimated and what quantities are unknown (and must be calculated). Unfortunately,
equation (21) clearly shows there is only one equation for each of the j items and two
unknowns (price effect and fixed effect).10 We will address this issue shortly, meanwhile we
note that the preceeding expression can be further refined as follows
βj = βb + γb¯Pb − γpj +
1
T
T
t=1
logsjtSbt.(22)
To understand how the identification problem caused by the introduction of regressors
complicates matters, consider the following example. Using simulated data we again esti-
mated the item-level and brand level-intercepts from the respective models, but this time
included the price variables. We found that calculating the item-level intercepts based on
the relationship in equation (15) derived earlier (for models that do not have covariates) and
ignoring the effect of price specified in equation (21) produces fixed effect estimates which are
consistently inaccurate. Figure 2 plots the estimated versus calculated item-level intercepts
resulting from this procedure.
––––––––––––––
[ Figure 2 about here ]
––––––––––––––
Clearly the difference between the points on the plot and the y = x line is due to the
inclusion of the price variable which renders the earlier derivation inappropriate. At the
same time equation (21) is not directly useful because the price and fixed effects cannot be
separately identified. Fortunately, we now demonstrate that both problems can be circum-
vented such that the true item-level fixed effects can be recovered from an appropriately
reformulated model. In addition, we show that a second-stage regression can be used to
recover the other parameter of interest – the unknown price effect, γ.
10It is also not possible to use information from other attribute equations (e.g., size) to identify the fixed
effect or price effect. This is because each attribute-level model yields equations of the form given in (21).
13
Fixed Effect Recovery: Model Reformulation With Mean-Centered Covariates
Continuing our discussion with price as the covariate, we reformulate the mean utilities of
the attribute and item-level models by including mean-centered prices rather than actual
prices to give
µbt = βb + γb(Pbt − ¯P b) + bt, and(23)
µjt = βj + γ(pjt − pj) + jt(24)
where βb and βj are defined here as the “true” fixed effects for each model. As indicated
above, mean-centering occurs at the appropriate level of aggregation – with respect to the
attribute-level average for each attribute level, and with respect to the item-level average for
the corresponding item. It is important to note that the covariate effect γb will be completely
unaffected by this change (e.g., Raudenbush and Bryk 2001). This can be seen explicitly by
observing that equation (23) can be rewritten as
µbt = βb − γb¯P b
β1b
+γbPbt + bt
where β1b is simply the fixed effect estimate that results when the covariate has not been
mean-centered. The difference between β1b and the “true” βb is simply equal to the price
estimate for the attribute multiplied by the average price for that level (and analogously for
the item-level model). It is still the case that: (1) the item-level and attribute-level price
effects will not be equal (i.e., γ = γb) and (2) the fixed effects βj will need to be calculated
from estimated values from the attribute-level models {βb, γb} and average log market sharesand prices { ¯P b, pj}. Using equation (18) but with the mean-centered covariates we obtain
βj + γ(pjt − pj) + jt = βb + γb(Pbt − ¯P b) + bt + log sjt − logSbt.
Taking expectations with respect to time we find 1T
Tt=1 γb(Pbt − ¯P b) = γb0 = 0 and
1T
Tt=1 γ(pjt − pj) = 0 which implies that item-level intercepts can be calculated from
E[βj] = βb +1
T
T
t=1
logsjtSbt.(25)
14
The analytical form of equation (25) is identical to that obtained for models that do not
include fixed effects, except that the covariates need to be mean-centered. We tested this
relation with simulated data (see Appendix A) and were able to recover the item-level in-
tercepts from the brand-level intercepts, just as we did for the models without covariates.
In addition, the true item-level fixed effects can be recovered using any attribute that the
researcher desires to use as a basis for aggregation, and with any number of appropriately
mean-centered regressors in the model.
Covariate Effect Recovery: Second Stage Regression Analysis
So far we have shown how true item-level fixed effects can be recovered using a parsimonious
attribute-level model. Recovery of the covariate effect – γ in our example – is still com-
plicated by the fact that while mean-centering solved the identification problem for the βj
parameters, it introduced an indeterminancy in process: γ × 0 = γb × 0. In fact, it was theintroduction of this indeterminancy in part, that made recovery of the βj possible. It is also
the case that knowledge of γb (nor any γa for any attribute a alone or in combination) tells
us nothing by itself about the true price effect, γ.
We can however obtain an estimate of the true price effect without recourse to an item-
level model, through the following second stage regression. First, having estimated the
attribute-level model we calculate the item-level fixed βj effects using equation (25). Second,
we subtract these calculated item-specific fixed effects from log-transformed share informa-
tion that enters the item-level regression model. That is, we use item-level data for each
item j and time period t and form the following regression equation
(yjt − βj) = γ(pjt − pj) + jt where
(yjt − βj) = log(sjt)− log(s0t)− βb +1
T
T
t=1
logsjtSbt
or
yjt = γpmcjt + jt,(26)
where pmc is the mean-centered price. OLS estimation retrieves the same γ as that obtained
from a regression model based on the original item-level market shares.11
11This approach has implications for the standard errors and these are given in Appendix B.
15
2.4 Summary
In the preceding sections we have developed the analytical and empirical relationships that
exist between market share models estimated at different levels of aggregation. In particular,
we show how one can recover via calculation both the underlying item-specific fixed effects
and the response parameters for covariates in an item-level model using a very parsimonious
model estimated at a higher level of aggregation. The implications for researchers and
practitioners are powerful: There is no need to resort to arbitrary aggregation schemes,
disregard data, or focus only on categories with a relatively small number of alternatives.
Our formulation allows the researcher to be true to the properties of the underlying data,
and implement models in a very straightforward fashion using standard software.
In the next sections we describe our empirical analysis of the toothpaste category and
also develop unique insights afforded by our model into how consumers value attributes. In
particular, we show that attribute-level equations are useful not only as a methodological
shortcut, but also as a tool for delivering substantive insights into consumer sensitivity to
attribute-level changes.
3 Data
We analyze toothpaste sales data from a single store in the Stanford Market Basket Data-
base.12 It is well known that retailers use some categories to drive store choice decisions
(Chen, Hess, Wilcox and Zhang 1999) and that toothpaste is one category typically not as-
sociated with traffic building objectives (Dreze and Hoch 1998). In addition, the toothpaste
category contains a good deal of variety and large changes in the total number of items
available due to stock outs, product introductions and deletions (see Table 1 and Figure
1). Collectively, these features make it an ideal candidate category for calibrating our new
12These data have been used in a number of studies published in the marketing and economics lit-
eratures. See Bell and Lattin (1998) for a more detailed description of the entire dataset and visit
http://wrds.wharton.upenn.edu for additional details and documentation. The dataset contains both panel
and store level data for each product category.
16
approach. Our model analyzes the market shares of items and is therefore conditional upon
consumer decisions to first chose the store. Furthermore, they are conditional upon the
product offerings available in a particular week.
Raw Data
Data Cleaning. In assembling the data for the study, we use information from Euromonitor,
the IRI Marketing Fact Book and the Stanford Market Basket Database.13 The toothpaste
sales data come from a large supermarket retailer operating in Chicago and cover 104 weeks
(week 1 is the week beginning 6/1/1991 and week 104 is the week beginning 5/22/1993).
All prices are in September 1991 dollars as the original price series were deflated using the
Bureau of Labor Statistics’ Consumer Price Index. Sales are in ounces, and prices are price
per ounce of toothpaste.
The “outside good” (measured in ounces) is created as follows. The dataset contains
information on the number of panelists visiting our chosen store in a particular week, as well
as total sales for each item in the toothpaste category. The group of panelists in the store
represents a sample of customers and we assume that their purchase behavior represents
that of the entire population of shoppers visiting the store. We use panelist sample means
to estimate the total amount spent by an average customer (average market basket dollar
value), and divide total dollar sales for each week by this value to generate an estimate for
the total number of visitors that week. We follow the standard approaches (e.g. Nevo 2000)
to estimate the outside good and include any shopper who visited the store in that week as
part of the “market potential.” We further assume that if such a shopper decides to purchase
in the toothpaste category, they purchase an average-sized pack.
To determine the item set we start by selecting all 246 items that were sold in this store
at some point during the observation period. From this group we deleted 40 items for which
we had no attribute information, leaving 206 items.14 Another 35 items for which we had
13The full dataset, documentation and MATLAB code for estimation are available from the authors upon
request.14All but one of these items had less than 0.3 percent category share overall.
17
category information but were otherwise problematic – either because of very small share
or because they were other types of tooth cleanser (e.g., liquid cleansers or teeth whiteners)
– were removed. Their combined category share was less than two percent. Finally, we
removed the brand “Topol” which is a small share brand designed for smokers. The end
result is a choice set of 168 unique items that represent just under 92 percent of the entire
category sales for our store. Although the potential choice set is 168 items, on average only
95 of these are available on the shelf at any one time. This discrepancy speaks to the large
amount of diversity in the category in terms of rate of entry and exit of products, and the
prevalence of stock outs.
Attributes and Levels. Using the definitions provided by IRI, we identified the unique set
of attributes and levels that form what we will refer to as the attribute basis for the product
category. Each item is described completely by a combination of unique single-valued levels
of each attribute. There are four attributes (in addition to brand)
1. Flavor – the taste of the toothpaste (e.g., mint or bubble gum).
2. Form – texture and product delivery (e.g., translucent gel or more traditional opaquepaste).
3. Function – the purpose of the toothpaste (e.g., “anti-plaque” to protect the teethagainst plaque build-up, a hard substance that would otherwise require the consumerto visit a dentist to remove it).
4. Size – the pack size in ounces. While there are numerous levels of size, many of theseare essentially identical (e.g., 3.45oz versus 3.50oz). We pre-classify size into mutuallyexclusive and collectively exhaustive categories: small, medium, large, and extra large.
As noted earlier, the number of unique levels for each attribute is as follows: brand (10),
flavor (14), form (3), function (7), and size (4).
Within Brand and Attribute Share Distributions
As noted in the Introduction, full enumeration of the attribute basis leads to 11,760 possi-
bilities yet only 168 are observed in the dataset, suggesting that choice of product location
by manufacturers and retailers is non-random. A typical approach to modeling demand is
to use a market share function where the individual equations are aggregated items within
18
an attribute (usually brand). This approach is more readily justified if the brand shares for
each level of each attribute possess the same distribution as brand shares within the brand
attribute (see, e.g. Kamakura and Russell, 1989), yet may be problematic if this condition
does not hold.
When we analyze these distributions for our data we find multiple “gaps” in the attribute
grid representing no presence at that level by the corresponding brand. Some brands do not
compete at all at some attribute levels (approximately 40 percent of the attribute-brand
levels are not currently occupied) which suggests that aggregation over brand and simple
execution of the analysis at this level alone may be problematic for a category such as
this. Moreover, there are many discrepancies in the shares of brands across attribute levels.
For example, Crest toothpaste sells relatively more “gel” than “paste” form toothpaste and
seems to dominate many of the flavor levels to a greater extent than Crest’s brand share
would predict. In short, this argues that one should attempt to build a model that draws
on all of the data and not resort to trimming or aggregation of an arbritrary nature as an
end in and of itself.
4 Empirical Results
Our empirical analysis of the toothpaste category proceeds in two parts. First, we estimate
the attribute-share models for all five attributes (brand, flavor, form, function and size) and
report the model fits and parameter estimates for both the fixed effects and the price response
parameter. Second, we show how these estimates can be used to recover via calculation the
underlying item-level fixed effects and true price response parameter from the item-level
model. We defer an analysis of the substantive implications of our empirical findings to the
next section.
Attribute Share Models
Table 3 summarizes the findings from the five attribute share models. Recall that in accor-
dance with our analytical development, these models are estimated with an appropriately
19
mean-centered price. If this were not the case, we would be unable to recover the item-level
parameters (details are given below). The number of parameters and observations differs
across models in accordance with the number of levels, La contained in each attribute a.
The brand model, for example, contains 10 levels and therefore 980 observations (all models
are estimated over 98 weeks of data) with a total of 11 parameters (10 fixed effects and 1
price effect).
––––––––––––––
[ Table 3 about here ]
––––––––––––––
For ease of exposition we do not report all attribute-level fixed effects (these are available
upon request), but simply summarize them according to the minimum, maximum and median
values. The model fit is good in all cases, with adjusted R2 ranging from 89% for the flavor
model to 94% for the brand model, and all fixed effects are statistically significantly different
from zero. The price response parameter γa varies across attributes a = 1, . . . , A and provides
some indication of differential price sensitivity with form appearing to be the most price-
responsive attribute.
Calculation of Item-Level Effects
Table 4 presents a comparison of the estimated item-level coefficients (fixed effects and price
effect) from the item level model and the calculated item-level coefficients from the attribute-
level models shown in Table 3. For ease of illustration we present the results from the brand
attribute only (results for other attributes are identical). Several interesting observations
are immediately apparent
• Fixed Effect Recovery. The item-level fixed effects recovered via calculation are essen-tially identical to those obtained by direct estimation. We summarize the 168 item-levelintercepts using the minimum, maximum, median and mean values. The column enti-tled MSE reports the average squared difference between item-specific estimates fromthe OLS item-level model and those generated through the attribute model. It is clearthat regardless of whether one specifies a brand-share or form-share model it makes nodifference to the ability to recover the item-specific estimates.
• Price Effect Recovery. The attribute-level model recovers the price effect exactly.Recall that this calculation proceeds in two steps – calculated fixed effects are fed
20
into a second stage regression of log-differenced item shares less calculated fixed effectson mean-centered item-level prices. The researcher is free to specify any convenientattribute-share model and can be confident of obtaining the true price coefficient.
• Fit and Precision. The R2 column indicates a value of 0.56 for the item-level modelwhich is well below that obtained for the attribute-level models (see Table 3), suggest-ing considerably more unexplained variation in items. For the attribute-share modelthe value of 0.07 is for the second-stage regression used to obtain the price effect (i.e.,equation 26). The item-level model does however produce the smallest standard er-rors for fixed effects suggesting that our calculation approach that proceeds from theattribute-level estimates suffers from a loss of precision (see Appendix B). Nevertheless,no loss of precision is observed for the price effect.
––––––––––––––
[ Table 4 about here ]
––––––––––––––
Attribute-Level Price Elasticities
Our model allows computation of “elasticities” for attributes and provides insights into con-
sumer sensitivity to price changes for different product freatures. Table 5 reports elasticities
derived from estimates in Table 3, and under the assumption of a temporary price change.
––––––––––––––
[ Table 5 about here ]
––––––––––––––
All the price elasticities are larger (more negative) than -1. On average, the elasticities
increase across attributes as follows: size, brand, function, flavor and form. Consumers
appear relatively more willing to switch among say gel and paste than they do trade among
sizes and as one might expect, price elasticity increases as sizes get smaller. It interesting that
the “triple” function yields the lowest price elasticity (anti-cavity, tartar control and anti-
plaque are all more price-elastic) which is consistent with the notion that customers readily
substitute from products with individual functions to those with the bundled function.
21
5 Discussion and Conclusion
In recent years researchers and practitioners alike have devoted considerable time and re-
sources to the development and estimation of market share models. An important substantive
challenge in this literature is the identification of patterns of choice and competition that
allow one to draw inferences about market structure and market response. Unfortunately,
in many instances researchers are forced to make data pruning or aggregation decisions that
facilitate model estimation but are not necessarily true to the underlying structure of the
data. Drawing on insights from the panel data literature (FH 1996) we develop a modeling
approach that exploits analytical relationships between item-level parameters and composite
or attribute-level parameters obtained from models estimated at higher levels of aggregation.
We are able to recover the item-level parameters using a parsimonious specification for
the market shares and exploiting information generated by attribute-level analysis. The first
order of contribution is therefore methodological as this recovery procedure is implemented
without recourse to estimation of the underlying model. This new procedure also offers
the potential for substantive contributions. By examining the attribute-level results we can
develop additional insights into to demand response to changes in product characteristics.
Attribute-Level and Item-Level Price Elasticities
Russell and Bolton (1988) examine item-level price elasticities as a function of “sub-market”
price elasticities and item-level conditional market share elasticities. They show analytically
how given a proper definition of sub-markets one can derive a hierarchy of own and cross-
price elasticities. In our setup there exists a large number of potential sub-markets so our
approach to uncovering the relationship between attribute-level elasticities and item-level
elasticities is an empirical one. We first compute item-level elasticities based on fixed effect
parameters recovered by our analytical procedure and then regress these on the appropriate
attribute-level elasticities. That is, the elasticty for the SKU with the following features:
Colgate (Brand), Mint (Flavor), Gel (Form), Tartar Control (Function) and Medium (Size)
is regressed on the attribute-level elasticities for Colgate, Mint, Gel, Tartar Control and
22
Medium. The estimated coefficients from this regression represent the importance weights
for the attributes in determining the overall item-level elasticity.
One possible application is the estimation of likely price response to new items that
are not currently part of the category. To examine this idea, we selected eight new items
introduced in the last seventeen weeks of the data and calibrated the model using the first
eighty-seven weeks of data. The new items included five by Colgate, two by Crest and one
by Arm & Hammer. Table 6 shows: (1) parameter estimates from the regression of item
elasticities on attribute elasticities (given in the first row), and (2) price elasticity estimates
for the eight new items. The weights indicate the order of attribute importance and show
that size, brand and function are statistically significant while flavor and form are not. Actual
and predicted elasticities for the eight new items are shown in the first two columns of Table
6 (actual elasticities are calculated using the procedure set out in Appendix C, while the
predicted values are obtained from the regression). We are able to recover the item-level
price elasticities in the holdout sample fairly well (with the MAD at around 0.45, and the
MSE at around 0.26).
––––––––––––––
[ Table 6 about here ]
––––––––––––––
Comparison with a Pure Characteristics Approach
The FH model estimated on panel data – a pure characteristics approach – has generated
a lot of interest among both academic researchers and practitioners. This owes a lot to the
parsimony of the formulation and simplicity of estimation. In the FH model individual items
are expressed as additive linear functions of parameter values for levels of attributes that
describe the item. A reasonable question to ask is: Why not implement a direct replication
of the pure characteristics model on market share data (i.e., regress log differenced market
shares on the dummy variables for the levels of the different attributes)? To address this
question Table 7 reports the “part-worths” from the pure characteristics approach. The first
three columns contain results for a model with no mean-centering for price. In this case, the
23
estimated price coefficient (γ = −6.539) is underestimated. The results based on data witha mean-centered price correct this and replicate the price coefficient value obtained earlier
and shown in Table 4.
––––––––––––––
[ Table 7 about here ]
––––––––––––––
Points of difference between these findings and those from our method emerge
• Fixed Effects. The attribue-level parameter estimates (or “part-worths”) are consis-tently over-estimated when no mean centering is performed for the covariate (as notedearlier this also results in a biased coefficient for the covariate). Furthermore, there is arelatively substantial difference between the true fixed effects and those implied by thepure characteristics model. As shown in Table 4 our recovery procedure reproduces thefixed effects exactly (MSE is essentially zero), however, under the pure characteristicsapproach the comparable MSE is 0.503.
• Model Fit. The model fit (with R2 = 35 − 40%) is considerably worse for the purecharacterstics specifation. The attribute-level models have values of 90% and above.
• Attribute-Level Price Response. As shown in Table 3 our model allows one to examineattribute level price sensitivity while a pure characteristics approach does not. More-over, our approach facilitates forecasts of the likely price sensitivity to new items (seeTable 6) by relating the item-level elasticities to their attribute-level counterparts.
More insight into the relative merits of these approaches can also be seen via an in-sample
comparison of predicted item-level fixed effects for the top and bottom ten selling items.
Table 8 reproduces item-level fixed effects for three models: (1) estimated effects from the
true item-level model (IL), (2) calculated fixed effects from the attribute-level model (AL),
and (3) estimated fixed effects from the pure characteristics model (PC). The MSE difference
for the IL and AL models (see also Table 4) is very small (0.00014). For the IL and PC
models however this is considerably larger (0.50). This is also reflected in the rank ordering
of items. Column 1 shows that the IL and AL models produce identical rank orderings for
items, whereas column 5 shows markedly different rankings for the PC model.
––––––––––––––
[ Table 8 about here ]
––––––––––––––
24
While these findings are based on in-sample prediction the superior performance of our
model is also evident in holdout tests. Specifically, we compute the out-of-sample mean
utilities and market shares for the eight new items discussed earlier. To generate forecasts
for these new items from the AL model we translate the recovered item-level intercepts to
a set of part-worths for attribute levels. To accomplish this, and to allow comparison with
the PC model, we regress the recovered item-level intercepts onto the design matrix used
by the PC model.15 Using these recovered estimates, we forecast the mean utilities and the
market shares of each item in the holdout sample. The calibration sample was constructed
from the first 87 weeks of data and the holdout sample from the last 17 weeks (all eight new
items were introduced in this period). For the PC model the MAD and MSE values for out
of sample prediction of mean utilities for the new items are 0.2738 and 0.3298, respectively.
The AL model produces better out of sample forecasts with values of 0.2598 and 0.3022.
This pattern is repeated in market share prediction with the AL again producing better
results.16
Conclusion
We show analytically that a market share model specified at the SKU level is related to
another market share model aggregated to the attribute level, and that this relationship holds
for any desired attribute. This knowledge facilitates construction of item-level parameter
values by calculation from estimates obtained from the simpler aggregated model and avoids
the need for direct estimation of the item level model. This approach is validated empirically
using store-level scanner data from the toothpaste category.
Although the number of items changes dramatically from week to week and there is
15In this regression the number of observations is equal to the number of recovered fixed effects (i.e., 168).
This step in our method is essentially identical to that employed by Nevo (2000).16These values are obtained assuming that the prices of the new items are not known in advance. This
assumption is handled naturally in the AL model as mean-centering enforces the notion that the calculated
fixed effects capture mean levels of item share given an “average” marketing mix environment. If we use
knowledge of prices then the PC forecasts of mean utilities improve to yield MAD and MSE values of 0.2708
and 0.3085, respectively, however these are still below the values for the AL model.
25
the possibility of considerable interaction among attributes (e.g. Colgate’s tartar control
toothpaste may be better than that produced by Crest), our methodology is able to recover
parameters of the item level market share model very well. Out of sample forecasts reveal that
the attribute level model produces better predictions that those obtained from an obvious
alternative – the pure characteristics approach. The attribute-level model’s simplicity,
parsimony and reliance on analytical relationships make it an appealing method for both
researchers and practitioners. In subsequent work we plan to apply this new method to
substantive problems that require use and understanding of data from all SKUs in a product
category.
26
References
[1] Bell, David R. and James M. Lattin (1998) “Consumer shopping behavior and retail
price format: Why ‘large basket’ shoppers prefer EDLP,” Marketing Science, 17 (1),
66-88.
[2] Besanko, David, Sachin Gupta and Dipak Jain (1998) “Logit demand estimation under
competitive pricing behavior: an equilibrium framework,” Management Science, 44 (11,
part 1 of 2), 1533-1547.
[3] Brodie, Rod and C. deKluyver (2001) “Attraction versus linear and multiplicative mar-
ket share models: an empirical evaluation,” Journal of Marketing Research, 21, 194-201.
[4] Bucklin, Randolph E. and Sunil Gupta (1999) “Commercial use of scanner data: indus-
try and academic perspectives,” Marketing Science , 18 (3), 247-273.
[5] Camerer, Colin and Ho, Teck-Hua (1999) “Experience-weighted attraction learning in
normal-form games,” Econometrica, 67 (4), 827-874.
[6] Chen, Yuxin, James D. Hess, Ronald T. Wilcox and Z. John Zhang (1999) “Accounting
profits versus marketing profits: a relevant metric for category management,”Marketing
Science , 18 (3), 208-229.
[7] Chintagunta, Pradeep K., Bonfrer, Andre and Inseong Song (2002) “Investigating the
effect of store-brand introduction on retailer demand and pricing behavior,” Manage-
ment Science, 48 (10), 1242-1267.
[8] Cooper, Lee G. and Masao Nakanishi (1974) “Parameter estimation for a multiplicative
competitive interaction model,” Journal of Consumer Research, 1 (2), 36-44.
[9] ––— and ––— (1982) “Simplified estimation procedures MCI models,” Marketing
Science , 1 (3), 314-323.
[10] Dreze, Xavier and Stephen J. Hoch (1998), “Exploiting the customer base using cross-
merchandising and category destination programs,” International Journal of Research
in Marketing , 15 (5), 459-472.
27
[11] Fader, Peter. S. and Bruce G. S. Hardie (1996) “Modeling consumer choice among
SKUs,” Journal of Marketing Research, 33, (4), 442-452.
[12] Ho, Teck-Hua, and Chong, J.K. (2003) “A Parsimonious Model of SKU Choice,” Work-
ing Paper, Haas School of Business, University of California, Berkeley.
[13] Kamakura, Wagner A. and Gary J. Russell (1989) “A probabilistic choice model for
market segmentation and elasticity structure,” Journal of Marketing Research, 26 (4),
379-391.
[14] Leeflang, Peter S.H., Dick R. Wittink, Michel Wedel and Philippe A. Naert (2000)
“Building Models for Marketing Decisions,” Kluwer Academic Publishers: Nordwell,
MA.
[15] Montgomery, Alan L. (1997) “Creating micro-marketing pricing strategies using super-
market scanner data,” Marketing Science, 14 (4), 315-337.
[16] Nevo, Aviv. (2000) “A practitioner’s guide to estimation of random-coefficients logit
models of demand,” Journal of Economics and Management Strategy , 9 (4) 513-548.
[17] ––— (2001) “Measuring market power in the ready-to-eat cereal industry,” Economet-
rica, 69 (2), 307-342.
[18] Raudenbush, Stephen W. and Anthony S. Bryk (2002) “Hierarchical linear models:
applications and data analysis methods,” Sage Publications: Thousand Oaks, CA.
[19] Russell, Gary. J. and Ruth N. Bolton (1988) “Implications of market structure for
elasticity structure” Journal of Marketing Research, 25 (3), 229-241.
28
A Model Simulations
We first generated a set of market shares for items j = 1, . . . J over time t = 1, . . . , T (with outsidegood) based on an additive specification for the indirect utility as a function of attribute utilities. Theattribute utilities, and the simulated item level utilities, are already relative to a hypothetical “outside”good. That is, if the utility of the consumer with this structure were to be smaller than zero, then nopurchase would occur of this item. We used a combination of brand and size, and had a complete designof two sizes and three brands for a total of six “items”. In addition, we generated a set of prices withaverage values for prices unrelated to the mean utilities for the items’ attribute levels. The indirectutilities thus consist of the two components
Ujt = βj + γpjt
where βj = f(bj , sj) is the item level intercept, as a linear additive function f() of brand and sizeattributes respectively. The γ and βj are parameters to be recovered in the simulation. We add astandard normal residual to Ujt and transform the data to obtain regression equations for item-levelshares (see equation eqn:trans). To obtain the simulated attribute share values, we simply add upmarket shares of items that belong to a common level (e.g., same brand or same size) of a particularattribute. Formally, this can be done using the following transformation matrices
Bx =
1 0 0
1 0 0
0 1 0
0 1 0
0 0 1
0 0 1
Sx =
1 0
0 1
1 0
0 1
1 0
0 1
The matrix Bx will be used to produce attribute share equations for brands and the Sxmatrix attributeshares for sizes. A kronecker delta operator multiplies the transformation matrix by an identity matrixIT of dimension (T ×T ) so that we have the mapping of item shares to brand shares Sb = (IT ⊗Bx) sand item shares to size shares Ss = (IT ⊗ Sx) s where s denotes the item share vector of dimension(J × T )× 1. The design matrix for brand and size regressions can be created using Bx and Sx, simplystacked T times. For the item market share regressions we simply stack the identity matrix IT , T times.
With the simulated matrix of market shares, we build the estimation equations using the transforma-tions log sjt − log s0t, where the outside good value for the both attribute share and the item sharemodels have the same values for each observation t = 1, . . . , T (see equation 6. The estimation procedureis as in equations (15) or (25) and (26).
B Standard Errors
The standard errors of the item level intercepts are equal to
Var[βj ] = E[β2j ]− E[βj ]2
which is
Var[βj ] = E βb + ( bt − jt) + logsjtSbt
2
− E βb +1
T
T
t=1
logsjtSbt
2
29
= E ( bt − jt)2 + 2E ( bt − jt) log
sjtSbt
+ E logsjtSbt
2
− 1
Tlog
sjtSbt
2
= E ( bt − jt)2 +
1
T
T
t=1
logsjtSbt
2
− 1
T
T
t=1
logsjtSbt
2
(27)
where the second step is a result of expanding the terms in brackets and taking expectations.
This is a decomposition of the standard error into a component due to the item (unobserved at thispoint) and a component due to the brand, plus a component due to a mean-centered log ratio ofthe item share to the attribute share b. For the standard error term jt we can use the estimatedOLS residual from the pricing equation (26). The estimate of bt comes from the residual of the OLS
estimation of brand share. The standard error of the price estimate, γ, is s2(X X)−1, where Xis the (mean-centered) covariate, and s2 is the estimated variance, e e/(N − k). We use X here forgeneralizability, since it is possible to add any mean-centered covariate and preserve the validity of theestimated item-intercepts.
C Price Elasticities
Within any market share model, the estimated price coefficient γ is the same regardless of whethermean-centering of the price covariate is done or not. The usual calculation of a point elasticity for pricein the market share model is
ηj,k =∂sj∂pk
pksj
where ηj,k is the price elasticity for price of option k on the share of option j, and sj and pj are themean share and price, respectively, for option j. In the specification of market share given in equation(1) this becomes
ηj,k = γpj(1− sj) ∀ j = k
= −γskpj ∀ j = k
In the case of the mean-centered market share specification, the price “point” at which we take thiselasticity is zero, so normally this implies the price elasticity is zero. Recall that the price coefficientfor the mean-centered price is identical to the price coefficient for the non-mean centered price marketshare model. Therefore, we can utilize this. Note, however, the mean utility of choice j in the non-meancentered model is related to the attribute utility βmcj of the mean-centered model in the following way:
µjt = βj
=βmcj −γpj
+γpjt + jt
Now we need to take into account this relation in our elasticity calculation. Note that any such marketshare models that do not mean-center prices, should take this into account. By adding the covariateinto the specification, unless we mean center, the elasticity should take into account the indirect effectof any price change via the average of the covariate, on mean utility. The significance of this problemis likely to be small if one were only to predict a temporary impact of a price change on market share.However, if the analyst were interested in understanding how a permanent shift in price would affectmarket share, the inference would be biased by this non-centering relation. Thus the own-price elasticitybecomes:
ηj,j =∂sjt∂pjt
pjsj
30
= (γ − γ∂pj∂pj
)sj(1− sj) pjsj
= γ(1− ∂pj∂pj
)(1− sj)pj
So this implies that we need to consider the impact of −γ ∂pj∂pj
in calculating the elasticity. This value is
likely to be small if one is considering a temporary change in price (e.g. trying to calculate the marketshare impact of running a price-based promotion), but this result would be more important given amore permanent price change. A similar effect is present in the cross-price elasticity:
ηj,k = −γ(1− ∂pk∂pk
)skpk
Again, if these are temporary price changes, the impact is likely to be small. If the price change is morepermanent, the cross-price elasticity will need to take into account the impact of the price change onthe average price.
31
Table 1: Summary Statistics for the Toothpaste Category
A: Availability of Items
Inventory Status # of Items % of Items Share
Occasional stock-out 27 16.1 20.7
Established (always on shelf) 22 13.1 30.9
Added during observation period 41 24.4 20.5
Deleted during observation period 49 29.2 15.6
Added then deleted 29 17.3 12.2
B: Attribute Levels and Item Locations
Flavor Form Function Size
Brand levels 10 10 10 10
Attribute levels 14 3 7 4
Possible locations 140 30 70 40
Locations used 29 18 26 26
Percentage used 20.7 60.0 37.1 65.0
32
Table 2: Item Ownership
Manufacturer Share
Block Drug Co Inc 0.009
Church & Dwight Co Ltd 0.078
Colgate Palmolive 0.339
Procter & Gamble 0.373
SmithKline Beecham 0.099
Unilever 0.102
Herfindahl Index (1/n) 0.280 (0.167)
Attribute-level Herfindahl (1/n)
Brand 0.251 (0.100)
Flavor 0.490 (0.071)
Form 0.376 (0.333)
Function 0.417 (0.143)
Size 0.294 (0.250)
33
Table 3: OLS Estimation Results for Attribute-Share Models
Parameter Attribute-Level Estimates (s.e.)
Brand Flavor Form Function Size
Intercept (min) -6.87 -7.74 -4.39 -8.07 -5.98
(0.04) (0.04) (0.03) (0.05) (0.04)
Intercept (max) -2.92 -2.42 -2.43 -2.72 -2.60
(0.04) (0.04) (0.03) (0.05) (0.04)
Intercept (med) -5.16 -5.54 -2.94 -4.83 -3.29
(0.04) (0.04) (0.03) (0.05) (0.04)
Price -6.79 -10.30 -14.15 -8.15 -4.70
(0.35) (0.64) (0.74) (0.51) (0.86)
Adj R2 0.94 0.89 0.92 0.93 0.93
N (La × T )1 980 995 294 682 392
1All levels for all periods are present for Brand, Form and Size.
Table 4: Item-Level Intercepts: Estimated and Calculated
Model Parameter Values
Fixed Effect Summary Price R2
Min Max Median Mean MSE1
Item-Level: Estimated -9.081 -4.898 -7.089 -7.114 -7.473 0.561
(s.e.) (0.08) (0.12) (2.64)
Brand-Level: Calculated -9.061 -4.902 -7.099 7.112 1.44e-04 -7.473 0.076
(s.e.) (0.76) (0.96) (2.64)
1Mean squared difference between estimates from item-level and attribute-level models.
34
Table 5: Attribute-Level Price Elasticities
Brand Aim Aquafresh A&H Closeup Colgate Crest Gleem
-2.0186 -1.9581 -2.9584 -2.0632 -1.9312 -1.9793 -1.9505
Pepsodent Sensodyne Ultrabrite
-1.2634 -5.9942 -1.8801
Flavor1 BGum BMint CMint XFresh FMint Mint Orig
-3.7297 -3.7600 -3.0015 -2.8740 -4.1894 -3.3991 -3.1723
Reg WFresh ClMint NatMint HPunch OBGum SMint
-2.7708 -3.0954 -3.4396 -2.5192 -3.0045 -2.9872 -3.0503
Form Gel Paste Gel + Pst
-4.2748 -4.0655 -4.0795
Function Cavity Plaque Reg Tartar Triple Plq+Tar Sens
-2.6623 -2.6904 -2.3530 -2.3914 -2.2506 -3.3939 -6.9949
Size Small Medium Large X-Large
-2.4989 -1.7425 -1.3156 -1.1953
1Non-abbreviated flavor descriptions can be found in Table 7.
35
Table 6: Forecasted Elasticities for New Items
Price elasticities Regression Coefficients
Intercept Brand Flavor Form Function Size
Item Actual Predicted 0.5715 0.3321 0.0645 -0.001 0.248 0.9942
(0.357) (0.065) (0.050) (0.079) (0.039) (0.085)
23 -3.364 -2.578 -2.958 -4.189 -4.275 -2.391 -1.316
90 -2.720 -2.136 -1.931 -2.771 -4.066 -2.353 -1.316
91 -2.720 -2.157 -1.931 -3.095 -4.275 -2.353 -1.316
92 -2.723 -2.146 -1.931 -2.771 -4.066 -2.391 -1.316
93 -2.720 -2.145 -1.931 -2.771 -4.275 -2.391 -1.316
94 -1.830 -2.016 -1.931 -2.771 -4.275 -2.353 -1.195
140 -2.116 -2.073 -1.979 -3.399 -4.275 -2.353 -1.195
141 -2.496 -2.192 -1.979 -3.399 -4.275 -2.353 -1.316
MAD 0.452
MSE 0.259 R2=0.7655
36
Table 7: Results for the Pure Characteristics Model
Standard Mean-centered PriceParameter1 Estimate s.e. t Estimate s.e. tPrice -6.539 0.179 -36.487 -7.473 0.330 -22.616
Brand: Aim -0.318 0.101 -3.153 -0.587 0.105 -5.597Aquafresh -0.900 0.088 -10.260 -1.591 0.089 -17.836A & H 0.559 0.079 7.101 -0.164 0.079 -2.070Closeup -0.062 0.077 -0.804 -0.324 0.080 -4.056Colgate 0.611 0.073 8.432 0.141 0.074 1.895Crest 0.559 0.061 9.169 0.085 0.062 1.377Gleem -0.514 0.097 -5.320 -0.729 0.100 -7.266Pepsodent -0.797 0.112 -7.150 -0.758 0.116 -6.527Sensodyne -1.169 0.159 -7.372 -5.642 0.105 -53.894
Flavor: Bubble gum -0.085 0.090 -0.943 -0.246 0.094 -2.612Bubblemint 1.200 0.139 8.638 1.356 0.145 9.375Cool mint -0.403 0.073 -5.555 -0.593 0.075 -7.874Extra fresh -0.278 0.097 -2.863 -0.507 0.101 -5.034Fresh mint 0.193 0.073 2.637 -0.250 0.075 -3.334Mint -0.117 0.067 -1.747 -0.412 0.069 -5.981Original 0.062 0.073 0.846 -0.316 0.076 -4.186Regular -0.014 0.069 -0.211 -0.280 0.071 -3.943Winterfresh -0.099 0.089 -1.117 -0.213 0.092 -2.313Clean mint -0.033 0.274 -0.122 -0.230 0.285 -0.806Natural mint -0.557 0.446 -1.251 -0.699 0.464 -1.507Hawaiian punch -0.128 0.197 -0.649 0.059 0.205 0.290Orange bubble gum 0.239 0.193 1.234 0.436 0.201 2.168
Form: Gel -0.636 0.056 -11.373 -0.978 0.057 -17.045Paste -0.385 0.050 -7.639 -0.620 0.052 -11.912Paste and gel 0.000 0.000
Function: Anti-cavity -4.673 0.104 -45.085 -5.602 0.105 -53.533Anti-plaque -3.993 0.103 -38.676 -4.910 0.104 -47.090Regular -4.110 0.095 -43.434 -4.867 0.096 -50.611Tartar control -4.095 0.094 -43.639 -4.817 0.096 -50.412Triple protection -3.866 0.102 -38.067 -4.435 0.105 -42.431Anti-plaque/tartar -4.098 0.125 -32.922 -5.175 0.126 -41.092
Size: Small -0.628 0.062 -10.218 -2.057 0.049 -41.678Medium -0.593 0.036 -16.473 -1.283 0.032 -40.269Large -0.161 0.032 -5.018 -0.515 0.032 -16.157Adjusted R2 0.401 0.351N 9348 9348
1 Ultrabrite, Smooth Mint, Sensitive and Extra-Large normalized to zero.
37
Table 8: Comparison of Item Fixed Effects
IL1 IL AL PC PC2 brand flavour form function size
Top 10 Ranked By Item Utility1 -4.898 -4.902 -5.669 9 Colgate Regular Paste Plaque X Large2 -5.193 -5.207 -6.140 26 Colgate Regular Paste Regular large3 -5.284 -5.288 -5.576 2 Colgate Regular Paste Tartar X Large4 -5.293 -5.307 -6.140 27 Colgate Regular Paste Regular large5 -5.325 -5.321 -6.140 28 Colgate Regular Paste Regular large6 -5.430 -5.428 -5.626 5 Colgate Regular Paste Regular X Large7 -5.517 -5.523 -5.626 6 Colgate Regular Paste Regular X Large8 -5.543 -5.543 -5.631 8 Crest Regular Paste Tartar X Large9 -5.575 -5.575 -6.182 32 Crest Original Paste Tartar large10 -5.615 -5.609 -5.576 3 Colgate Regular Paste Tartar X Large
Bottom 10 Ranked By Item Utility159 -8.370 -8.357 -7.971 157 Aquafresh Regular P+G Tartar medium160 -8.394 -8.371 -7.217 117 Colgate Regular Gel Tartar medium161 -8.452 -8.426 -7.202 113 A’fresh Regular P+G Tartar large162 -8.474 -8.483 -8.436 165 A’fresh X Fresh Paste Triple medium163 -8.491 -8.507 -7.935 154 Colgate Wtr frsh Gel Cavity medium164 -8.501 -8.469 -6.519 60 Crest B’Gum Gel Regular large165 -8.711 -8.717 -7.908 152 A & H Fr Mint Paste Tartar small166 -8.843 -8.820 -6.306 37 A’fresh Regular P+G Triple X Large167 -9.025 -8.992 -7.589 136 A’fresh Regular P+G Triple medium168 -9.081 -9.061 -9.364 168 A’fresh Regular Paste Tartar small1 Rank order for the IL and AL models (order is identical).2 Rank order for the PC model.
Figure 1: Time Series of Toothpaste SKUs
Figure 2: Calculated Versus Estimated Fixed Effects