social media measurement & the holy grail.42011
TRANSCRIPT
1
THE HOLY GRAIL?• Don’t believe it for a minute, it doesn’t exist!• But some metrics are better than others.• There are many social media metrics available today.
– Many simply count words– Some have rudimentary sentiment differentiators
• We have developed a special metric called the Semantic Engagement Index (or SEI).– It is based on the principle that conversations on social media
are more than just words or what is said, but how it is said and the context of the conversation. Semantics matter.
– We directly compare our SEI with other social media sentiment metrics from a leading social media aggregator company.
SEI AND SOCIAL SENTIMENT METRICS• The comparison is with our Semantic Engagement Index and a Social Media
Sentiment indicator from a leading social media data company. As you can see, the Sentiment Indicator has poor and somewhat contradictory correlations with our selected company’s sales. The SEI metric, however, shows strong and robust correlation and logically consistent to the positive and negative forms.
SEMANTIC ENGAGEMENT INDEX: NUTS & BOLTS
• This metric is based on an algorithm devised with the assistance of Boyd Davis, Professor of Applied Linguistics at the University of North Carolina and a vendor partner named Linguistics Insights. The process we go through here involves four steps and it leverages linguistics science heavily.1. First we secure large quantities of social media conversations
filtered for topics on the specific brand of interest. In the case of the hospitality client, this was focused on online review sites pertaining to hotels, resorts and cruise lines.
2. Next we parse these conversations data into positive and negative toned conversations.
3. Next we apply our proprietary algorithm which quantitatively “scores” each of the positive and negative groups along the two dimensions of “emotional affect” and “personalization”
4. Finally, we time-code each conversation and aggregate into a time series metric.
CASE STUDIES• To fully leverage the SEI for our clients, the task is to understand its
impact on their business.• To do this, we do exploratory analysis to see how relevant the
metric is to the customer demand of a number of clients.• Then we utilize the SEI within a full marketing response (aka, mix)
model in order to not only understand its impact on the business, but also how it interacts with and is affected by direct marketing.
• First, however, we compare the customer SEI metric to 2 clients’ customer demand over time.– One client is a food & beverage retailer, while the other is in the
hospitality industry– The specific SEI metric we use here, and in our model, is the SEI ratio
of positive to negative tonality conversations.
TOTAL HOSPITALITY BOOKINGS• For our hospitality client, the key focus of our SEI metric was “online” review sites
for hotels, resorts and cruise lines. This makes the metric a proxy for customer satisfaction. As shown, the correlation is a very robust +76%!
• Net positive "engagement" (SEI) mirrors company seasonal patterns suggesting that the metric captures more than just social networks, with a correlation of 87%!
• We think it is likely a reflection of total "word-of-mouth" and a proxy for consumer good will.
7
TOTAL FOOD & BEVERAGE SALES
Correlation of Sales to SEI Metric is 87%(Total 156 Weeks)
SEI AND MARKETING RESPONSE MODELS
• To fully leverage the SEI for our clients, the task is to understand its impact on their business.
• By incorporating SEI metrics into marketing response (aka, mix) models, we can:– Come to a better & more precise understanding of how
social media buzz affects a client’s business performance– Understand the impact and interactions of the client’s
marketing and media as it affects social media conversations about their brands.
– Provide strategic guidance as to the most effective ways for monitoring and managing social media conversations on brands
9
THE IMPACT OF SOCIAL MEDIA TONALITY• A key insight we uncovered across clients is the difference between “positive” and
“negative” conversations about a brand. In absolute terms, the negative-toned conversation has a significantly greater net impact. It is actually better for a firm to try to manage a reduction in negative toned conversations than to increase positive ones.
Sale
s Im
pact
Sale
s Im
pact
Change in "Engagement" by Tone
Total Retail Sales Elasticity of Response(change in Sales versus change in Tone components of "Engagement")
Total Retail Impact of 90% Change(assumes 100% Decrease in Negative is not realistic)
negative
positive
10
THE IMPACT OF SEI, SALES AND MARKETING
• We also learned that a there is both a direct and an indirect effect from client media & marketing. As you can see, the impact of Semantic Engagement is quite large. The “indirect” effect of marketing is due to the impact on the SEI metric, itself. Overall, this indirect effect is shown to greatly enhance the impact of the client’s marketing and significantly improves marketing spend ROI.
net driven by marketing
8% +6% +13% = 27%
Engagement
Sub-model
Total Retail Sales Contribution
LESSONS LEARNED
• Through a number of case studies, BBDO’s “the Worth” has attempted to push back the frontier on social media metrics and understanding
• By linking a metric of “Semantic Engagement” to client sales and demand, we have shown that this approach shows great promise as a diagnostic for understanding social media’s impact on a client’s business by including it as an input into marketing response (aka, mix) models.
• Some of the key lessons that we have learned include:– For a service-based hospitality client, the consumer’s rating of the quality of
the firm’s service, is in fact, the most important driver of that firm’s business.– That negative brand conversations have a greater absolute impact on a
client’s business than corresponding positive reviews, i.e. negative word-of-mouth travels wider & deeper than positive.
– That the direct impact of SEI on business is large and significant– We learned that the value and ROI of marketing is greatly enhanced due to
the indirect effect it has on sales through its direct impact on Social Media Engagement (SEI)
– That our SEI metric is no Holy Grail, but it shows much promise in delivering un-matched insights on how social media conversations have a direct and tangible impact on company performance.