using facebook messenger bots for market...
TRANSCRIPT
Using Facebook Messenger Bots for Market Research
By Kate DuHadwayMICHIGAN STATE UNIVERSITY MASTER’S OF SCIENCE IN MARKET RESEARCH
A Proof of Concept Case Study
%a
AgendaA BRIEF HISTORY OF CHATBOTS
• What are they?
• What do they mean for the industry?
WHAT WE DID
• Look and feel of a FacebookMessenger-based survey
KEY FINDINGS
• Devices used and overall experience
• Demographics and tech adoption
• Costs and response rates
• Best practices and opportunities for further research
What Are Chatbots?
MINI APPLICATIONS THAT EXIST WITHIN
MESSAGING SERVICES
RESEMBLEONE-TO-ONE OR
PERSON-TO-PERSON CONVERSATIONS
USUALLY INCLUDE SOME FORM OF
ARTIFICIAL INTELLIGENCE
Applications that are designed to respond to conversational language - like narrow versions of digital assistants like Apple's Siri, Amazon's Alexa or Google Assistant, designed to perform specific tasks.Steve Ranger, ZDNet
A Brief History of Chatbots
I N T H E U . S . , C H AT B O T S H AV E T A K E N O F F J U S T W I T H I N T H E L A S T Y E A R
ELIZA,acomputerprogramdesignedbyJosephWeizenbaum,
professorofcomputerscienceatM.I.T.,iscreatedasa
therapist(butdoesn’tpassthe
Turingtest)
ClippyofferstipsforusingMicrosoftOffice
WeChatopensitsplatformtochatbots
inChina
FacebookopensMessengerplatformtobotdevelopers
inApril,nowhasover10,000 bots
MicrosoftlaunchesbotplatformforBing,Skype
AmazonlaunchesbotdevelopmentplatformforAlexa
Siricreatedin2008,debutsoniPhonein2010
KiklaunchesbotshopinApril2016,chatbotstakeoffontheplatform
1966 1996 2013 2016 20172014
AmazonlaunchesitsEchodigitalassistant,poweredbyAlexa
FacebooklaunchesGroupchatbots
GoogleannouncesChatbase,chatbotanalyticsplatform
2008
Are Chatbots theNext Mobile Apps?
*Source:Companies,BIIntelligence,2016,Global
45%
18%
10%
#1mostusedapp
#2mostusedapp
#3mostusedapp
Smartphone app use - % of time
TOTAL 73%Of time spent on top 3 mobile apps
TOP MOBILE APPS 150M
130M
116M
Montlyuniquevisitors
ComScore2016MobileAppreport,U.S.18+
Chatbots and Market Research
% %a
Using chatbots for market research is something that
hasn’t been fullyexplored yet
Researchers need to be where their consumers
are, which is increasingly mobile
Messaging appsare a natural and
intuitive way to reach people on mobile
Proof of Concept:What We Did
METHODOLOGYTwo experimental groups recruited via Facebook news feed promotions to take a survey on Facebook Messenger
INCENTIVES: POINTS FOR PANELISTS, AMAZON GIFT CARD FOR NON-PANELISTS
One control group recruited from our existing panel to take the same 10 min survey on our website
1ST GROUPActive panelists (targeted via email list)
2nd GROUPAnyone but active panelists
959TOTAL COMPLETES
VIA MESSENGER BOT
Total completes via website survey: n=1660
Look and Feel
Keys to a GoodUser Experiencea Simple, clear instructions
a Short response options
a Limited open ends
a Use of “quick reply” feature
Look and Feel
a Simple, clear instructions
a Short response options
a Limited open ends
a Use of “quick reply” feature
Keys to a GoodUser Experience
Look and Feel
a Simple, clear instructions
a Short response options
a Limited open ends
a Use of “quick reply” feature
Keys to a GoodUser Experience
Look and Feel
a Simple, clear instructions
a Short response options
a Limited open ends
a Use of “quick reply” feature
Keys to a GoodUser Experience
86% 80% 80% 77%
68% 69% 79%
74%
Smartphone Tablet Desktop Laptop
Overall Survey Experience by Device used, Top 3 Box
FacebookMessengersurvey Web-basedsurvey
Denotessignificanceatthe95%confidencelevel
Base:AlltechnologyintendersWeb-basedsurvey:n=1660FacebookMessengersurvey:n=620
User Experience
Easy to use and understand. The questions were simple and straight forward. Then the answers that I wanted to choose were easy to choose…didn't even have to use a drop down menu. –Danielle H.
76%
35%
Key Findings – Devices
DEVICES USEDTO TAKE SURVEY
9%
7%
7%
23%
32%
9%
Denotessignificanceatthe95%confidencelevelBase:Alltechnologyintenders.Web-basedsurvey:n=1660.FacebookMessengersurvey:n=620
Typically take our web-based surveys via smartphone
1 in 3
Smartphone
Desktop
Laptop
Tablet
Web-based survey
Facebook Messenger survey
Key Findings – Costs
Comparing Apples to Apples
Cost comparisons
reflect cost to recruit
a Messenger-based panel
With opt-in, we can
send notifications for new
surveys directly to panelists’
Messenger inboxes, pre-
screened based on previous
survey responses
Total costs were similar to
slightly
lower than previous panel
acquisition campaigns via
BEST PRACTICE
Using traditional panel for
longer/more complicated
survey designs, using
Facebook Messenger
for quick reads,
in-the-moment surveys,
mobile audiencea
Significantly more cost effective to target our current active panelists due to name/brand recognition
$
Key Findings – Response Rates
Qualified completesas a percentageof survey starts
%
39%42%Facebook Messenger Web-Based Survey
34%
41%
25% 25%
35% 40%
18to34 35to54 55andolder
General population (US) Messenger surveyrespondents
Key Findings –Demographics
Target:Gen Pop U.S.
U.S. FACEBOOK USERS
Female
52%Male
48%
MESSENGER SURVEY
Female
Male - 29%
71%
Data Findings – Intent to Purchase Yes/NoSignificant differences only among TV, VR Headset, and Streaming Device purchase intenders
10%
13%
16%
16%
16%
16%
18%
25%
31%
50%
19%
15%
12%
7%
23%
20%
17%
34%
28%
44%
Streamingdevice
Desktop
Smartwatch
VRheadset
Tablet
GamingConsole
FitnessWearable
TV
Laptop
Smartphone
In the next 6 months, do you intend to purchase a …?
Web-basedsurvey FacebookMessengersurvey
3%
3%
5%
6%
7%
7%
9%
10%
10%
5%
7%
2%
9%
5%
11%
6%
17%
16%
e-Reader
HandheldGamingDevice
ARglasses
Smartwear
Wearablecamera
DigitalHomeAssistant
2-in-1
Speaker/stereosystem
Homevideoplayer
In the next 6 months, do you intend to purchase a …?
Denotessignificanceatthe95%confidencelevel
Base:AlltechnologyintendersWeb-basedsurvey:n=1660FacebookMessengersurvey:n=620
Tech Adoption
§ Facebook Messenger respondents were more likely to be innovators when it comes to tech adoption
§ Web-based survey respondents were more likely to be part of the late majority
31%
15%
31%
13%
7%
2%
12% 12%
33%
23%
14%
5%
I’mactivelyonthelookoutto
buynewtechnologydevicesandservices
I'malwayskeentousenewtechnologyproductsassoonastheyenterthemarket
Iliketogetnewtechnology
productsafterthey’vebeenoutforawhile
IsometimesbuynewtechnologyproductsbutonlywhenIreallyneed
them
Ionlyreplacetechnology
productswhentheygowrongorarebroken
Noneofthese
FacebookMessengersurvey Web-basedsurvey
Denotessignificanceatthe95%confidencelevel
Base:AlltechnologyintendersWeb-basedsurvey:n=1660FacebookMessengersurvey:n=620
Questions?
Learnings andFuture ResearchOpportunities
Bot experience works best within
Facebook’s native Messenger app, or
within the Messenger-specific interface
in a browser.
Using 3rd-party scripting tool
vs developing a bot
from scratch
Biggest challenge:
Data collection
and analysis
Incentives – what amount is
ideal for this medium?
Qualitative research:
Seamless live chat
integration
Varied user inputs:
Audio, images, video, opt-in geolocation, app
data, or wearable data
Group chatbots for
Messenger-based
MROCs
Gamified research:
Chatbot games and
market research integrations