using predictive analytics for customer intent mining
TRANSCRIPT
Using Predictive Analytics for Customer Intent Determination
INFORMS Annual 2014
© 2013 24/7 Customer, Inc. All rights reserved. CONFIDENTIAL
Saturday, April 15, 2023
Samik Raychaudhuri, Ph.D.Ravi Vijayaraghavan, Ph.D.
Introduction
© 2013 24/7 Customer, Inc. All rights reserved. CONFIDENTIAL 2
• [24]7 is a software company based out of Bay area, US and Bangalore, India, delivering customer support solutions enhanced by predictive technologies
• Using predictive models to drive enhanced customer experience is an emerging and niche area of application of analytics and big data
• Our machine learning models on big data predict the customer intent across various touchpoints in real time, helping us provide an intuitive experience when the customers (of our clients) contact us
3© 24/7 Customer, Inc. 2013. All rights reserved. CONFIDENTIAL
2.5BDigital Interactions/Year
4.5TBInteraction Data/Week
90%+CSAT across channels
100M Visitors/Year
1stTrue Multi-modal Solution
1stOmni-channel Solution
We deliver a cloud-based software platform that uses predictive analytics and big data to make company-to-
consumer connections intuitive.
[24]7 - World’s Largest Self-Service Network
4© 24/7 Customer, Inc. 2013. All rights reserved. CONFIDENTIAL
[24]7 – Recent TechCrunch Article
http://techcrunch.com/2014/11/10/14-years-in-the-making-247-buys-intelliresponse-for-its-customer-service-suite/
5
Assist (for Chat)
Smart chat platform for online and mobile engagement
Assist (for IVR)
Call deflection to mobile web chat for higher NPS and ROI
Assist (for Voice)
Smart voice agent platform for multi-modal engagement of voice callers
SELF SERVICE
PRODUCTS
ASSISTED SERVICE
PRODUCTS
© 2014. 24/7 Customer, INC. All rights reserved. CONFIDENTIAL
Predictive Sales
Drive higher incremental revenue and customer acquisition
Predictive Service
Reduce customer effort to increase CSAT and NPS in customer service
Chat Agents
Chat agent services that engage customers and help reduce costs, generate revenue, and improve CSAT
Voice Agents
Voice agent services that engage customers and help reduce costs, generate revenue, and improve CSAT
SOLUTIONS
SERVICES
Social
Social sharing
Mobile
Mobile self-service
Vivid Speech
Mobile for IVR
Speech
Speech self-service IVR
[24]7 iLabs: A Quick Snapshot
Data Sciences @ [24]7 iLabs
Chief Data Scientist
R&D Data Infra Structure Speech Science Client Analytics
Client and Delivery Orientation
IP Asset Generation
25%PhDs
80+
DataScientists
50
Patents
75%Masters+
• Areas of Expertise - Machine Learning, Data Mining, Statistics, Operations Research, Speech Recognition,
Natural Language Processing, Econometrics, Math Modeling.
• IP assets created in critical areas such as – Natural Language and Speech Recognition, Omni-channel Intent
Prediction, Design of Experiments, Agent performance, Text Mining, Social Mining etc.
Data Science and Customer Experience
© 2014 24/7 Customer, Inc. All rights reserved. 77
V0.0•No Data•No Tools•No Scale•Ad Hoc Metrics
V1.0•Structured Data•BI Tools•Offline Analysis & Decisioning •Scale•Metrics on Efficiency, Quality and Compliance
V2.0•Structured and Unstructured Data•Analytics tools (SAS)•Offline Analysis & Decisioning•Scale•Metrics on segment level CSAT, NPS, loyalty, value•Siloed Channels
V3.0•Structured and Unstructured Data•BIG DATA infra/tools•Offline Analysis and Real-time Decisioning•Massive Scale•Metrics on Individual Customers - preferences, issues, sentiments•Integrated Channels and Devices
Dat
a S
cien
ce M
atu
rity
Data Science – What it means for [24]7
© 2014 24/7 Customer, Inc. All rights reserved. 8
fn (Customer type,
location, Identity, interaction context, journey, behavior …)
Intent: Purchase; issue with product or service, …
Customer Intent Engine
Intent Models
fn (Identity, ntent type,
history, channel affinity, customer value…)
Measure: usage, containment, repeat…
Engagement Engine
Guided self-
service
“ ”
Chat
Phone
Sales
Resolution
Experience
Retention
Metrics: conversion rate, revenue, CSAT, …
Outcomes
Machine Learning At Scale
Creating Personalized Intuitive Consumer Experiences
Anticipate Simplify Learn
Big Data in [24]7
© 2014 24/7 Customer, Inc. All rights reserved. 9
Big Data
• Web & IVR Logs• Web Journeys• Transcripts• Social media• CRM• Customer history• Product mix
• Surveys• Switch data• Agent performance• Agent dispositions• Agent notes
Big Data Platform: Technologies
© 2013 24/7 Customer, Inc. All rights reserved. CONFIDENTIAL 10
• We use varieties of open-source or free technologies to power our platform. Some of the technologies in use:
• Real Time Data Platform (RTDP)
• Apache Cassandra ring [http://cassandra.apache.org/]
• Jetty server for execution [http://www.eclipse.org/jetty/]
• BDP
• Apache Hadoop [http://hadoop.apache.org/]
• Apache Hive [http://hive.apache.org/]
• Apache Spark [http://spark.apache.org/] [Upcoming]
• Others
• Apache Kafka [http://kafka.apache.org/]
• Apache Avro [http://avro.apache.org/]
• HP Vertica database [http://www.vertica.com/]
• Apache Pig [http://pig.apache.org/]
• Apache Storm [https://storm.apache.org/]
Use case of intent prediction: Web visits
© 2013 24/7 Customer, Inc. All rights reserved. CONFIDENTIAL 11
• For our clients in the retail vertical, we provide chat agents who are experienced in providing differentiated support
• The differentiation is based on:• Current phase of the journey
• Specific persona of the visitor
• Essentially using targeted web chat to drive an intuitive experience
12
How to Engage - All customers are not the same...
At any point of time on a website..
Geeks
• Attention to details
Deal seekers
• Looks for discounts
Convenience buyers
• Has a specific need• Wants to finish sale
fast
Chatty-Cathy
• Gives a lot of context on the purchase
Novice
• Doesn’t have a specific need in mind
We find customers with different behaviors
Each persona needs a different kind of engagement.
27%
6%
8%
14%
15%
13
How to Engage - Personalizing experiences improves engagement and increases revenue
Conv
ersi
on R
ate
Geeks
• Attention to details
Chatty-Cathy
• Gives a lot of context for every action
Geeks when engaged with more technical details, are more likely to
purchase
Customers with Chatty-Cathy persona when engaged with more
questions, are more likely to purchase
Geeks Others0.0%0.5%1.0%1.5%2.0%2.5%3.0%3.5%4.0%4.5%
Less technical details More technical details
Chatty-Cathy Others0.0%
0.5%
1.0%
1.5%
2.0%
2.5%
Less probing questions More probing questions
1.38X 1.46X
38%46%
0.3
0.3
0.9
0.8
0.8
0.5
0.3
0.3
0.8
0.30.20.1
0.3
0.3
0.3
0.9
0.6
0.3
0.7
0.6
0.40.3
0.10.9
0.3
0.8
0.30.6
0.2
0.8
0.7
0.10.5
0.4
0.3 0.10.2
0.30.3
0.7 0.50.3
0.3
0.9
Targeting with Business Rules
14
Rule 1
Rule 2
Rule 3
0.3
0.3
0.9
0.8
0.8
0.5
0.3
0.3
0.8
0.30.20.1
0.3
0.3
0.3
0.9
0.6
0.3
0.7
0.6
0.40.3
0.10.9
0.3
0.8
0.30.6
0.2
0.8
0.7
0.10.5
0.4
0.3 0.10.2
0.30.3
0.7 0.50.3
0.3
0.9
Targeting with ML Models
15
The Inside Story: Data Fusion to Predicting Intent
© 2014 24/7 Customer, Inc. All rights reserved. 16
Web Self Service
•User ID•Search•Referrer•Journey•Behavior•Time
Speech Self Service
•User ID•Journey•Geo•Effort•Recognition•Completion
Chat
•User ID•Transcript•Agent Tags•Experience•Resolution
Mobile Self Service
•User ID•Geo•Journey•Behavior
Big Data
Identity
Location
Prior Context
Current Journey
Intent Models
17
Web Self Service: Data from Weblogs
© 2014 24/7 Customer, Inc. All rights reserved.17
1.23.172.207 - - [30/May/2012:00:08:21 -0400] "wid=1338350897094&mid=0&vid=d47c88d3-6e3c-4206-a780-1b7c240a808&bsid=1338350897097-379718&ts=1338350897131&...&title=Essential G570 15.6" Laptop | Shop | [24]7| US& meta_NumRating=37&meta_AvgRating=4.3& meta_TaxoTyp=SubSeriesPage& meta_ModelName=G Series& meta_ModelNum=[24]7 G570 - Best & prod_List=$699.00`& prod_ECoupCode=DOORBUSTER0524`& vc=1& ref=http://www.google.com/url?sa=t&rct=j& q=[24]7 Essential G570 &source=web&cd=1&ved=0CMkEEBYwAA&url=…&"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5"
RAW WEBLOGS
•Search term•Organic/Paid•Search Engine•Campaign•Referrer•Geo•……….
•Journey/Path•History•Internal Search•Time•Product•Behavior•………..
+
Web Logs
PRE-DOMAIN DOMAIN
Web Self Service: Interaction Data
© 2013 24/7 Customer, Inc. All rights reserved. CONFIDENTIAL 18
• We also capture interaction data from web self-service journeys
• page-level activity
• Vertical specific activity
• Retail:
• products wishlisting
• products checkout
• Banking
• Opening new account
Model Trained to Extract Intent from Chat Interaction
© 2014 24/7 Customer, Inc. All rights reserved. 19
Intent Type: PurchaseProduct Choice - Tablet
Collaborative Tagging
Customer: I'm looking to buy a tablet?
Machine Learning
Availability Check forVisualpad A170e
Availability and price check for Visualpad tablet
Promotions for Ideapad A745 Accessory Availability
INTENT Classification
Agent: Thank you for contacting us. How may I help you today?Customer: I am looking to buy a tablet.Agent: May I know the price range you are considering?Customer: Around 400$Agent: I would recommend X series.Customer: Ok.Agent: I need your fist and last name to create a quote.Customer: Phillip Jones.
Customer Intent
Chat Transcripts
Specific Case Study – Example Types of Models
20
Chat Propensity
Inte
nt
to P
urc
hase Target
Population: Chat &
Purchase
Target Population:
Chat & Purchase
Low
High
Chat Propensity ModelLog (Chat/No Chat) = a + b* # of pages + c*present
page + d*previous pages + e*interaction history +
f*referral + g*search
Purchase Propensity ModelLog (Purchase/No Purchase) = a + b* time on
session + c*present page + d*previous pages + e*referral
+ f*landing page + g*browser
Assist if X & Y> Thresholds
Evaluation for cut-off
score
Visitor Purchase
Propensity Score - X
Visitor Chat PropensityScore - Y
Iteration Every 10 secs
A Computer Manufacturer with Global Reach
21
Nov-Dec '12 Jan-Mar'13 Apr-Jun'13 July-Sep'13 Oct-Dec'130
1
2
3
4
5
6
7
8
Results: Steady revenue growth via deployed models
© 2014 24/7 Customer, Inc. All rights reserved.
*Monthly revenues averaged for quarter*Program started in Nov’12
First set of target and when to invite models
for conversion and engagement
Campaign segment target models for e-mail,
display ads and SEO with contextual invites
Staffing models and User experience initiatives
around look and feel of chat invite
Segmented targeting models by key domains
Segmented targeting models for bouncers and Repeat Visitors
22
And Incremental Performance
© 2014 24/7 Customer, Inc. All rights reserved.
1 20
1
2
3
4
5
6
7
Accurate targeting, measurement and data driven performance
management ensured that incremental lift is created through chat
Con
vers
ion
X
5.94X
Hot LeadSelf Serve
Hot LeadChat
Technology
© 2013 24/7 Customer, Inc. All rights reserved. CONFIDENTIAL 23
• The trained models are made available on our big-data platform
• The platform uses Apache Cassandra for data storage and Apache Storm for real-time reporting
• Data from Cassandra is used to evaluate the prediction function in real time
Cassandra DB Raw Data Transformed Data Prediction
Challenges and Future Research
© 2013 24/7 Customer, Inc. All rights reserved. CONFIDENTIAL 24
• Data preparation• Quality framework and dashboarding
• Performance of models in real-time
• Continuous monitoring
• Automated retraining of models• Determining session length
• Standard 30 mins from last interaction
Questions
© 2013 24/7 Customer, Inc. All rights reserved. CONFIDENTIAL 25