Copyright © 2016 Earley Information Science1
Predictive Analytics, AI and the
Promise of Personalization
May 25, 2016
Copyright © 2016 Earley Information Science
Seth Earley, EIS
Dino Eliopulos, EIS
Julie Penzotti, Amplero
Adam Pease, Articulate Software
Copyright © 2016 Earley Information Science2
Today’s Agenda
• Welcome & Housekeeping
• Dino Eliopulos, Managing Director, Earley Information Science
(@DEliopulos)
• Session duration & questions
• Session recording & materials
• Take the polls & the survey!
• The Panelist Point of View
• Seth Earley, CEO, Earley Information Science (@SethEarley)
• Julie Penzotti, VP, Customer Analytics, Amplero
• Adam Pease, CEO & Principal Consultant, Articulate Software
• Expert Panel Discussion
• Questions & Answers
• Join the conversation: #earleyroundtable
Copyright © 2016 Earley Information Science3 Copyright © 2016 Earley Information Science
Predictive Analytics, AI and the Promise of
Personalization
Copyright © 2016 Earley Information Science4
Dino Eliopulos - Biography
Dino Eliopulos
Managing DirectorEarley Information
Science
Deep specialization User experience and highly complex business
applications
Over 20 years of experience Machine Learning, Data Mining and other AI techniques
applied to deliver rich content-driven solutions
Financial Services, Retail / CPG, Telecommunications,
Travel and Entertainment, Healthcare, Pharmaceuticals,
Hi-Tech Manufacturing and Energy
Strategy, planning, forecasting, budgeting, measurement,
sales, talent acquisition / management and retention,
career stewardship, program management and service
delivery
IT professional services
Highly collaborative and results-oriented management
style delivers outstanding outcomes for clients, employers
and teams
Industry experience
Experienced leader and innovator in industry and high-end professional IT consulting
Outstanding outcomes
Copyright © 2016 Earley Information Science5
Predictive Analytics, AI and the Promise of Personalization
5
Personalization has been the big promise for the past 15 years. The problem is that this
vision is still a long way from reality.
Meaningful personalization requires
• meaningful knowledge and content assets
• the use of analytics to understand and model customers
• prediction to anticipate what they need and principles of AI
to fulfill the promise
This roundtable will review the state of the industry and discuss the problems and
challenges inherent in understanding and anticipating user needs and ways that
organizations can move the needle, improve engagement and move up the
personalization functionality maturity curve.
Copyright © 2016 Earley Information Science6
Personalization, predictive analytics and product data
6
Personalization, Predictive Analytics and Product Information three aspects of making
more appropriate recommendations for customers.
• Well curated product information in an ontology is at the core of ecommerce
offerings
• Predictive Analytics draws from sales and customer data sources in order to
provide product recommendations
• Personalization depends on rich structured product content and digital assets
• Machine Learning algorithms can improve the effectiveness of targeting and
improve effectiveness of merchandizers
• A combination of crowdsourcing, merchandizing expertise, curation and
automated techniques can be leveraged in an optimal solution
Copyright © 2016 Earley Information Science7
Personalization Components
Grouped Product
SetsProduct Data
Sales Data
Contextualized
filtering
Customer
Profile Data
Search History
Solution DomainsSolution
Landing Pages
Natural Language
Processing &
Inference
Domain
KnowledgeInformation
Extraction
PDF/
Unstructured
7
Copyright © 2016 Earley Information Science8
Seth Earley - Biography
Seth EarleyCEO and Founder
Earley Information Science
Over 20 years experience
Current work
Co-author
Editor
Member
Former Co-Chair
Founder
Former adjunct professor
Guest speaker
AIIM Master Trainer
Course Developer &
Master Instructor
Data science and technology, content and knowledge
management systems, background in sciences (chemistry)
Enterprise IA and Semantic Search
Information Organization and Access
US Strategic Command briefing on knowledge networks
Northeastern University
Boston Knowledge Management Forum
Long history of industry education and research in emerging fields
Academy of Motion Picture Arts and Sciences, Science
and Technology Council Metadata Project Committee
Editorial Journal of Applied Marketing Analytics
Data Analytics Department IEEE IT Professional Magazine
Practical Knowledge Management from IBM Press
Cognitive computing, knowledge and data
management systems, taxonomy, ontology and
metadata governance strategies
Copyright © 2016 Earley Information Science9
• Personalization is based on “electronic body language”
– Web site behaviors, click streams, downloads, consumed content
– Past purchases
– Social media
– Social graph
– Explicit preferences
– Derived attributes
– Hidden characteristics
Personalization signals
Copyright © 2016 Earley Information Science10
• The question is, what do we offer?
• Once we know something about a user, what do we do with that
knowledge?
• We are trying to give them something we think they want
– In the context of their task
– To meet a specific need
– Solve a particular problem
• Personalization is making a recommendation about a product, service,
solution, piece of content or next action based on what we know in
advance and what the customer is telling us at that moment
Personalization as Recommendation
Copyright © 2016 Earley Information Science11
• Offers – need offers that can be recombined
• Content – content to support the user’s task
• Products – what products will be presented to the user
• Rules (derived or developed) – how will assets and content be
assembled for the user
• Need to identify and understand customer segments, behaviors
and content to drive desired behaviors
Components of Personalization
Copyright © 2016 Earley Information Science12
Mine data sources for customer behaviors and product groupings
12
• Product Attributes derived from analytics
• Correlating POS data, PIM, Tech Support
• Structured textual data mining text
• Reuse rich and mature ontology
• Inference engine to deduce relationships
• Derived, curated and synthesized product data to support customer tasks and processes
• Integrated into user experience to generate custom suggested search results
How can we infer what products customers want to see when they enter a search term?
Can we improve conversions of products based on search engine marketing (paid search)?
Knowledge Content
• Portion of revenue from high value customers
• Time between purchases
Sales analysis
• High value customers
• One time buyers
• Lapsed customers (retargeting)
• Tasks, solutions, interests
Customer Profiles
• Keyword searches and subsequent behaviors (conversions vs abandonment)
Web Behaviors
• Hi value product bundles, product bundles
• Segment and product bundle relationships
Product Data
• Organizing principles and related content
Competitors/Suppliers
PRODUCT DATA ENHANCEMENT
DATA MINING
DATA SOURCESEXISTING ECOMMERCE PLATFORM
RESULTING DATA ASSETS
What products are purchased together?
What keywords lead to what behaviors?
How are customers described and grouped?
+
Copyright © 2016 Earley Information Science13
Analysis Approach
13
Use case Input Analytical
approach
Output Purpose or Benefit
Sales pattern
analysis
Order size, product
mix
Unsupervised to
identify clustering
Sales correlated with
customer types, segments
and product combinations
Repeat customers, one time buyers, lapsed customers (for personalized
retargeting offers), time between purchases – customer journey
(segment and product bundles), customer value, top value customers
generating most revenue, highest profit, portion of revenue from high
value customers and related clickstream behaviors
Real time behavior
(electronic body
language)
Search logs, paid
search, click
stream data, email
marketing results
Supervised learning
to identify keywords
leading to high
margin sales clusters
Keywords and messaging
clustered with concepts and
customer attributes leading
to conversions
Insights on keywords and concepts related to purchase funnel – what
people do once they are on the web site, where they abandon based on
intent inferred from email traffic, organic search, paid search and onsite
search and subsequent behaviors
Competitor
analysis
Product results
from keywords
Crawling, content
mining, graph
building
Target product classes for
optimization
Compare search results from failed searches with competitor results to
identify opportunities for improved experience
Interests
questionnaire
Customer
responses, sales
data
Combination of
supervised and
unsupervised
Topic map of possible
offering areas aligned with
customer interests
Correlate interests with RFM, seasonality, industry, product bundles,
high value customers, tasks, solutions
Failed conversions Web traffic logs,
web analytics
Tree search
methods (binary,
monte carlo, etc)
List of
terms/phrases/concepts for
optimization
Supervised learning to identify customers who are abandoning cart
versus not abandoned, what search terms are driving customers to
abandon, areas for remediation (content for search terms)
Copyright © 2016 Earley Information Science14
Discovery of product combinations
Identify competitive differentiators, strategic initiatives, priority categories.
5 – 10 target processes
Products grouped to support task, process or solution
MERCHANDIZERSTarget categories
Target processes
Intelligent Parser
USE CASES TARGET PROCESSES
PRODUCT COMBINATIONS
KNOWLEDGE AND EXPERTISE CONTENTCustomer Support Content
Maintenance manuals
Key Opinion Leaders
What products are used in combination?
Supports SEO, surfaces expertise and related content
RELATED CONTENT
14
Copyright © 2016 Earley Information Science15 Copyright © 2016 Earley Information Science
Poll Question #1
What is the maturity level of your knowledge and use of data-driven
personalization?
Copyright © 2016 Earley Information Science16
• None
• Dabbling
• Successful proof-points
• Concentrated capability development
• Core strength
What is the maturity level of your knowledge and
use of data-driven personalization?
Copyright © 2016 Earley Information Science17
• Continued development of Amplero, a self-learning
personalization technology platform for B2C marketing
automation
• Focus on deep data understanding, developing key findings,
driving customer interpretation/understanding
• Previously a scientist and consultant in the pharmaceutical
industry, specializing in data mining and analytics for drug
discovery
• 25+ publications and several patents.
• Earned a Ph.D. in Bioengineering and M.S. in Physical Chemistry
from the University of Washington and received her B.S. in
Biomedical Engineering from Duke University.
Julie Penzotti - Bio
Julie PenzottiVP, Customer Analytics
Amplero
CONFIDENTIAL
Purchases,Usage,Contacts,Demographics,Social Connections& their demographics,Propensity ModelsOther…
Big Data and the Age of the Customer
Channel,Day,Time,Location,Other…
Execution
Offer,Offer Expiry period,Incentive Type,Incentive Amount,Message,Creative,Semantic Tags,Other…
Experience
Context
Customer
Today’s customers…• Expect you know them• Are fickle and jaded• Tell others what they think• Are always connected• Are empowered to act
CONFIDENTIAL
Unfortunately, rules-based approaches don’t scale for B2C
[ 19 ]
With20 – 30
targeting ruleswhich onewill work
best?
?
CONFIDENTIAL
Machine Learning to automate and optimize targeting at scale
[ 20 ]
Modelling & Enrichment
Marketing Asset
Library
Machine Learning
Experimenting
Enriched Data
Decision Tree
OffersCustomer
DataCustomers
Marketer
Decisioning
Tomorrow’s marketer…• Agile and responsive• Runs campaigns in a loop• Gathers and applies
insights constantly• Thinks empirically rather
than intuitively• Let’s the machine do the
heavy lifting
CONFIDENTIAL
Machine learning to discover personalized contexts that optimize performance
[ 21 ]
DISCOVERED BY AMPLERO
Revenue Lift: +4%Confidence: Low
CONFIGURED BY CAMPAIGN MANAGEROffer: Unlimited Upgrade
Eligibility: International Saver Plan Subscriber
Revenue Lift: -4%Confidence: Medium
Revenue Lift: +8%Confidence: Medium
Condition:+Voice Consumption Cluster 5
Condition:+Voice Consumption Cluster 4
Revenue:+14%High
Revenue: -1%High
Revenue:-5%High
Offer Price: $10 $15 $20
Revenue:+6%High
Revenue:-10%High
Smart Package Owner: No Yes
KPI
Targets
KPI
Controls
Revenue Lift: +10%
Confidence: High
CONFIDENTIAL
Multi-armed bandits to manage decisioning for marketing contexts:
– Hedge bets about which choice is best
– Increasing certainty as more response data is gathered from customers
– Exploration/exploitation trade-off permits agility and adaptation
– Generalized learning over customer and marketing attributes
– Automatically segments population according to responses to different experiences
[ 22 ]
Machine learning for adaptive personalization and maximum benefits
Mean Lift Estimates of PerformanceContext 1 Context 2 Context 3 Context 4
Probabilityof Selection
Bandit Policy
Customer Attributes + Experience + Execution
OptimizationModels
Copyright © 2016 Earley Information Science23 Copyright © 2016 Earley Information Science
Poll Question #2
What is the appetite and interest of applying machine learning in
your organization?
Copyright © 2016 Earley Information Science24
• Organization still skeptical
• Open to, but not sure where to dive in
• Trying out techniques
• Clear identified ROI and priorities
What is the appetite and interest of applying machine
learning in your organization?
Copyright © 2016 Earley Information Science25
Adam Pease - Bio
Adam Pease
CEO & Principal
Consultant
Articulate Software
History
Undergrad CompSci, Doctorate Linguistics
Program Manager, Teknowledge (mostly DARPA contracts)
CEO, Articulate Software (commercial consulting in data
modeling)
Cognitive R&D Manager, IPsoft
Specialties
Suggested Upper Merged Ontology
Open source, higher-order logic, 15 yr history, mapped to
WordNet
http://www.ontologyportal.org
Sigma
Open source, Reasoning, ontology modeling, deep NLP
“Ontology: A Practical Guide” - 2011
Adam Pease – Articulate Software Earley Executive Roundtable
Don't forget about knowledge based methods
• What's the problem you're trying to solve?
• There's more than just matching to do
• Matching methods reaching asymptote on many tasks
• Semantics is often what's missing
• Semantics and KR matters
• What's most popular may not be the best technical
solution
Adam Pease – Articulate Software Earley Executive Roundtable
Personalization as Dialogue
• Are we making the problem too hard?
• Billings and Reynard (1981) – 73% of air traffic incident
reports involved problem in communication
• People have problems answering questions and communicating too
• Dialog is how we address the problem with people
Adam Pease – Articulate Software Earley Executive Roundtable
Knowledge Discovery
• Use Data Mining to discover trends and relationships
• Express them in computable semantics
• Can be explained
• Spurious correlations can be understood and corrected
• Consolidate gains – don’t learn things that are already known
Adam Pease – Articulate Software Earley Executive Roundtable
Suggested Upper Merged Ontology
• Initial versions: 1000 terms, 4000 axioms, 750 rules
• Mapped by hand to all of WordNet 1.6
• then ported to 3.0 and continually updated
• Associated domain ontologies totalling 20,000 terms and 80,000 axioms
• Now linked with factbases including YAGO for millions of facts
• New ontologies of Hotels and Dining
• If-then rules, not just a taxonomy or semantic web structure
• Free
• SUMO is owned by IEEE but basically public domain
• Domain ontologies are released under GNU
• www.ontologyportal.org
Copyright © 2016 Earley Information Science31 Copyright © 2016 Earley Information Science
Poll Question #3
Does your organization take a rules-based or statistical based
approach to personalization?
Copyright © 2016 Earley Information Science32
• Primarily rules-based
• Primarily statistical-based
• Both
• None
Does your organization take a rules-based or
statistical based approach to personalization?
Copyright © 2016 Earley Information Science33 Copyright © 2016 Earley Information Science
Panel Discussion
Copyright © 2016 Earley Information Science34
Roundtable Discussion
Dino Eliopulos
Managing Director Earley Information
Science
Seth Earley
CEOEarley Information
Science
Adam Pease
CEO & Principal ConsultantArticulate Software
Julie Penzotti
VP, Customer AnalyticsAmplero
Copyright © 2016 Earley Information Science35
Suggested Resources
• Introductory video on ontology - http://www.youtube.com/watch?v=EFQRvyyv7Fs
• Pease, Adam. “Ontology: A Practical Guide” - http://www.ontologyportal.org/Book.html
• SUMO on line - http://54.183.42.206:8080/sigma/Browse.jsp?kb=SUMO
• Ontology Publications - http://www.adampease.org/professional/
• Adam Pease’s Podcasts & Blog - http://www.ontologyportal.org/
• Earley, Seth. "Cognitive Computing, Machine Learning and Personalization: New Marketing Constructs or
New Capabilities?" KMWorld, November/December 2015. http://www.kmworld.com/
• “Making it Personal: Strategies for Creating Meaningful Customer Interactions”
http://www.earley.com/blog/making-it-personal-strategies-creating-meaningful-customer-interactions
• “Contextualizing Customer Journeys” Earley Executive Roundtable, Nov 2015
http://info.earley.com/roundtable-contextualize-customer-journeys
• Penzotti, Julie, “Marketing in the Age of Machine Learning: How optimising personalization granularity
leads to better performance in a dynamic market”, Applied Marketing Analytics, Vol 2 (1):41-51
• Amplero research links: http://www.amplero.com/research/article/?s=why-offer-response-rate-is-the-wrong-
metric-for-evaluating-marketing-performance
Copyright © 2016 Earley Information Science36
Earley Information Science
(EIS)
Information Architects
for the Digital Age
Founded – 1994
Headquarters – Boston, MA
www.earley.com
For more info contact:
Thanks to our Sponsors
Next Roundtable topicJune 22 – Site Search: The Battle for Relevance
Adam Pease – Articulate Software Earley Executive Roundtable
SUMO+Domain Ontology
Military
Geography
Elements
Terrorist Attack TypesCommunicationsPeople
Transnational Issues
Finance
Terrorists
EconomyNAICS
TerroristAttacks
DistributedComputing
BiologicalViruses
WMDECommerce
Services
Government
Transportation
World Airports
Total Terms Total Axioms Total Rules
20977 88257 4730
Relations: 1280
Hotel
Food
Hotel
Dining
Media Domain
Cars
UI/UX
SUMO
Mid-Level
Qualities
Mereotopology
Graph ProcessesMeasure Objects
Structural Ontology
Base Ontology
Set/Class Theory TemporalNumeric
Adam Pease – Articulate Software Earley Executive Roundtable
WordNet
• A dictionary for computational linguistics applications
• 100,000 word senses, hand-created
• Mapped by hand to SUMO
• Open source
• Semantic links
• Aid in computation
• Verification of meaning during construction
Adam Pease – Articulate Software Earley Executive Roundtable
Formal Ontology
• WordNet has synsets for “earlier” etc
• But nothing in WordNet would allow a computer to assert that the
end of one event precedes the start of another if one event is earlier
than the other
• This is not a criticism of WordNet
time
(<=>
(earlier ?INTERVAL1 ?INTERVAL2)
(before
(EndFn ?INTERVAL1)
(BeginFn ?INTERVAL2)))
Interval 1 Interval 2
Adam Pease – Articulate Software Earley Executive Roundtable
Example Rules
(=>
(instance ?DRIVE Driving)
(exists (?VEHICLE)
(and
(instance ?VEHICLE Vehicle)
(patient ?DRIVE ?VEHICLE))))
“If there's an instance of Driving, there's a Vehicle that participates in that action.”
Not just an English definition for humans to read, but a logical definition that can be used in proofs.