© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

Building a Stronger Story for Your Machine Learning Pilot

Amanda Shiga

April 5, 2019

#sugcon

Quick Introduction

Formerly VP Marketing Science @Valtech

Currently Product Strategy @Sitecore

Sitecore MVP 2012-2018

2

• Please insert a background image that suits your presentation,

or leave it empty.© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

Part 1 of 3

In 2019, what are

practitioners facing as they

seek to build pilots?

4

• Please insert a background image that suits your presentation,

or leave it empty.© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

AI != ML | AI⊃ML

• Please insert a background image that suits your presentation,

or leave it empty.© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

• Machine learning lets us replace human judgement with

prediction (via statistical methods/algorithms)

§ Automate tedious processes by replacing human judgement with

prediction or pattern recognition

§ Discover new patterns, groupings, segments and behaviours we

didn’t know existed

Reminder: What is ML?

• Please insert a background image that suits your presentation,

or leave it empty.© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

What is still true?

• It’s still hard to pitch a project with unknown outcomes

• Everyone’s still struggling with data cleansing and consolidation issues

• Machine learning is hyped and it can be hard to define a business story

• The industry and the tech stacks are increasingly fragmented

A few observations

• Please insert a background image that suits your presentation,

or leave it empty.© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

What is still true?

• It’s still hard to pitch a project with unknown outcomes

• Everyone’s still struggling with data cleansing and consolidation issues

• Machine learning is hyped and it can be hard to define a business story

• The industry and the tech stacks are increasingly fragmented

What has evolved?

• An explosion of ML-related services and platforms with cost-effective pricing and easy, accessible APIs

• Customer experience and marketing use cases are covered by a growing number of AI applications

• Competitive advantage is increasingly related to trust, care of data and privacy concerns

A few observations

Part 2 of 3

Highlighted places applying

ML and AI to digital

experience

• Please insert a background image that suits your presentation,

or leave it empty.© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

Easy places to start

Recommenders

Examples:

• Users similar to you bought [x]

• Users who bought [x] also bought [y]

• Based on your social feed, we recommend [x]

Where it can help:

• Upselling, cross-selling, content suggestions

Measurements: Conversion rate, visitor engagement

Sample techniques: Train a recommender or call to a service

Segmentation

Examples:

• Propensity score

• Likelihood to renew

• Likelihood to churn

Where it can help:

• More targeted personalization

• Insight into customer segmentation and segment discovery

• Insight into behavior influencers; prevention of undesirable behaviors

Measurements:

Conversion rate, visitor engagement, churn rate

Sample techniques: Predictive modelling and Contact facet update

Helpers for content metadata

Examples:

• Automated tagging of metadata and profiling for content and images

• Content topic clustering

Where it can help:

• Shorter time to value for marketer, reduction of heavy lifting

Measurements: Time to execute workflows

Sample techniques: Semantic and sentiment analysis, NLP, image pattern recognition services

Helpers for author experience

Examples:

• Bots embedded in content author / marketer experience

• Intelligent/predictive search

• Query parsing and results relevance

Where it can help:

• Faster and assisted experience for content and marketing workflows

• Better discovery and usage of content

Measurements: Time to execute workflows

Sample techniques: Bots, NLP, predictive modelling

Customer Experience Marketing / Content Authoring Experience

Part 3 of 3

Recommended ingredients

to pitch and start your

project

• Please insert a background image that suits your presentation,

or leave it empty.© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

• Access to raw data and content

• Cortex Processing Engine – a way to scale out data

processing and call out to ML services (v9.1)

• Access to content and customer journeys

• Tangible use cases

• An amazing, helpful community

Basic ingredients with Sitecore

• Please insert a background image that suits your presentation,

or leave it empty.© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

1) Map out the datasets you can easily work with or generate

2) Define the use case or problem you’re solving

§ Keep it small and contained

§ Think about shortening time to value and low barriers to entry

3) Research platforms that provide help – whether tutorials, templates, pre-built pipelines or sales engineers

4) Don’t be afraid of a quick, “test and learn” approach but do be sure to close the loop. Plan where the benefit will ultimately manifest.

5) Build your story!

Planning your pilot

• Please insert a background image that suits your presentation,

or leave it empty.© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

Enriching your pilot ingredients

Basic Sitecore Goodies – Experience + Data

ML technology of choice Business benefit

framing

Broader enterprise benefits

Measurement

Data collection and transformations

Production readiness and scale

Risk assessment

New potential revenue streams

Operationalization of processing and output

Data governance and legislation

Competitive advantage

Basics

• Please insert a background image that suits your presentation,

or leave it empty.© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

• “It takes 60 minutes to create and

deploy a landing page”

• “Our conversion rate has improved

2% in the last year”

• “Our members’ level of engagement

has fallen 25% in the last quarter”

• “The marketing team spends 40

person-hours per month running

our personalization operations”

Special Mention: Measurement

• Please insert a background image that suits your presentation,

or leave it empty.© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

Special Mention: Measurement

• Please insert a background image that suits your presentation,

or leave it empty.© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

More story strengtheners

Help get your data house in order Explore further influencing factors Explore competitive advantage

• Start with a small, easy data footprint, such as your content tree or your xDB

• Quickly demonstrate additional value of data

• Catalyze broader efforts to cleanse and catalog enterprise data

• Make data a team sport

• Add a simple exercise of adding a wildcard stakeholder such as the planet, the city or a child citizen

• Ask questions such as “how your system could negatively impact individuals?”

Grab the attention of responsibility-minded stakeholders

• Add breadth to your xDB dataset with use of other Sitecore features or feature engineering

• Get creative with other data sources such as social feeds or open data sets

Think through possible risks

• Asynchronous jobs vs. real-time call• Production site traffic vs. authoring • COTS vs. in-house solution • Skillset for ongoing model operations

Expand your collection window

• Explore whether you have an advantage in the data you collect or your visitor population

• Do you have a unique way of applying existing services or models?

• Do you have unique access to other business processes where predictions could surface?

• You may start with a short collection period

• Take the opportunity to keep collecting ideally over one year to account for seasonal variations. What changes?

• Please insert a background image that suits your presentation,

or leave it empty.© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

Recall the use cases

Recommenders

Examples:

• Users similar to you bought [x]

• Users who bought [x] also bought [y]

• Based on your social feed, we recommend [x]

Where it can help:

• Upselling, cross-selling, content suggestions

Measurements: Conversion rate, visitor engagement

Sample techniques: Train a recommender or call to a service

Segmentation

Examples:

• Propensity score

• Likelihood to renew

• Likelihood to churn

Where it can help:

• More targeted personalization

• Insight into customer segmentation and segment discovery

• Insight into behavior influencers; prevention of undesirable behaviors

Measurements:

Conversion rate, visitor engagement, churn rate

Sample techniques: Predictive modelling and Contact facet update

Helpers for content metadata

Examples:

• Automated tagging of metadata and profiling for content and images

• Content topic clustering

Where it can help:

• Shorter time to value for marketer, reduction of heavy lifting

Measurements: Time to execute workflows

Sample techniques: Semantic and sentiment analysis, NLP, image pattern recognition services

Helpers for author experience

Examples:

• Bots embedded in content author / marketer experience

• Intelligent/predictive search

• Query parsing and results relevance

Where it can help:

• Faster and assisted experience for content and marketing workflows

• Better discovery and usage of content

Measurements: Time to execute workflows

Sample techniques: Bots, NLP, predictive modelling

Customer Experience Marketing / Content Authoring Experience

1 23 45 6 7 8 910 11 12 13

• Please insert a background image that suits your presentation,

or leave it empty.© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

Community Pilots, POCs and Projects

Contacts Type of Solution Category Technologies Detail

Amanda Shiga Predictive model for high engagement value and micro-conversions Personalization / segmentation

RapidMiner Link

Una Verhoeven Content entity clustering for better tagging Marketer efficiency RapidMiner Link

Una Verhoeven Book recommender based on purchase data combined with personality insights data from Twitter

Personalization IBM Personality Insights, Watson, Cortex

Link

Mark Stiles Image tagging, NLP and chatbot integrations with “Sitecore Cognitive Services” available as a module

Marketer efficiency and personalization

Microsoft Cognitive Services Link

George Tucker / Richard Quance

Product recommender called within purchase funnel Personalization Azure ML Link

Sergey Baranov Predictive model for segment assignment based on scoring (on contact facet) Personalization / segmentation

ML.NET, Cortex Link

James B. Williamson / Chris Auer

Predictive model for membership renewal (on contact facet) Personalization / segmentation

RapidMiner Link

Varun Nehra / Kelly Brennan / Zachary Kniebel

Key Phrase Extraction to automatically profile the entire content tree Marketer efficiency Azure Text Analytics Cognitive Services

Link

Sandy Foley Retrieves Sitecore data for Alexa via Cognigy AI flows Marketer efficiency Cognigy AI Soon

Varun Nehra / Dmytro Shevchenko

Symposium session recommender via text classification of tweets Personalization Azure ML Link

Alistair Deneys Album recommender based on past purchases (via custom Outcome) Personalization Spotify Recommender, Cortex

Link

Jason Wilkerson Movie recommender based on past rentals (via custom Outcome) Personalization Movie Recommender Service, Cortex

Link

Rodrigo Peplau Contact identification via face recognition and personalization based on profile behavior

Personalization Microsoft/Kairos Face Detection

Link

1

2

3

4

5

6

7

8

9

10

11

12

13

• Please insert a background image that suits your presentation,

or leave it empty.© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

• eConsultancy / Sitecore – The Business Case for Digital Investment

• Benedict Evans (Andreessen Horowitz) – Does AI Make Strong Tech Companies Stronger?

• Gartner - The Business Value of Artificial Intelligence, Worldwide, 2017-2025

• Gartner - Applying AI to WCM and DXP: Key Use Cases

• Forrester - Artificial Intelligence Will Spark A Real Marketing Renaissance

• Gartner – How Data Science Projects Deliver Business Impacts

• Susan Etlinger (Altimeter)– The Customer Experience of AI

• Dr. Ann Cavoukian (Privacy By Design) – Privacy Leads To Innovation

• Integrate.AI – Responsible AI in Consumer Enterprise

Reading List

© 2019 Sitecore User Group Conference Europe and its respective speakers. All rights reserved.

I am a proud community member! Please contact me on the following handles:

sitecorechat.slack.com twitter

Questions?

@amandashiga @amandashiga

Thank you!@amandashiga