sapientnitro strata_presentation_upload
DESCRIPTION
PDF of presentation given by John Cain, Sheldon Monteiro, Thomas McLeish for Strata London 2013: Using big data to understand the mobile in-store shopping experience.TRANSCRIPT
Understanding experience: a mobile in-store shopping case study
Image by Ma* Bri*
Sheldon Monteiro John Cain
T J McLeish
2
§ A brief history of now: technology and the connected consumer
§ Moving from what to why: developing intelligence from the internet of things
§ Engines for insight: analytic architectures
The story in 3 parts
01 A brief history of now
Private & Confidential, SapientNitro © 2013
http://info.cern.ch/hypertext/WWW/MarkUp/Tags.html Private & Confidential, SapientNitro © 2013
2013
Private & Confidential, SapientNitro © 2013
Private & Confiden7al, SapientNitro © 2013
Image by Google Chrome team
8
customer experience
9
enterprise technology
Source: go-‐globe.com
The potential of all this data? Discovering the new, beyond the edge of current understanding
Étienne-Jules Marey 1901
Copernicus, 1473 Art Olson Scripps Research Institute 2011
THE MORE THINGS CHANGE…
13
Data is next
14
But wait!
What is the value of a single piece of data?
Image: Science Educa/on Resource Center, Carleton College © 2013 SAPIENT CORPORATION | CONFIDENTIAL
What is the value of a single piece of data?
Image: Science Educa/on Resource Center, Carleton College © 2013 SAPIENT CORPORATION | CONFIDENTIAL
02 Moving from what to why Instrumenting models of context, not isolated events
“What is” today: Data From the Real World
In Market Execution
Common mistake: poke and hope
Plan
© 2013 SAPIENT CORPORATION | CONFIDENTIAL
“What is” today: Data From the Real World
Overall Business Objectives
In Market Execution
A better way? The more info AND context we have, the better!
Models of understanding
Insight
Measure & Optimize
Working Business Hypothesis
Innovation
Plan
© 2013 SAPIENT CORPORATION | CONFIDENTIAL
20
21 Hugh Dubberly EPIC Keynote ‘Why Modeling is Crucial to Design and Design Research’
22 Hugh Dubberly EPIC Keynote ‘Why Modeling is Crucial to Design and Design Research’
23 Hugh Dubberly EPIC Keynote ‘Why Modeling is Crucial to Design and Design Research’
24 Hugh Dubberly EPIC Keynote ‘Why Modeling is Crucial to Design and Design Research’
25
Why are sales shrinking in my store? Where do people enter the baby section? Are smartphones cannibalizing in-store sales?
THE CONNECTED RETAIL EXPERIENCE
Image by Internetretailing.net
How did we get there?
Private & Confidential, SapientNitro © 2013
Start with a Question Often a question is framed in a way that
limits the power of data and analytics to
reveal a new paradigm
Questioning everything will create stratified
hierarchies of context
“Why are my in-store sales shrinking?” “What are people doing on their smartphones in my store?”
Become “hunts”
Understand the shopping experience, not just the buying moment. “Birth of a sale” and the broader customer journey
What are customers doing in my store?
WHY?
WHY?
Retail example
28
Where do good questions come from?
Aim of “question” -answering a broad or specific level of understanding
Purpose/use Context of imagined use
ASer Donald Stokes
Testable hypotheses Instrumention plan
In-store cart sensing -location -movement
Digital Activity 24x7 –smartphone -laptops -tablets
In-store observation -video -interviews
In-store engagement -paths & heat maps -attention -interactions
Conversion data -sales data -shopping lists
Transaction data
+ + + +
Aggregated View
03 Analytic Architectures Engines for insight
31
A look behind the curtain
Sense Store Analyze Model § Data e.g. noise
levels, light levels, accelerometer, selective video
§ “Anchored” to people, places, and things
§ Store + information extraction - real time and long term time series
§ “Lambda” architecture
§ Machine learning, statistics
§ Corroborating evidence and contextual awareness
§ Deduce meaning from activity
§ Stories and typologies of people and events
§ Narrated by hard data
© 2013 SAPIENT CORPORATION | CONFIDENTIAL
32
Event Collection Services
Non-‐blocking I/O using Play!, Scala, Java
Event collector 1
Event collector N
Serialization(Avro)
Distributed messaging
.
.
.
Visualization services(Non blocking, Play!, Scala, Java)
Dynamic/real-‐time visualization
ETL
Event StorageEvent Storage
HBase
Index(ElasticSearch, BigQuery,
CloudQuery, etc)
S3
MySQL
Modeling & Visualization
D3
Web pages
PDF/PPTs
Staticvisualization
Excel sheets
Mahout
R
AnalysisRandom/Interactive Queries
Batch Analysis/Processing outputArchive (S3/HDFS Sink)
Stream Services
Non-‐blocking I/O using Play!, Scala,
Java
Real time data
Kafka Node 1
Kafka Node N
...
Redis Node 1
Redis Node N
...
Spark
Time ranged queries
Custom map reduce/batch
jobs
HDFS
In Stream Processing Engine
In Stream Processing Engine
Storm
Spark streaming
Samza
External sources / partners
AnalystsAnalysts
Sanity testing, build algorithms,
etc
QueryingQuerying
ElasticSearch
Apache Drill
Hive/Pig/Map Reduce
Indexing API/Console
Cloudera Impala
AnalystsAnalysts
A look behind the curtain
Sense Store Analyze Model
33
Event Collection Services
Non-‐blocking I/O using Play!, Scala, Java
Event collector 1
Event collector N
Serialization(Avro)
Distributed messaging
.
.
.
Visualization services(Non blocking, Play!, Scala, Java)
Dynamic/real-‐time visualization
ETL
Event StorageEvent Storage
HBase
Index(ElasticSearch, BigQuery,
CloudQuery, etc)
S3
MySQL
Modeling & Visualization
D3
Web pages
PDF/PPTs
Staticvisualization
Excel sheets
Mahout
R
AnalysisRandom/Interactive Queries
Batch Analysis/Processing outputArchive (S3/HDFS Sink)
Stream Services
Non-‐blocking I/O using Play!, Scala,
Java
Real time data
Kafka Node 1
Kafka Node N
...
Redis Node 1
Redis Node N
...
Spark
Time ranged queries
Custom map reduce/batch
jobs
HDFS
In Stream Processing Engine
In Stream Processing Engine
Storm
Spark streaming
Samza
External sources / partners
AnalystsAnalysts
Sanity testing, build algorithms,
etc
QueryingQuerying
ElasticSearch
Apache Drill
Hive/Pig/Map Reduce
Indexing API/Console
Cloudera Impala
AnalystsAnalysts
A look behind the curtain
Sense Store Analyze Model GridEyes
Spider
Floor viz
Receivers API
Streams API
Messaging Blob
detec7on (in-‐stream)
HBase – Time series
DB
Map – reduce (custom jobs)
Blob detec7on (batch mode)
D3 / Processing Viz
Blob tracker APP / demo
© 2013 SAPIENT CORPORATION | CONFIDENTIAL
34
Connecting questions to sensors & analytics…
© 2013 SAPIENT CORPORATION | CONFIDENTIAL
35
What measured Acoustic, sound, vibration Electric current, magnetic, radio frequency Chemical presence, quantity Moisture, humidity Fluid velocity, flow Position, angle, displacement, acceleration Optical, light, imaging Force, density, level Thermal, heat, temperature Proximity, presence
Sensor library
Example ‘hard sensors’ Microphone Radio receiver, capacitive touch Chemical electrodes Capacitive humidity sensor Gas meter, water meter Accelerometer, bend sensor Photodiode, photocell Load cell, piezoelectric sensor Thermistor, calorimeter PIR motion sensor, RF transceiver …and so on
Use/application Sound recording, seismograph GPS, NFC, RF detection, touchscreen Fuel injector, smoke detector HVAC controls, weather, soil Utilities usage, air speed, Human activity, movement, motion Image recognition, light level detection Digital home scale, vibration sensing Digital thermometer Distance gauge, motion detector
36
pill bottle motion detection
bag tag regime tracking
Position and gesture analysis
bluetooth beacon
hacked scales
ambient environment sensor
observation gnomes
Custom hardware
Device activity logging
© 2013 SAPIENT CORPORATION | CONFIDENTIAL
From sensors to ‘instruments’
37
5 Key Takeaways
1. Dare to question everything –all understanding begins in doubt
2. Expand your toy box –no single data source contains the truth!
3. Create and instrument models of context –models are central to human understanding
4. Beware the observer effect –try to limit the impact of the measurement on the observation
5. Data scientists must amp up hermeneutics –it’s all about (your) intelligence
© 2013 SAPIENT CORPORATION | CONFIDENTIAL
Page 38
Recognized for Experience Design #1 Agency
#1 The Drum's Top 100 Digital Companies #1 Advertising Age- Largest Digital Agency US 2012
Highly Awarded for Creative Excellence 2013 Gold Mobile Lion for RBS GetCash Most Awarded Digital Agency at Cannes 2010 and 2011 176 recognitions in 2013 to date
Forrester ranks SapientNitro a leader in image-led and transaction-led web design
Global Leader in Omni-Channel Commerce eCommerce Integrator for over 50 major brands, including 2 of the top 5 most trafficked eCommerce Platforms for Black Friday 2012,
Leader in Mobile Capabilities Ranked #1 among US digital agencies by Forrester, April 2012
Consistently Strong Financial Performance Founded in 1990, ~11,000 People In 38 Offices in North & South America, Europe and APAC, $1.1B+ 2012 annual revenues
SapientNitro is a new breed of Agency redefining storytelling for an always-on world.
WE’RE HIRING! © 2013 SAPIENT CORPORATION | CONFIDENTIAL
© 2013 SAPIENT CORPORATION | CONFIDENTIAL
Sheldon Monteiro www.linkedin.com/in/sheldonmonteiro John Cain www.linkedin.com/pub/john-cain/0/285/a74 T J McLeish www.linkedin.com/in/thomasjmcleish/