big data’s last crusade
TRANSCRIPT
Big Data’s Last Crusade
Dipock Das, VP Technology, HotSchedules
#FSTEC
Agenda
Why is Big Data critical to my business?
What is Big Data?
What Big Data challenges should I expect?
Big Data’s Last Crusade
Image: Raiders of the Lost Ark
Big Data
You
#FSTEC
Why is Big Data critical to my business?
Insight ForesightHindsight
A Data Driven Business
What happened?
Descriptive
Why did it happen?
Diagnostic
What will happen?
Predictive
How can we make
something happen?
Prescriptive
#FSTEC
QuestionResearchers at Cornell University found that using tip data from charge card sales reliably predicted food sales in the following month
1. Could you run the same calculations on your data today for all your stores?
2. How would you go about doing it?
3. How long would it take to run through the exercise?
Source: “Tips Predict Restaurant Sales” by Michael Lynn, Ph.D. and Andrey Ukhov, Ph.D.
#FSTEC
Which is OK if
you’re a Jedi...
So, without Big Data, you’re flying blind
#FSTEC
What is Big Data?
#FSTEC
big data is all around you
#FSTEC
Customers
Floor space
Sales
Utilities
Loss Prevention
Kitchen Staff
Transactions
labor
In-store Application DataRoster
InventoryMenu
Tables
Reservations
Punch in/out
Voids
Bar Staff
Wait Staff
Tasks
Counts
Deliveries Orders
#FSTEC
In Store Device Data - The Internet of Things
Ovens
KDS
Refrigerator
Thermostat
A/C Unit
Lighting Sensors
Probes
WiFi
Drive-Thru
Thermometer
Light sensor
Acoustic sensor
#FSTEC
Facilities
SurveysMarketing
Regional TotalsSales Roll Ups
Above Store DataLoyalty
Invoices
Utility BillsInventory Roll Ups
TaxationSuppliers
#FSTEC
Weather
Schools
Food SafetyCrime
Digital
Mobile Orders
Traffic Patterns
Demographics
Social Media
Near Store Data
Sports
News
Liquor License ApplicationsFood Inspections
House prices
#FSTEC
Big Data Characteristics
More systems, more data
Volume
Data changes state rapidly
Volatility
Data accuracy and truthfullness
Veracity
Data in many forms
Variety
Data in motion at high speeds
Velocity
#FSTEC
Something to keep in mind
It is not about the size of the data, it is the value within the data.
“It is about finding that data and it’s relationships that are important to achieve a goal, optimally.”
Bernard Marr, author “Big Data”
Dipock Das. FSTEC, 2015
#FSTEC
Big Data Use Cases
#FSTEC
How and where Big Data is used
Methods
Classification
Regression
Recommender
Clustering
Benchmarks
Predictions
Recommendations
Exceptions and Filters
Results Demand-based forecasting
Supply chain optimisation
Promotion effectiveness
Market Basket analysis
Category management
Price optimization
Loyalty Programs
Gamification
Applications
#FSTEC
Before Big Data - you only know what happened
Sales
Descriptive - you focus on transactions
#FSTEC
With Big Data you understand why it happened
marketing campaign
negative reviews
positive reviews
college starts
football event
Diagnostic - you focus on events
Sales
#FSTEC
Example #1: Pre Big Data - what happened
After many hours study, there is no insight into
what is causing the uptick.
Susan sees unexpected uptick seen in signature
menu item.
#FSTEC
With Big Data - why it happened
Customer posts a photo, leading to
increase in signature menu item orders.
There is a positive correlation between post
event and sales transaction uptick.
#FSTEC
Opportunity realised - Increased sales
Susan offers free beverage to people who post photos of
entrees.
Increase in patrons coming to the
restaurant.
#FSTEC
Opportunity realised - all stores benefit
Recommendation engine shows other store owners what Susan did and provides measurable results.
#FSTEC
Sally sees a decrease in profit
margin on the most profitable entree.
No change in the price of ingredients or change in sales
transactions.
#$
Example #2: Pre Big Data - what happened
#FSTEC
12:09PM notification“outlier detected
item used < item ordered”.
Correlate transaction data with video
found employee stealing.
With Big Data - why it happened
#FSTEC
Reduced theft and waste.
Problems detected automatically - saving
Sally time and reducing stress.
QSR lose up to 7% of sales to employee theft (NRA)Dunkin Doughnuts saved between 2-13% in sales in 2012.
$
Opportunity realised - Loss prevention
#FSTEC
But presenting data is not enough
Sales DOWN 8% in Oakland store after a 5% spike in sales.
#FSTEC
Give users actionable data
Sales UP 20% in San Francisco store.
#FSTEC
Big Data Challenges Image: Raiders of the Lost Ark
#FSTEC
Big Data Management Process
Identify objectives
Business
Evaluate model and conclusions
Test
Manipulate and draw conclusions
Model
Select and Cleanse
Prepare
Collect and review
Data
Apply conclusion to
business
Distribute
#FSTEC
#1: Single Vendor solution
Not best of breed, monolithic stacks, slow innovation, monopolistic behavior
SDrive-Thru
Mobile Orders
Social Media
Sales
Web Orders
Inventory
Loyalty
Labor
#FSTEC
#2: Best of breed, great, but creates silos
POSInventory MarketingSocial
#FSTEC
Results in manual data integration processes
Store 1
Manager
Store 2 Store 3 Store n
Spreadsheets, files, manual steps to consolidate dataPeople make mistakes, take vacation, files get lost
#FSTEC
#3: Platform Silos
I work in real time
I work periodically
Call me tomorrow
#FSTEC
Results in slow data pipeline processing
Transactional data
Analytical data
POS
I needed this yesterday.
Capture Extract, transform,
load
Dashboard
Deliver
#FSTEC
#4: Organizational Silos
Line of business focusResult: No information sharing, cross functional initiatives fail
Stores Human ResourcesMarketing
#FSTEC
#5: Big Data analysis needs a Data Scientist
Data Science is the exploration and quantitative analysis of all available
structured and unstructured data to develop understanding, extract knowledge, and
formulate actionable results.
Information does not convey what needs to be done.
#FSTEC
#6: Big Data solutions are too expensive
● Single Vendor solutions prohibitively priced
● Generic solutions priced for Enterprise (Banks, Telcos)
● Designed for structured data - some social, few machine
2001 A Space Odyssey
#FSTEC
#7: Results delivered to your desktop
● Takes you away from the store and the customers
● The information is not front of mind when you walk away
● Alerts don’t get to you when you need them
#FSTEC
Big Data Challenges
Consolidating data is time consuming and costly.
Dashboards not useful when you are away from the PC.
Data processing is too slow.
?
Information does not convey what needs to be done.
$Expensive
#FSTEC
Big Data’s Last Crusade
#FSTEC
Cloud services drive down Infrastructure cost
Prices have dropped by 25% in the last 3 years (CitiGroup)Source: Business Insider
“Analyze petabytes of data for less than $1,000 per TB/yr”
#FSTEC
Open source drives down the software costPlatforms and Tools
Spark, Hadoop, MapReduce, GridGain, HPCC, Storm
Databases / Data WarehouseCassandra, HBase, MongoDB, Neo4j, CouchDB, OrientDB, Terrastore, FlockDB, Hibari, Riak, Hypertable, BigData, Hive, InfoBright Community Edition, Infinispan, Redis
Business IntelligenceTalend, Jaspersoft, Palo BI Suite/Jedox, Pentaho, SpagoBI, KNIME, BIRT/Actuate
Data Mining RapidMiner/RapidAnalytics, Mahout, Orange, Weka, jHepWork, KEEL, SPMF, Rattle
File System Gluster, Hadoop Distributed File System
Programming Languages Pig/Pig Latin, R, ECL
Search Lucene, Solr
Data Aggregation and Transfer Sqoop, Flume, Chukwa
Source: 50 Top Open Source Tools for Big Data
#FSTEC
Cloud providers delivering more services
Big Data storage and compute services
Data analysis and Machine learning
Mobile support
#FSTEC
HotSchedules Platform as a Service
Cloud Platform, Extensible Model & Development Tools
Point of SaleSystems
KitchenDevices
CustomApplications
CorporateIT Systems
3rd PartyWeb
Services
RBCServices
Mobile Application Platform
Our Apps Your Apps 3rd Party Apps
Public API
In Store Integrations Internet Integrations
BUILD APPLICATIONS 10X FASTER @ 1/10th THE COST
#FSTEC
Getting started
● Create a cross functional team● Identify your project objectives● Create an inventory of the data you have● Insist your vendor provides an API for your data ● Look for solution providers who can
○ Help integrate your data○ Provide cheap cloud storage and compute○ Provide real time data analytics○ Deliver to mobile devices as actions and alerts
in addition to dashboards
#FSTEC
Roadmap for real time actionable intelligence
Cloud-based, recommendations,
exceptions and decisions.
Real time, integrated,
best of breed systems.
Automated data cleansing,
transformation and analysis of
inbound streaming data.
#FSTEC
Don’t forget - Going the last mile
Deliver actionable content to the users who need it right now.
#FSTEC
Deliver actionable content based on context
Guided instruction and recommendations on the next best action.
Right place and time yields better results and
faster response.
How did we do today?
#FSTEC
Thank you!
To find out more, please visit us at
Booth #1120.
https://www.hotschedules.com