extending bi with big data analytics
TRANSCRIPT
Fastest Time to New Insights
© 2014 Datameer, Inc. All rights reserved.
Extending Analytics Beyond BI!
Audio!▪ Audio will be streamed over
the web for today’s webcast ▪ Make sure your computer
speakers are turned up and the volume is adjusted
▪ If you are having trouble connecting, please send the host a chat message through the chat window
Claudia Imhoff President, Intelligent Solutions, Inc. A thought leader, visionary, and practitioner, Claudia Imhoff, Ph.D., is an internationally recognized expert on analytics, business intelligence, and the architectures to support these initiatives. Dr. Imhoff has co-authored five books on these subjects and writes articles (totaling more than 150) for technical and business magazines. She is also the Founder of the Boulder BI Brain Trust (BBBT), an international consortium of independent analysts and experts. You can follow them on Twitter at #BBBT or become a subscriber at www.bbbt.us.
Email: [email protected] Phone: 303-444-6650 Twitter: Claudia_Imhoff
About Our Speaker!
Azita Martin @datameer CMO Azita Martin is Chief Marketing Officer at Datameer with extensive marketing leadership experience at high-growth start-ups and category-creating public companies like Salesforce and Siebel. Azita has global responsibility for scaling all aspects of Datameer’s product and corporate marketing, including defining go-to-market strategy, driving thought leadership, and increasing brand awareness and customer acquisition. Prior to Datameer, Azita built and led marketing teams for both fast-growing start-ups and major public companies, including Get Satisfaction, Moxie Software, LiveOps, Salesforce, Siebel and SGI.
#datameer @datameer
About Our Speaker!
Matt Schumpert @datameer Senior Director, Solutions Engineering Matt has been working in the enterprise infrastructure software space for over 14 years in various capacities, including sales engineering, strategic alliances and consulting. Matt currently runs the pre-sales engineering team at Datameer, supporting all technical aspects of customer engagement from initial contact through roll-out of customers into production. Matt holds a BS in Computer Science from the University of Virginia. #datameer @datameer
About Our Speaker!
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Agenda
§ Extending the Data Warehouse Architecture § Use Cases § Major Trends and Examples
7
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Disruptive Forces
§ Deployment Options
§ Mobile Work Force
§ Advanced Analytics
§ Big Data
§ Data Management
BUT disruption does not have to mean CHAOS!
8
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Next Generation BI
Based on a concept by Shree Dandekar of Dell Slide compliments of Colin White – BI Research, Inc.
New business insights
Reduced costs
New technologies
Enhanced data
management
Advanced analytics
New deployment
options
Next generation
BI
DRIVERS
TECHNOLOGIES
9
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
A Complex Environment
Multiple user devices
Multiple output formats
Multiple deployment options
Sophisticated analytics + complex analytic workloads Multiple data sources
Increasing data volumes & data rates
DW historical data
Web & social content
Sensor data
Operational data
Text & media files
Decision management
Data management
Data integration
Data analysis
Decision management
Slide compliments of Colin White – BI Research, Inc.
10
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Next Generation – Extended Data Warehouse Architecture (XDW)
Traditional EDW environment
Investigative computing platform
Data refinery
Data integration platform
Analytic tools & applications
Operational real-time environment
RT analysis platform
Other internal & external structured & multi-structured data
Real-time streaming data Operational systems
RT BI services Slide created by Colin White – BI Research, Inc.
11
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Agenda
§ Extending the Data Warehouse Architecture § Use Cases § Major Trends and Examples
12
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Systems of Record
§ Remember – It all starts here! § Transactional systems generate most of the data used for all other
activities – operational processes, BI & analytical capabilities, etc.
§ The point here is a reminder: § Extend OLTP systems of record as a “key” source of data § Many companies do not (or can not) leverage data they already
have in their operational systems
13
Operational systems
RT BI services
Other internal & external structured & multi-structured data
Real-time streaming data
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Use Case: Traditional EDW
14
Most BI environments today: § New technologies can be incorporated
into the EDW environment to improve performance, efficiency & reduce costs
Use cases: § Production reporting § Historical comparisons § Customer analysis (next best offer,
segmentation, life-time value scores, churn analysis, etc.)
§ KPI calculations § Profitability analysis § Forecasting
Data integration platform
Traditional EDW environment
Analytic tools & applications
Operational systems
RT BI services
real-time models & rules
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Use Case: Data Refinery
Ingests raw detailed data in batch and/or real-time into a managed data store
Distills the data into useful business information and distributes the results to downstream systems
May also directly analyze certain types of data
Employs low-cost hardware and software to enable large amounts of detailed data to be managed cost effectively
Requires (flexible) governance policies to manage data security, privacy, quality, archiving and destruction
Traditional EDW environment
Investigative computing platform
Data refinery
Data integration platform
15
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Use Case: Investigative Computing
New technologies used here include: § Hadoop, in-memory computing,
columnar storage, data compression, appliances, etc.
Use cases: § Data mining and predictive modeling
for EDW and real-time environments § Cause and effect analysis § Data exploration and discovery (“Did
this ever happen?” “How often?”) § Pattern analysis § General, unplanned investigations
of data
16
Data refinery
Data integration platform
Analytic tools & applications
Operational real-time environment
RT analysis platform
Investigative computing platform
Operational systems
RT BI services
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Use Case: Real Time Operational Environment
Embedded or callable BI services:
§ Real-time fraud detection § Real-time loan risk assessment § Optimizing online promotions § Location-based offers § Contact center optimization § Supply chain optimization
Real-time analysis engine: § Traffic flow optimization § Web event analysis § Natural resource exploration
analysis § Stock trading analysis § Risk analysis § Correlation of unrelated data
streams (e.g., weather effects on product sales)
17
Operational real-time environment
RT analysis platform
Other internal & external structured & multi-structured data
Real-time streaming data
Operational systems
RT BI services
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
All Components Must Work Together
18
analytic models analyses
Analytic tools & apps
Investigative computing platform
Data refinery Operational systems
existing customer
data
next best customer offer
3rd party data location data social data
feedback
RT analysis platform call center dashboard or web event stream
Slide created by Colin White – BI Research, Inc.
Traditional EDW environment
Other internal & external structured & multi-structured data
Real-time streaming data
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Agenda
§ Extending the Data Warehouse Architecture § Use Cases § Major Trends and Examples
19
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
1. What is the IoT?
20
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Investigative Computing: Hadoop Example
§ Predictive Analytics to Reduce Patient Re-admittance
o Goal is to predict the likelihood of hospital re-admittance within 30 days after discharge
o Patients with congestive heart failure have a tendency to build up fluid, which causes them to gain weight
o Rapid weight gain over a 1-2 day period is a sign that something is wrong
o Heart patients at home have a scale that wirelessly transmits data (uses iSirona) to Hadoop where an algorithm determines risk of re-admittance and alerts a clinician
o All home monitoring data will be viewable in the EMR via an API to Hadoop
21
“If Hadoop didn’t exist we would still have to make decisions
about what can come into our data warehouse or the electronic
medical record (and what cannot). Now we can bring
everything into Hadoop, regardless of data format or
speed of ingest. If I find a new data source, I can start storing it the day that I learn about it. We
leave no data behind.”
Source: Hortonworks
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
2. Evolution of Analytics
22
§ Select few § IT managed § Reflecting the business § What & why? § Within the four walls § Command/control § Discrete activities § Configured § A conscious thought § Tactical necessity
Expanding to From
§ Empowered many § Business led § Driving the business § What could & should? § The world around us § Sense/respond § Embedded everywhere § Composed § In everything we do § Strategic advantage
*From IBM
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Four Forms of Analytics
23
Based on Delen, Dursun and Demirkan, Haluk, “Decision Support Systems, Data, information and analytics as services,” from Elsevier, published online May 29, 2012
Business Analytics
Descriptive (Reactive)
Prescriptive (Proactive)
Predictive (Proactive)
What happened? What is happening?
• Business reporting • Dashboards • Scorecards • Data warehousing
Well-defined business problems and opportunities
What will happen?
• Data mining • Text mining • Web/media mining • Forecasting
Accurate projections of the future states
and conditions
What should I do? Why should I do it?
• Optimization • Simulation • Decision modeling • Expert systems
Best possible business decisions
and transactions
Out
com
es
Ena
bler
s Q
uest
ions
Diagnostic (Reactive)
Why did it happen?
• Behavioral analysis • Cause and effect analysis • Correlations
Cause and effects of changes in business
activities
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Predicting the Future
§ Netflix uses predictive analytics to produce “House of Cards” = most streamed piece of content in 40 countries § Netflix knew it was a hit BEFORE filming began by analyzing 30 M “plays” a day, 4 M ratings, etc.
§ They also analyzed the director’s track record, Kevin Spacey’s appeal, reaction to the British version, etc.
§ Benefit? To breakeven, Netflix needed to gain 565,000 more members. They brought in more than 17 Million!
§ Downside – impact on quality, diversity, even creativity?
24
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
3. Making Analytics More Consumable
25
§ Use of BI for decision making continues to be a high priority for organizations § Recent survey1 of 2,500 CIOs showed 83% of CIOs see BI &
analytics as the way to enhance an organizations’ competitiveness
§ But reach of BI is often restricted to those users with experience to exploit analytics for business benefit § 59% of users say that they miss information that might be
of value to their jobs because they can not find it § 27% of managers time is spent searching for information § 50% say the information they obtain has no value to them
§ BI must be more easily understood and consumed! § You need an architecture
1 “IBM Global CIO Study: The New Voice of the CIO”
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Making BI More Consumable – Information Consumers
26
Access Integrate Manage Report
Analyze Deliver
Make it easy to access and
Blend data
Make DM solutions fast to deploy & easy to manage
Make BI tools easy to use
Make BI results easy to consume
& enhance
Office product integration Portal integration + search
Business glossary & data lineage BI automation
Mobile BI Collaborative BI
Data visualization
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Making BI More Consumable – Information Producers
27
Access Integrate Manage Report
Analyze Deliver
Make it easy to access & blend data
Make DM solutions fast to deploy & easy to manage
Make BI tools easy to use
Make BI results easy to consume
& enhance
Customizable BI components
Ad hoc visual analysis tools Investigative BI workbench
Cloud computing BI sandboxes
Investigative BI platform
Data virtualization Big data
connectors Data blending
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Getting Started
§ Education is mandatory § This is not just training on BI tools § Education includes how to think analytically, how to interpret
results, who to ask for help § Advanced BI analysts (business analysts, data scientists, etc.)
must evangelize value of analytics
§ Many business people don’t know where to get training § May be embarrassed to ask for it or intimidated by it § May not even know what BI resources are available or what data
is available
28 From www. business-help.org
Copyright © Intelligent Solutions, Inc. 2014 All Rights Reserved
Getting Started
§ Governance still has an important role § Determine whether data used is “governed” (e.g., in a data
warehouse or MDM environment) or “ungoverned” (e.g., individual spreadsheets, external source)
§ IT must have monitoring and oversight capability § BI/DW builder needs to administer and manage infrastructure § Must be able to monitor the environment § Must have oversight into the environment
§ Note: LOB IT or experienced information producers may have to take on some previously traditional central IT roles § Security of data, adherence to privacy policies
29
Use Cases!
Social Media
Mobile Ads
Web Logs
CRM
Product Logs Transaction
Call Center
Are keywords related to customer segments?
Which campaign combinations accelerate conversion?
Which product features drive adoption?
What content works be best for each lead segment?
Which features do users struggle with?
What behavior signals churn?
Understand Your Customer Journey!
Call Center
Public Data
CRM
Web
Reduced customer churn by 50%
Reduce Customer Churn!
Public Data
Connected Home
Energy Consumption
Data
Time & cost savings for IT Reduced false alarms
User Behavior
Internet of Things!
Improved customer experience
Household Data
7 Billion lbs. reduction In CO2 Output
$500M/Year in Energy Savings
Energy Consumption
Data
Smart Meter
Smart Meter Analytics!
Demo
@Datameer www.datameer.com
For the webinar:http://bit.ly/1zxY3Nl
!