harness the power of big data lawrence weber
DESCRIPTION
Harness the Power of Big DataTRANSCRIPT
Disruptive forces impact long standing business
models across industries
“Data is the new oil.
Data is just like crude.
It’s valuable, but if unrefined
it cannot really be used.”
– Clive Humby
“We have an economy based on a
resource that is not only renewable,
but self-generating. Running out is
not a problem, drowning in it is.”
– John Naisbitt
Shift of power to the consumer
Pressure to do more with less
Proliferation of big data
© 2013 IBM Corporation 3
The demand for big data solutions is real
The healthcare industry spends roughly $250 billion on healthcare
fraud, per year. By 2016, this could grow to more than $400 billion a
year.1
One rogue trader at a leading global financial services firm created
$2 billion worth of losses, almost bankrupting the company.
6 billion global subscribers in the telco industry are demanding
unique and personalized offerings that match their individual
lifestyles.2
$93 billion in total sales is missed each year because retailers
don’t have the right products in stock to meet customer demand.
Source: 1. FBI Financial Crimes Report 2010-2011
2. UN Telecom Agency Report, Oct 2012
4
Hadoop
Streaming
Data
New
Sources
Unstructured
Exploratory
Iterative
Structured Repeatable
Linear
Data
Warehouse
Traditional
Sources
Traditional Approach Structured, analytical, logical
New Approach Creative, holistic thought, intuition
Enterprise
Wide
Integration
Web logs, URLs
Social data
Text Data: emails, chats
RFID, sensor data
Network data
Internal App Data
Transaction Data
ERP data
Mainframe Data
OLTP System Data
The fact is analysis is expanding from enterprise data to big data,
creating new cost-effective opportunities for competitive advantage
5
And everyone can develop and leverage big data
…it’s not just for ‘data scientists’
External Data
Business Users
...offer personalized
price promotions to
different customer
segments in real-time
Business Development ... find and deliver new
mechanisms to monetize
network traffic and partner
with upstream content
providers
Administrators ...secure, manage, and optimize data
access and analysis operations
Business Executives ...get real-time reports and analysis
based on data inside as well as
outside the enterprise (web, social
media etc.)
Business Analysts ... analyze social media buzz
for the new services/offerings
to gauge initial success and
any course correction needed
Developers ... develop new Apps and
detailed algorithms in response
to user and business
requirements
Data Scientists ... analyze subscriber usage pattern
in real-time and combine that with the
profile for delivering promotional or
retention offers
Big data adoption
When segmented into four groups based on current levels of big data activity, respondents showed significant consistency in
organizational behaviors Total respondents n = 1061
Totals do not equal 100% due to rounding
A recent Institute for Business Value study highlights how organizations are adopting big data in four phases
Big Data Exploration Find, visualize, understand all big data to improve decision making
Enhanced 360o View of the Customer Extend existing customer views (MDM, CRM, etc) by incorporating additional internal and external information sources
Operations Analysis Analyze a variety of machine data for improved business results
Data Warehouse Augmentation Integrate big data and data warehouse capabilities to increase operational efficiency
Security/Intelligence Extension Lower risk, detect fraud and monitor cyber security in real-time
How Clients are Using Big Data: 5 Key Use Cases
Find, visualize and
understand all big data
to improve business
knowledge
• Greater efficiencies in
business processes
• New insights from
combining and analyzing
data types in new ways
• Develop new business
models with resulting
increased market
presence and revenue CM, RM, DM RDBMS Feeds Web 2.0 Email Web CRM, ERP File Systems
Connector Framework
App Builder
BigInsights
Integration & Governance
UI / User
Streams
Big Data Exploration - Illustrated
Warehouse Data Explorer
Enhanced 360º View of the Customer - Illustrated
Master Data Management
Unified View of Party’s Information
CRM
J Robertson
Pittsburgh, PA 15213
35 West 15th
Name:
Address:
Address:
ERP
Janet Robertson
Pittsburgh, PA 15213
35 West 15th St.
Name:
Address:
Address:
Legacy
Jan Robertson
Pittsburgh, PA 15213
36 West 15th St.
Name:
Address:
Address:
SOURCE SYSTEMS
Janet
35 West 15th St
Pittsburgh
Robertson
PA / 15213
F
48
1/4/64
First:
Last:
Address:
City:
State/Zip:
Gender:
Age:
DOB:
360 View of Party Identity
BigInsights Streams Warehouse
Unified View of Party’s Information
Security Info & Event
Management (SIEM)
Co
nn
ec
tors
Data
Warehous
e
Surveillance Monitoring
System
Criminal Information
Tracking System
Co
nn
ec
tors
Un
str
uctu
red
/Str
ea
min
g D
ata
Tra
ditio
nal S
tructu
red
Data
• Deep analytics
• Operational
analytics
• Large scale
structured data
management
Netw
ork
Tele
metr
y M
on
ito
rin
g
Ap
pli
an
ce (
Op
tio
nal)
InfoSphere
Streams
Real-time Ingest & Processing
• Video/audio
• Network
• Geospatial
• Predictive
Big Data Storage & Analytics
InfoSphere
BigInsights
• Text/entity analytics
• Data mining
• Machine learning
I2 Analyst’s
Notebook
Security/Intelligence Extension - Illustrated
Raw
Logs a
nd M
achin
e D
ata
Indexing, Search
Statistical Modeling
Root Cause Analysis
Federated Navigation
& Discovery
Real-time Analysis
Only store
what is needed
Operations Analysis - Illustrated
Machine Data Accelerator
Data Warehouse Augmentation: Illustrated
Pre-Processing Hub Query-able Archive Exploratory Analysis
Information Integration
Data Warehouse
Streams Real-time processing
BigInsights Landing zone
for all data
Data Warehouse
BigInsights
Can combine with unstructured
information
Data Warehouse
1 2 3
12
Find and view the data
Data Explorer
Data Explorer
BigInsights
Streams Offload analytics for microsecond
latency
13
Asian telco reduces
billing costs and
improves customer
satisfaction.
Capabilities:
Stream Computing
Analytic Accelerators
Real-time mediation and analysis of
8B CDRs per day
Data processing time reduced from
12 hrs to 1 min
Hardware cost reduced to 1/8th
Proactively address issues
(e.g. dropped calls) impacting customer
satisfaction.
14
Transaction
Systems
Web, e-business
and SOA
New Era Analytics
Pla
tfo
rms
1960-1980s 1990-2000s 2010s
Mainframe, IMS
and CICS
WebSphere Big Data
Platform
IBM is evolving a new type of IT platform centered on
delivering the value from big data
The Big Data Platform
Process any type of data
– Structured, unstructured, in-
motion, at-rest
Built-for-purpose engines
– Designed to handle different
requirements
Analyze data in motion
Manage and govern data in the
ecosystem
Enterprise data integration
Grow and evolve on current
infrastructure
15
Solutions
IBM Big Data Platform
Analytics and Decision Management
Big Data Infrastructure
Accelerators
Information Integration & Governance
Hadoop
System
Stream
Computing
Data
Warehouse
Systems
Management
Application
Development
Visualization
& Discovery
© 2013 IBM Corporation 16
Constant Contact Analyzes
Billions of Unstructured Emails
with InfoSphere BigInsights
Need
Customers send 35 billion e-mails yearly
• Ability to analyze e-mail effectiveness critical
to customer success
• Analyze trends
• Reduce time to send emails
Learn what content is most effective in campaign
Benefits
40X Performance improvement
• Analysis time reduced from hours to seconds
• Directly improves customers experience
• 15%-25% increase in campaign performance
Easy to use across the organization
Enhances existing analytics and information
systems (IBM big data platform)
17
We are moving in on closing the skills gap
www.ibmbigdatahub.com
18
And we are driving hundreds of new Analytics and Big
Data programs
Active Relationships with 200+ Universities Around the World – Building new programs and enhancing existing programs to train the skilled
professionals of the future in: •Analytics
•Big Data
•Business Intelligence and Analytics
•Predictive Analytics
•Computing and Informatics
•Math, Statistics
•Operations Research
•Data Science
Majority of new programs are Masters
level in Business Schools
19
CC&G Partners
And building an extensive business partner ecosystem
for Big Data and Analytics (Over 330 partners!)
With all this being said, how do you get started?
Educate: – Read ‘Harnessing the Power of Big Data’
– Read the IBV study on big data
– Visit the IBM Big Data Hub
• www.ibmbigdatahub.com
Explore:
– Determine which use case or entry point is right for you
Engage:
– Put the pressure on your IBM sales rep to show to you how
IBM is solving big data challenges around the globe
Execute:
– Determine where to start and build out an action plan
20
