big data platform overview - meetupfiles.meetup.com/13471152/indianapolis bd developer day... ·...
TRANSCRIPT
Big Data Platform Overview
Alex Hay ([email protected]), Big Data CTPMeridee Lowry ([email protected]), Big Data CTP
April 30th, 2014
Big Data is a Concept
Big Data
2
IBM Big Data and Analytics Offerings
Watson Foundations
Information Integration & GovernanceINFORMATION SERVER, MDM, G2, GUARDIUM, OPTIM
Exploration, landing and
archive
Trusted data
Reporting & interactive analysis
Deep analytics & modeling
Real-time processing & analyticsSTREAMS, DATA REPLICATION
Operational systems
BIGINSIGHTS
PUREDATAHADOOP
DB2, INFORMIX
PUREDATA TRANSACTIONS
PUREDATA ANALYTICS
DB2 BLU
PUREDATA ANALYTICS
DB2 WAREHOUSE
PUREDATA OPERATIONAL
ANALYTICS
Actionable insight
Decision management
Predictive analytics and modeling
Reporting, analysis, content
analytics
Discovery and exploration
SPSS MODELER
COGNOS BICOGNOS TM1
WATSON EXPLORER
SPSS MODELER GOLD
Image and video
Data types
Transaction andapplication data
Machine andsensor data
Enterprise content
Social data
Third-party data
3
Online Retailer Scenarios
4
The Big Outdoors Company
Fictitious company
IBM-generated data for sales
Company stats
144 Products
3,500 Unstructured Data Items
5
Scenario Zero Demo
6
Scenario 1: Populate the Landing ZoneValue: Process and explore data more quickly
Information Movement, Matching & Transformation
Landing, Exploration& Archive
Data
Sources
Structured
Operational
Unstructured
External
Social
BigSheets
Actionable
Insights
Explore Landed
data using
BigSheets
7
Scenario One Demo
8
BigSheets - Spreadsheet Interface
9
Scenario 2: Perform Contextual DiscoveryValue: More quickly search all forms of data
Landing, Exploration& Archive
Data
Sources
Structured
Operational
Unstructured
External
Social
Actionable
Insights
Exploration &
Discovery
10
Bringing Big Data to the Point of Impact
Commenting
Rating
SharedFolders
Tagging
Social Tools
FileSystems
RelationalData
ContentManagement
CRM
SupplyChain
ERP
RSS Feeds
ExternalSources
Cloud
CustomSources
Application/Users
Wa
tso
n E
xp
lore
r
11
BIG DATA PLATFORM
Stream Computing
Hadoop System
Analytics – Clustering
12
Scenario 3: Perform Text Analytics & Merge DataValue: Report and Retain Lineage from Hadoop
Information Movement, Matching & Transformation
Landing, Exploration& Archive
Data
Sources
Structured
Operational
Unstructured
External
Social
Actionable
Insights
Apply Text
Analytics
Merge Structured
w/ Unstructured
Data
BI & Performance
Management
13
Scenario Three Demo
14
Text Analytics – Processing of Unstructured Data
How it works• Parses unstructured text and detects
meaning with annotators
• Understands the context in which the text is analyzed
• Hundreds of pre-built annotators for names, addresses, phone numbers, and others
Parts of speech support forp English, Spanish, French, German, Portuguese, Dutch, Japanese, Chinese
• Distills structured info from unstructured text
Benefits
• More precise and correct answers
• 50% faster than manual method
• Run faster text analysis
Football World Cup 2010, one team distinguished themselves well, losing to the eventual champions 1-0 in the Final. Early in the second half, Netherlands’
striker, Arjen Robben, had a breakaway, but the keeper for Spain, Iker Casillasmade the save. Winger Andres Iniesta
scored for Spain for the win.
Unstructured text (document, email, etc)
Classification and Insight
15
Scenario 4: Enhanced Product and Consumer Data Value: 360-degree view
Information Movement, Matching & Transformation
Landing, Exploration& Archive
PureData for
Operational
Analytics
Data
Sources
External
Social
BI & Performance
Management
Actionable
Insights
Apply Text
Analytics
with SDA
Enterprise Warehouse
Add value to
existing
Customer data
by applying
Social Profiles
16
Scenario Four Demo
17
Scenario 5: Better decision making using Predictive AnalyticsValue: Predict more often, more granular using more data
Landing, Exploration& Archive
Data
Sources
Structured
Operational
Unstructured
External
Social
Predictive Analytics
& Modeling
Actionable
Insights
18
PureData for
Operational
Analytics
Enterprise Warehouse
Scenario 6: Real-time data mining & Text Analytics Value: Improve Customer Satisfaction & Sales
Real-Time Analytics
Data
Sources
Structured
Operational
Unstructured
External
Social
Sensor
Geospatial
Time Series
Streaming Predictive Analytics
& Modeling
Actionable
Insights
Real-time
(Predictions)
Real-time Text
Analytics
19
Scenario 7: Encryption and Auditing of BigInsightsValue: Improve Data Security through Governance
Analytic Appliances
Security, Governance and Business Continuity
Landing, Exploration& Archive Enterprise
Warehouse
Data Marts
Data
Sources
Structured
Operational
Unstructured
Sensor
Geospatial
Time Series
BI & Performance
Management
Actionable
Insights
20
IBM Big Data and Analytics Offerings
Watson Foundations
Information Integration & GovernanceINFORMATION SERVER, MDM, G2, GUARDIUM, OPTIM
Exploration, landing and
archive
Trusted data
Reporting & interactive analysis
Deep analytics & modeling
Real-time processing & analyticsSTREAMS, DATA REPLICATION
Operational systems
BIGINSIGHTS
PUREDATAHADOOP
DB2, INFORMIX
PUREDATA TRANSACTIONS
PUREDATA ANALYTICS
DB2 BLU
PUREDATA ANALYTICS
DB2 WAREHOUSE
PUREDATA OPERATIONAL
ANALYTICS
Actionable insight
Decision management
Predictive analytics and modeling
Reporting, analysis, content
analytics
Discovery and exploration
SPSS MODELER
COGNOS BICOGNOS TM1
WATSON EXPLORER
SPSS MODELER GOLD
Image and video
Data types
Transaction andapplication data
Machine andsensor data
Enterprise content
Social data
Third-party data
21
Thank You!
22