big data & analytics – beyond hadoop

19
© 2014 IBM Corporation Big Data & Analytics – beyond Hadoop Ian Radmore, IBM UKI Big Data Specialist June 18th, 2014

Upload: ibm-big-data-and-analytics-uk

Post on 12-Jan-2015

278 views

Category:

Data & Analytics


0 download

DESCRIPTION

Ian Radmore, an IBM Big Data Specialist spoke about the velocity aspect of the 4Vs associated with Big Data at the recent Internet World conference, this is the supporting presentation.

TRANSCRIPT

Page 1: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

Big Data & Analytics – beyond Hadoop

Ian Radmore, IBM UKI Big Data Specialist

June 18th, 2014

Page 2: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

Data: To have and to hold? Or to Analyse and Act!

Data in

Data at

2

Page 3: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

The auto industry is already the 2nd largest data generator AND 20% CAGR!

Ford Fusion: 145 actuators, 4700 relays and 70 sensors, including radar, sonar, accelerometer, camera, rain sensors. Collectively, these devices generate more than 25 gigabytes of data per hour, which is processed by more than 70 on-board computers.1 car year = 1TB

3

Page 4: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

A Big Data & Analytics approach helps provide a foundation for a Smarter Enterprise

Invest in a Invest in a big data & analytics big data & analytics

platformplatform

Be proactive about Be proactive about privacy, security and privacy, security and

governancegovernance

Imagine It. Realise It. Trust It.

Build a culture that Build a culture that infuses analytics infuses analytics

everywhereeverywhere

Confidence in Your Data

Confidence in Accelerating Value

Confidence in Your Skills

4

Page 5: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

� Deployed real-time CDR analysis solution to handle exploding data volume growth and performance requirements

� Analyzes call, internet usage, and text records in real-time to identify and address poorly performing cells

� Uses InfoSphere Streams and IBM Netezza

� Significant Benefits:

� Over 90% reduction in time to merge/load call record data

� Over 90% reduction in storage

� Increased network quality, improved customer satisfaction,

reduced churn

Sprint Increases Revenue & Improves Customer Satisfaction

“Over 90+% reduction in merge/load times and storage requirements”

“Over 90+% reduction in merge/load times and storage requirements”

Capabilities Utilised:• Stream processing

• Data Warehouse Analytics Appliance

5

Page 6: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

• Examines trends, volume, and content of millions of public Twitter messages in real-time

• Analytic accelerators to understand sentiment (positive, negative, neutral)

• Capabilities

• Stream Computing

• Visualization

• Benefits

• Real-time display of public sentiment as candidates respond to questions

• Debate winner prediction based on public opinion instead of solely political analysts

University of Southern California Innovation Lab Monitors Political

Debates

Solution to measure public sentiment during key

primary & general presidential debates

6

Page 7: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

7

KTH Swedish Royal Institute of Technology Reducing Traffic

Congestion

• Deployed real-time Smarter Traffic system to predict and improve traffic flow.

• Analyzes streaming real-time data gathered from cameras at entry/exit to city, GPS data from taxis and trucks, and weather information.

• Predicts best time and method to travel such as when to leave to catch a flight at the airport

Significant benefits:

• Enables ability to analyze and predict traffic faster and more accurately than ever before

• Provides new insight into mechanisms that affect a complex traffic system

• Smarter, more efficient, and more environmentally friendly traffic

7

Capabilities Utilised:

Stream Computing

7

Page 8: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

Pacific Northwest Smart Grid Demonstration Project

Capabilities:

Stream Computing – real-time control system

Data Warehouse Appliance – analyze massive data sets

Demonstrates scalability from 100 to 500K homes while retaining 10 years’historical data

60k metered customers in 5 states

Accommodates ad hoc analysis of price fluctuation, energy consumption profiles, risk, fraud detection, grid health, etc.

8

Page 9: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

Information Integration & Governance

Systems Security

On premise, Cloud, As a serviceStorage

IBM Watson Foundations

IBM Big Data & Analytics Infrastructure

New /

Enhanced ApplicationsAll Data

What action should I take?

Decision management

CognitiveWhat didI learn?

Landing,

Exploration and

Archive data

zone

EDW and

data mart

zone

Operational

data zone

Real-time Data Processing & Analytics What is happening?

Discovery and exploration

Why did it happen?

Reporting, content and analysisWhat could

happen?

Predictive analytics and modelling

Deep

Analytics

data zone

8

Realise It. Invest

Page 10: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

Realise It. In-Store Presence Zones

Intelligent location-based technology to gain deep insight into customer in-store behaviour

Enable retailers to integrate the physical and digital experience to facilitate an ongoing dialogue that creates loyalty via an exceptional in-store shopping experience

Presence Zones Sensors

9

Page 11: Big Data & Analytics – beyond Hadoop

© 2014 IBM CorporationIBM Internal Use Only

Realise It. The Customer Insight Appliance

10

\

\

\\

\\

\\

Page 12: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

Realise It. A Multichannel Korean retailer

Reliable insight

provides decision support for senior management

Targeted campaignscan be developed for marketing

Precise measurementof cross-channel campaigns

Business Challenge: As sales increased for this retailer’s online shopping mall,

management experienced increasing difficulty ensuring that an appropriate product mix was being presented to its customers.

The Solution: The company adopted sophisticated analytics and marketing automation to understand, predict and act on consumer buying behavior with confidence. Real-time

marketing automation delivers personalised content to each shopper, triggered by their interaction history. Delivered at the right place and time, these offers can move the

shopper toward a sale and even increase the size of the purchase.

“We have greatly improved our understanding of our customers, which is helping us to

make smarter decisions that significantly improve business performance.”—Spokesperson, multichannel Korean retailer

Combining marketing automation with analytics to personalisecommunications and optimise offerings

11

Page 13: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

Millions of events per

second

Microsecond Latency

Traditional / Non-traditional data sources

Real time delivery

PowerfulAnalytics

Algorithmic Trading

Telco ChurnPrediction

SmartGrid

CyberSecurity Government /

Law enforcement

ICUMonitoring

EnvironmentMonitoringValue

Clear business goals

Business change driven outcomes

VolumeTerabytes/second

Petabytes/day

VarietyAll kinds of data

All kinds of analytics

VelocityDecisions in microseconds

Massively scalable

VeracityScreening, validation & certification of data

Example Streaming Data Sources:

Video, Audio, Networks, Social Media, Sensor, Weather

Realise It. IBM InfoSphere Streams:

Real-Time Adaptive Analytics for Big Data In-Motion

Connected Car

113

Page 14: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

Create foundation of trusted data

Understand usage and monitor compliance

Model exposure and understand variability

Trust the factsTrust the facts Ensure privacy Ensure privacy and securityand security

Make risk Make risk aware decisions aware decisions

Trust It. Be proactive about privacy, security and governance.

14

Page 15: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

Big Data Uses Cases Delivered with Unique IBM Capabilities

Unique IBM Capabilities:

1. In-memory computing with BLU

Acceleration

2. Data privacy and security of big

data

3. Data Discovery and Exploration

4. Building Confidence in Big Data

with Information Governance

5. Stream computing

WATSON FOUNDATIONS

DecisionManagement

Planning &Forecasting

Discovery &Exploration

Business Intelligence & Predictive Analytics

ContentAnalytics

Information Integration & Governance

Data Mgmt &Warehouse

HadoopSystem

StreamComputing

Content Management

WATSON FOUNDATIONS

DecisionManagement

Planning &Forecasting

Discovery &Exploration

Business Intelligence & Predictive AnalyticsBusiness Intelligence & Predictive Analytics

ContentAnalytics

Information Integration & Governance

Data Mgmt &Warehouse

HadoopSystem

StreamComputing

Content Management

Real-time traffic flow optimisation

Low-latencynetwork analysis

Fraud & risk detection

Predictive asset maintenance

Understand and act on customer sentiment

Predict and act on intent to purchase

15

Page 16: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation16

Page 17: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

http://www.youtube.com/watch?v=FGp-h-x0Hss

17

Page 18: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

Building a real-time enterprise is a journey, which depends on a solid Big Data & Analytics

foundation for success

Be proactive

about privacy,

security and

governance

Build a culture

that infuses

analytics

everywhere

Invest in a

big data &

analytics

platform

Imagine It. Realise It. Trust It.

18

Page 19: Big Data & Analytics – beyond Hadoop

© 2014 IBM Corporation

Ian RadmoreIBM Big Data Specialist,

UK & Ireland

IBM United Kingdom Limited

City Gate West

Toll House Hill

Nottingham

NG1 5FN

Mobile +44 7843 [email protected]

19