big data? no. big decisions are what you want

Big Data? No. Big Decisions are What You Want.

Stuart Miniman

Wikibon, Senior Analyst

@stustu@wikibon.org

This presentation and more at http://wikibon.org/BigData

Big Questions

What is Big Data?

Evolution or Revolution of Business Intelligence (BI)?

Who is Using Big Data?

How Should Practitioners Proceed?

Massive Data Growth

Source: http://wikibon.org/blog/infographics/

Transforming Data Knowledge

BI as we know it has failed.

The Old WayCRM

ETLData Quality

Normalized DataTraditional Data Warehouse

ERP

Finance

Data

Data

Data Warehouse AdministratorBusiness Analyst

Business User

VOLUMETYPE

SPEED

BIG DATA

BIG DATA Process and Analyze ALL Your Data

Ask NEW Questions

Ask MORE Questions

Get Answers FASTERGet CLEARER Insight

MAKE BETTER BUSINESS DECISIONS

BIT FLIPSubsets

Historical

Structured (database)

Data growth as a burden & challenge

All Data

Near Real-time

Structured/Unstructured

Data as a new source of competitive opportunity

BI BIG DATA

Two NEW APPROACHES to BIG DATA

Hadoop is is open source framework for processing and analyzing massive amounts of distributed data.

Next Generation Data Warehouses use massively parallel processing, columnar architectures and data compression to analyze not-quite-so-massive data in close to real-time.

These two approaches overlap in some areas and compliment one another in other areas.

Next Generation Data Warehouses

Data Scientists10/90 rule for magnificent data success

http://www.kaushik.net/avinash/big-data-imperative-driving-big-action/

Over-invest in people, because without that investment big data will absolutely, positively, be a big disappointment for your company. Computers and artificial intelligence are simply not there yet. Hence your BFF is natural intelligence.

-Avinash Kaushik

BIG MONEY in BIG DATA

CAGR of 58%

Revenue mix today: 44% services, 31% hardware, 25% software

Recommendation EngineUse Hadoop to match and recommend users to one another or to products and services based on analysis of user profile and behavioral data

IT: BIaaS

Single Version of the

Truth

New ?sPredictions of Future Equipment Failures

Large Media Company: BIG DATA + WAN

Hadoop Clusters

DC17PB

Hadoop Clusters

DC27PB

10 Gbps Hadoop Traffic

Site 1: Advertising Analysis

Site 2: Content Customization

10 TB’s/day of source data: browsing pattern, click throughs, server logs

1 TB/day of results sent for integrated analysis with structured data

5 TB’s/day for inter-cluster sync

1 2 3

Customer Experience Analytics

Integrating data from previously siloed channels such as call center, online chat, Twitter, etc.

Source: Clickfox

New Revenue from Data

The Associated Press is combining a mix of decades of historical news releases with real-time additions to create new monetization opportunities for it’s data using a document-oriented database (rather than traditional relational database).

NYSE is delivering analytics on data that is seeing massive growth that adds up to Petabytes of information that can be offered as a cloud service to traders.

BIG DATA Infrastructure

Network optimization (low latency)

Share-nothing storage– Bring the computation to the data

Massive compute requirements– Emerging opportunity in the cloud

BIG DATA Organization

Broad cross-silo impact– Tight coordination needed between business

decision makers and technology/analyst

Organize for selling/buying data within organization and IT– Next-generation “chargeback”

What’s Your BIG DATA Strategy?

Enterprises should …

EVALUATE

ENGAGE

PLAN

CULTIVATE

EXECUTE

REPEAT

Creating a BIG DATA IT Plan

• Understand IO-centric technologies that allow near real-time big data processing

• Select key vendor partnerships• Start with small projects of integrated

design• Investigate opportunities to deliver big

data services for your industry

What’s Your BIG DATA Strategy?

Vendors should …

LISTEN

EDUCATE

INNOVATE

SELL

SUPPORT

REPEAT

Now Is the Era of BIG DATA

Special Thanks to David Floyer and Jeff Kelly

Big Data is the new definitive source of competitive advantage across all industries.

This presentation and more at http://wikibon.org/BigData