big data big expectations - dennis faucher

8/13/2019 Big Data Big Expectations - Dennis Faucher

1/18

But First, Something Fun

1. Pull Out Your Phone

2. Open Your Texting App

3. Prepare to Send a Text to 22333

4. Here are the Possible Text Responses 223-33
http://www.polleverywhere.com/multiple_choice_polls/xNwWZxlNGRbxN2Whttp://www.polleverywhere.com/multiple_choice_polls/xNwWZxlNGRbxN2W


2/18

Big Data - Big Expectations


3/18

Todays Agenda

Big Reality: Real Solutions

Big Data Framework

Real Customer Examples

Why

How

What


4/18

What Makes it Big Data?

VOLUME VELOCITY VARIETYVALU

VERAC

SOCIAL

BLOG

SMART

METER

10110010

00100110

10101110

01010010

Data that cannot be turned into business value fast eno


5/18

Where Does Big Data Come From?

Structured: aka "Process-mediated": Examples: ERP, CRM , POS

Characteristics: transactional, referential, relational, Traditio

managed

Semi-Structured: aka "Machine-generated Examples: XML, JSON, Network Logs, Sensor Data

Characteristics: Well suited to computer processing but ma

Volume and accumulation, often too large for EDW

Unstructured: aka "Human-sourced": Examples: CDR, Doctor's Notes, Social Media, Audio, Vide

fields

Characteristics: subjective record of personal experiences,

required to realize value

Process Mediated

Data

This is yourdata.This is placeholdertext

forwhateverbest representsthe structured and unstructured

data you want to query. tis, justo epellentesquemetus, et sollicitudin

diam lectus eu sapien. Cum sociisnatoque penatibus et magnis disparturient montes, nascetur ridiculu

s Do40% more ofthis rhon muscus aliquam, massa mauris por

ta tortor, vel porta odio ligula aelit. Sed laoreet, lectus ac dapibusplacerat, nulla turpis lacin

ia lectus, eget hendrerit nibh mauris vel ligula. Sed a

nisl dolor, a tincidunt leo. ab

Machine Generated

Data

Human Sourced

Data


6/18

Where is the Value?

Gain Market Share (Social Scrapes)

Identify Cost Savings (Streaming Fraud Detection, Doctors Notes)

Customer Retention (Call Detail Records) Real Time Business Automation (High Freq. Trading, Smart Utility

Business Process Re-invention (Personalized Risk vs Statistical Av

Modernization and Competitive Technological Advantage


7/18

How Does it Get to the User?

Hadoop

(Traditional)

EDW

NoSQL

Analytics

Mart

In-Memory

Process Mediated

Data

DataVirtualization

BI/Analytics

This isyour data.Thisisplaceholdertext

forwhatever bestrepresents

thest ructuredandunst ructured

datayouwant to query. t is, justo e

pellentesquemetus, etsollicitudin

diamlectus eusapien. Cumsociis

natoquepenatibus etmagnis dis

parturientmontes,nascetur ridiculu

s Do40%moreofthis rhonmuscus aliquam,massamauris por

tatortor,velportaodio l igulaa

elit.Sedlaoreet, lectus ac dapibus

placerat,nulla turpis lacin

ialectus, egethendrerit n ibhmauris velligula. Sed a

nisldolor, atincidunt leo.ab

Machine Generated

Data

Human Sourced

Data

Da

taIntegration

Disparate, non-performing Integrated, performance-optimized


8/18

Where Do We Put It?

(Traditional)

EDW

Analytics

MartHadoopNoSQL

Human

Information

Java /Open Source VeSQL, Windows, Linux


9/18

A Few Customer Examples


10/18

Volume - Financial Analysis

1T Rows


11/18

Executive Management /

Plant Management

ISA-95 Asset Hierarchical Data ModelMaintenance

Reliability

Operations

Engineering

Manual In

Capital Proj

Supply Ch

Finance

Health Safety &Environment

Outputs:Hundreds Standard KPIsReporting with Drill Down

Root Cause AnalysisRole Based DashboardsManagement of Change

Multiple Site Role upCross Functional

Collaboration Platform

SAPJDEOracle

Aspen EDOScheduling

PrimaveraMS ProjectSharepoint

IntergraphAspenTechAutoCADBentleyAveva

MeridiumCapstoneDB / Excel

IP21OSI PiHoneywellYokogawa

Siemens

MaximoSAPPMOraclePassportDataStream

DBExcel

SAPESS

Variety & Veracity

V i t & V it


12/18

Variety & Veracity


13/18

Using ALL the DataVolume & VarietyeBay

I was thinking if we created a n

engine with added functionality

me find that obscure R2-D2 ac

faster and easier?

Bob, what if we could answer

these questions What didsomeone buy?, What did they

bid on it? Its also Where were

they at the time? Its also Who

influenced them within their

social circle? All that data is

amassed. cool idea?

We can build a Big Datas infrastructure challen

Hadoop cluster consisted of 400 nodes and Two

this will allow us to bring the processing to where

the need for time-consuming data t

Larger Index than Voyage

More descriptions, history & metada

100 engineers, all new codebase,


14/18

Structured Data

VarietyFinancial Structured and Unstructured Data

Select Customers with < 150K in Assets pull

demographics

From a database get me all matches from the CRM and Call Detail Records that match the query

From unstructured sources get me all matches for calls, chat, email that were positive for the structured results

Unstruct

Reference check 21 image DB

Clickstream data from banking w

Meaning based positive commen

Columnar

Pull < total 30% of net worth from Check 21 Image database

Customers who conducted net worth

report from our banking web site

10,015,664,356,165 rows (10 trillion)22B-43B rows loaded daily

20node x86 cluster

1.794petabytes raw data

7:1 compression


15/18

Meter Data Management (MDM) System that 1) Customer segmentation 2) Anomalyvolume

Velocity - Smart Meters


16/18

Can we find any relationship between FDA recall actions & conversations within th

universe?

Variety - Public Health

48 million people get

128,000 are hospitaliz

and 3000 die each yefoodborne diseases

In 2009

Peanut butterrecall costproducers

$1,000,000,000

How to avoid unintended economic consequences

actions?

What where people talking about two weeks before the

How did they feel?

Where were they?

What were they eating?


17/18

How Do I Start?

Data Analytics Gap Analysis

Vendor Agnostic Assessment of

Current State and Go-Forward Strategy

Benchmark your Data ManagementInfrastructure against industry

Identify Critical Business andTechnology Gaps

Matrix/SWOT of product capabilitiesand environmental applicability

Strategic and Technical Execution andDeployment Planning

Proof of Value

Business Value Assess

Technical Assess. of ta

System Design and Arc

POC Installation/configu

Integration Testing

Performance Benchmar

System Tuning/Optimiz

Performance Evaluation

ROI Assessment


18/18

Thank You

big data big expectations - dennis faucher

Documents