data disruption and innovation - radiant advisors

© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000

DATA DISRUPTION!AND INNOVATION"

1

Closing Keynote"DBTA Data Summit – May 13, 2015""

John O’Brien | Radiant Advisors " @obrienjw [email protected]"!

!

@obrienjw [email protected]


DIGITAL BUSINESS TRANSFORMATION"Data Disruption and Innovation"

The world around us "

has changed "

to consumer-centric and data-centric economy"

where companies "

must transform "

to remain competitive "

…and survive"

2


3 STEPS TO FOLLOW"Data Disruption and Innovation"

3

Business Disruption" Platform of Speed"Capability by Design"


THE POWER OF DISRUPTION"Business Disruption"

•! Understand digital life"

•! Understand new consumers"

•! Experiment and move fast"

•! Think like a startup"

•! Competitors are moving"

•! Innovate then optimize"

•! Practice Creative Destruction"

4


FAMILIAR STORIES?Business Disruption"

5


“Advantages”"• Ready to compete on analytic ability• Deeper understanding of your customers• Deeper understanding of your products usage

“Data Centric”"• Treat data as an asset – Treat data as oxygen• Willing to give services for free• Willing to buy data

DIFFERENT MINDSET NEEDEDBusiness Disruption"

6


CUSTOMER LIFETIME VALUE"Business Disruption"

7


BRAND-TO-CUSTOMER"Business Disruption"

8


PRODUCT RELATIONSHIPS"Business Disruption"

9


“Brand Analytics”• Sentiment analysis

• Partnerships

• Influence

“Customer Analytics”• Identify potential valuable customers

• Nurture to customers

• Engage lifelong customers

BRAND-TO-CUSTOMER FOCUS"Business Disruption"

10

Brand Loyalty

Becomes

Customer Loyalty


“Mobile”• Individually centered – Me, Here, Now

• Generation “C” Connected

• Opportunity for more data

“Engaged”• Engaging experiences

• Throughout their day

• Platform invitation

CONNECTED CUSTOMER"Business Disruption"

11


ALWAYS ON THE GO"Business Disruption"

12


OPPORTUNITY TO INFLUENCE"Business Disruption"

13

RT Gr8 #Breakfast at @FavCasino #GoBuffet #Reno

Gr8 #datasummit #keynote w/ @obrienjw #GoBigData

#Empowered


EXPERIENCES CAPTURED"Business Disruption"

14


BE A PART OF SHARING"Business Disruption"

15


WHAT’S NEXT?"Business Disruption""

16

• What would it take for you to have wearable technology?


CAPABILITY BY DESIGN"Data Disruption and Innovation"

•! Discovery is iterative"

•! Experiment / Fail Fast"

•! Exploratory; can be unclear"

•! New paths; new ideas"

•! Driven by unknowns"

•! Foundation for discovery"

17


FOUR ELEMENTS OF DESIGN"Capability by Design"

18

Self-Sufficiency"

Visualization"

Collaboration"

Mobile"


DESIGNED FOR SELF-SERVICE"Capability by Design"

•! Self-sufficiency"

•! From access to ability"

•! Partners in discovery"

•! Greater autonomy"

•! Remove barriers"

19


DESIGNED FOR COLLABORATION"Capability by Design"

•! Investment strategy"

•! Fosters collective innovation"

•! Closes the business/IT gap"

•! Establishes transparency"

•! Data governance and security"

20


DESIGNED FOR VISUALIZATION"Capability by Design"

•! Cognitive hardwiring"

•! Makes analytics approachable"

•! Understanding to insight"

•! Curated images and color"

•! Visual discovery"

21


DESIGNED FOR MOBILITY"Capability by Design"

•! More than consumption"

•! Visually interact with data"

•! Portable discovery"

•! Personalized experience"

22


BI Casual Users - Consumers "• Represent 80% of BI Users

• Reporting and Monitoring generations of BI

Power Users and Business Analysts – Data Provisioners "• Represent 20% of BI Users

• Analysis and Prediction generations of BI

Data Scientists and Analytic Modelers"• A few professionals in the organization•! Creators of statistical models"

INCREASING VALUE GENERATION"Capability by Design"


ITERATIVE “FRICTIONLESS” DISCOVERY"Capability by Design"

24

Get Data Access

Explore Data

Agilely Blend Data Discover

Collaborate 99x "

for each"Discovery"

“I need to work with new data”"“I want to check something”"

“Is this data usable?”"“What is this data?”"

“Lets mix with other data”"“Does this help my data set?”"

“Is this what I’m looking for?”"“This is interesting…”"

“What do you think of this?” “How can we use this?”

“That helped, need more.”"“Nope, keep looking…”"


“FRICTIONLESS” DATA INGESTION"Capability by Design"

1 hour steps"x 99"= 495 hours"

= 62 days"""

1 minute steps"x 99"= 495 minutes"

= 8.25 hours"""

25

Get Data Access

Explore Data

Agilely Blend Data Discover

Collaborate

Self-Sufficiency"


Start with business goal, then imagine the data needed

Get external public and purchased data – Big Data

• Facebook, Pintrest, Twitter, Weather, demographics

Create new ways to collect the data you want

• Mobile apps, wearables, sensors, “Internet of Things”

Current systems are the feedback loop

1. COLLECT DATA FROM EVERYWHERECapability by Design

26


Customer Insights

Product Insights

Optimize Innovate

2. ANALYTICS CULTURECapability by Design

27

Customers,Their NetworkTheir Influence

Business Functions &Processes

New and Disruptive

Business

InteractionsRelationships

Usage


• Ongoing A/B experiments

• Continuously improving analytic models

• Tracking prediction and recommendation engines

• Lean product development and interaction

3. ACT ON ANALYTICSCapability by Design

28


Governing People, Roles, Responsibilities Define roles and responsibilities for accessing and working with data Define ownership and delegation for semantic context discovered

Governing Discovery Results Governing semantic context after discovery

Operationalizing and monitoring analytic models

Modern Data Platforms Managing data abstraction and mixed workloads

Defining discovery or analytic sandboxes as dev, prod or both

Design for Capability!GOVERNED DATA DISCOVERY


PLATFORM OF SPEED"Data Disruption and Innovation"

•! Built to Change"

•! Polyglot Persistence approach"

•! Data unification is key"

•! Platform oriented and not a bunch of database silos"

•! Requires data governance"

•! Enabling platform for analytics and discovery"

!

30


Op

UX

RDBMS

eCom

UX

Doc

Mobile

UX

Graph

DW/DM

BI Apps

UX

MPP

BI Apps

UX

HDFS

YARN

UX

Appl

icatio

n Da

ta S

ilos

Polyglot Persistence Principle

Op

UX

RDBMS

eCom

MPP

Mobile

UX

Hadoop

Doc

Graph

Relational SQL NoSQL

Optimized Columnar In-Memory

MPP OLAP

Specialized Key-Value Document Distributed

Graph

Transactions Analytics Behavioral Sensors

POLYGLOT PERSISTENCE PRINCIPLE Platform for Speed


32

Platform for Speed!DATA TECHNOLOGIES"

Key Value Store (Hadoop) Discovery Oriented

Highest Scalability

Lowest Cost Schema-less

Without Context

Analytic Database

Technologies

EDW RDBMS

Accessibility: Programming -> SQL SQL, MDX, UDF SQL Access

Workload: Flexible, Scalable Analytic Optimized Reference Data Mgmt

Maturity: Emerging Accepted Mature


33

Platform for Speed!MODERN DATA PLATFORM"

Flexible Class"

Data Warehouses"

ç"

Analytic Optimized Class" Reference Class"

ç"

Master Reference"Data"

Discovery, Scalable, Programmable" Stable, Context, SQL"Discovery and Analytics Oriented"

Hortonworks"

Cloudera"

MapR"

Cassandra"

MongoDB"

ç"

askdjfl"kasjdfl"iuyuiio"

Highly Specialized for BI"

Highly Optimized for BI"

In-memory"

ç"Columnar" MOLAP"

Graphs"Document"

Stores"Text "

Analytics"

MPP"


Platform for Speed!ARCHITECTED FOR DISCOVERY"

ç ç ç ç

3rd Party

Web

Mobile Mobile

Digital Servers Digital Servers

BI Tool

BI Tool

Social Networks

Replication Technology

ç ç ç ç

ç ç ç ç

ç ç ç ç

Excel

T

Modern Data Platform

T Integration Discovery T Institutional Integrations

Cache

Website Activity

Mobile Activity

IoT Activity

Social Activity

Report User

Users

User

User data

Exte

rnal

Dat

a

Data Lake

T ç ç ç ç

SAS

Analytic Tool

Connect

Connect

VoC Command Ctr

SAS Server

Connect

Inte

rnal

Ope

ratio

nal S

yste

ms

Operational Reporting Tier

ç ç ç ç T

STG EDW

Analytics Optimized Tier

Big Data Discovery Tier

Managed File Transfers

Partner Partner

Beha

vior

al D

ata

Essbase

Analytics Database

ç Analytics ç Analytics Database ç Analytics ç Analytics Database Analytics Database Analytics Database Analytics Analytics Analytics ç Database Database Database Database Database ç

Analytics DB

ç Analytics ç DB Analytics Analytics

ç Analytics Analytics ç Analytics DB Analytics Analytics DB Analytics Analytics Analytics DB DB ç DB DB DB DB DB ç

Statistics

Social Media Aggregator

Data Acquisition Data Unification (SQL/Data Services)

ç ç ç ç

ç ç ç ç T ç

ç

T

ç ç ç ç

ç ç ç ç T ç

ç ç

T ç ç ç

Cache Cache Cache

Tableau Server

Tableau Server

Tableau Server W

Business Objects Business Objects Business Objects U

T

Tableau

Tableau Mobile

Tableau Server

Tableau Server

Tableau Server

ç ç

Governed Discovery

Proxy Server

T

T Local Discovery

Governed Acquisitions

Business Reporting

Business Discovery

Analysts

Social Response

Executive

Data Science

MPP Col


35

Platform for Speed!KEY SQL CONSIDERATIONS"

Evaluation Criteria"

SQL Capability"•!Tools Compatibility"•!ANSI SQL"•!Analytic SQL"•!User Defined Functions"

Scalability"•!How many nodes max?"•!All nodes in cluster?"•!Subset of cluster?"•!Data duplication?"

Speed"•!Response time"•!Ad-hoc workloads"•!Without caching"•!Concurrency"

Architecture"•!YARN compatible"•!Data file formats"•!Data Lake strategy"•!Semantic Layer"


*Vor

tex"

PIG"

Hive

-QL"

MapReduce"

""

Hadoop HDFS"Hadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFS

Hadoop v1"

Map

"Re

duce

"

PIG"

Hive

0.

13"

YARN"

""

Hadoop HDFS"Hadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFS

Hadoop v2"

PIG"

Hive"

*Vor

tex

M/R"

YARN"

""

Hadoop HDFS"Hadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFS

Impa

la, H

AWQ

, Pr

esto"

MPP

Eng

ines"

Hadoop HDFSHadoop HDFS

Platform for Speed!EVOLVING ARCHITECTURE FOR SQL"

Tez"

Tez"

Batch-oriented SQL" Interactive SQL" Architectural SQL"

Hadoop v2"with more SQL options"


Flexibility Class"

37

Enterprise"Data

Warehouses"

ç"

ç"Master

Reference"Data"

Discovery, Scalable, Programs" Stable, Context, SQL"Discovery & Analytics Oriented"

Apache Hadoop "

ç"

Highly Optimized for Analytics"

In-memory" MOLAP"MPP"

Optimized Class" Reference Class"

Gen

erat

e"

Hive

SQ

L"

askdjfl"kasjdfl"iuyuiio"

Highly Specialized for Analytics"

Graphs"Document"

Stores"Text "

Analytics"

P

IG /

Hive"

Map

Red

uce"

Ope

ratio

nal S

yste

ms,

Big

Dat

a, S

tream

s"

HDFS

"

ç"Columnar"

Extending SQL Access to Big Data and Hadoop via Hive and other HDFS SQL engines"

Platform for Speed!MODERN DATA PLATFORM UNIFIED SQL"


Business Demands! Hadoop Generation!

Internet and e-Commerce companies needed to cost-e"ectively work with big data !(both operationally and analytically)!

Hadoop v1 inspired by Google papers and from Apache projects delivered a"ordable scalability and a parallel programming framework!

Business needed a more interactive environment for users and apps with improved resource management and multi-tenancy!

Hadoop v2 improved the two-tier architecture with YARN for managing resources and reorganized for YARN applications !

HADOOP EVOLVES FOR BUSINESS"Platform for Speed"


Business Demand! Hadoop Generations!

E#ciently integrating big data, BI and operational data apps in a singular data architecture meeting data management requirements!

Hadoop v2x focus on improvements in operational capabilities: SQL, performance, security, governance while Data Lake provides a strategy!

Long term “data operating system” architecture for e!ciently and e"ectively manage data workloads, reusability, and leveraging cloud#

Future Hadoop will focus on IT ecosystem integration, more Hadoop applications with improved distributed file systems#

HADOOP EVOLVES FOR BUSINESS"Platform for Speed"


YARN Applications M

apRe

duce

(Bat

ch)

UX

Hive

(SQ

L)

UX O

nline

(HBa

se)

UX

Data

Ope

ratin

g Sy

stem

=== === === === ===

=== === === === ===

=== === === === ===

=== === === === ===

HDFS

Sear

ch

(Sol

r)

Tez

(Inte

ract

ive)

Stre

aming

(S

torm

)

In-m

emor

y (S

park

)

ISV

Engi

nes

Data Engines on YARN

Hadoop Engines

ISV Engines

Future Engines

=== === === === ===

=== === === === ===

=== === === === ===

=== === === === ===

Optimized Distributed File Systems

Apps Mobile Machine

Cloud On-Premises

Hybrid Inter-Cloud

Data

Reu

sabi

lity

Avoi

d Da

ta D

uplic

atio

n Ce

ntra

lize

Info

Mgm

t

FUTURE DATA OPERATING SYSTEM Data Disruption and Innovation


3 STEPS TO FOLLOW"Data Disruption and Innovation"

41

Business Disruption" Platform of Speed"Capability by Design"


THANK YOU!"

For more information""www.RadiantAdvisors.com ""Twitter: " "@RadiantAdvisors "

"Email: " "[email protected]"LinkedIn: " "www.linkedin.com/company/radiant-advisors "

42

data disruption and innovation - radiant advisors

Internet

rights reserved v1

customer business disruption

sharing business disruption

design data disruption

john obrien radiant

datacentric economy

asset treat data

data different mindset