enrich a 360-degree customer view with splunk and apache hadoop

© Hortonworks Inc. 2014

Enrich a 360-degree Customer View with Splunk® and Apache™ Hadoop®

Your Presenters

• Brett Sheppard (@zettaforce) – Director, Big Data, Splunk – Former analyst at Gartner and DoD (civilian

contractor), big data, enterprise architectures – Weekend volunteer with my dog in hospitals

• Bob Page (@bobpage) – VP Products, Hortonworks – Ran eBay’s data platform – Enjoys good wine

Today’s Topics

• Drivers for the Modern Data Architecture

• From Raw Data to Digital Intelligence

• Demo: 360-degree Customer View

• Q&A



Hadoop Adoption “Hadoop’s momentum is unstoppable as its open source roots grow wildly into enterprises. Its refreshingly unique approach to data management is transforming how companies store, process, analyze, and share big data.”

-- Mike Gualtieri, Principal Analyst, Forrester


A Traditional Approach Under Pressure AP

PLICAT

IONS

DATA

SYSTEM

REPOSITORIES

SOURC

ES

Exis4ng Sources (CRM, ERP, Clickstream, Logs)

RDBMS EDW MPP

Emerging Sources (Sensor, Sen4ment, Geo, Unstructured)

Business Analy4cs

Custom Applica4ons

Packaged Applica4ons

Source: IDC

2.8 ZB in 2012

85% from New Data Types

15x Machine Data by 2020 40 ZB by 2020


Emerging Modern Data Architecture AP

PLICAT

IONS

DATA

SYSTEM

REPOSITORIES

SOURC

ES


RDBMS EDW MPP


OPERATIONAL TOOLS

MANAGE & MONITOR

DEV & DATA TOOLS

BUILD & TEST

Business Analy4cs

Custom Applica4ons



The Common Journey with Hadoop SC

ALE

SCOPE

More data and analytic apps

MDA/Data Lake Cost, Insight IT Driven

New Analytic Apps New Types of Data LOB Driven

Unlock Value in New Types of Data 1.  Social

Understand how people are feeling and interacting – right now

2.  Clickstream Capture and analyze website visitors’ data trails and optimize your website

3.  Sensor/Machine Discover patterns in data streaming from remote sensors and machines

4.  Geographic Analyze location-based data to manage operations where they occur

5.  Historical Logs Diagnose process failures and prevent security breaches

6.  Unstructured (txt, video, pictures, etc..) Understand patterns in files across millions of web pages, emails, and documents

Value

+ Online archive Data that was once purged or moved to tape can be stored in Hadoop to discover long term trends and previously hidden value



Example Journey Towards a Data Lake D

ATA

VALUE

Risk Management E.g., Fraud Reduction

Operational Excellence E.g., Network Maintenance

New Business E.g., Data as a Product

Customer Intimacy E.g., 360 Degree View of

the Customer

TB’s

P

B

PB

’s

DATA LAKE An architectural shift in the data

center that uses Hadoop to deliver deep insight across a large, broad, diverse set of data at efficient scale

Data Lake


Enabling Hadoop for the Enterprise

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015

Capabilities Ensure enterprise capabilities are delivered in 100% open source to benefit all

1 2 Integration

Interoperable with existing data center investments

Skills Leverage your existing skills: development, analytics, operations 3



Deployment Model Provide the efficient deployment op4on for your organiza4on

Presenta4on & Applica4on Enable both exis4ng and new applica4ons to provide

value to the organiza4on

Opera4ons Empower Current opera4ons and security tools to manage Hadoop

Core Capabilities of Enterprise Hadoop

Data Governance Integrate with exis4ng systems and move data in/out and within the

environment

Security Provide layered approach to

security through Authen4ca4on, Authoriza4on,

Accountability and Data Protec4on

Opera4ons Allow you to deploy and

effec4vely manage the environment

BROAD INSIGHT Data Access

Access your data simultaneously in mul4ple ways (batch, interac4ve)

EFFICIENT SCALE Data Management

Store and process all of your Corporate Data Assets

1 Capabilities Ensure enterprise capabilities are delivered in 100% open source to benefit all


Enabling Familiar and Existing Tools

DEVE

LOPE

R AN

ALYST

OPE

RATO

R

COLLECT PROCESS BUILD

SEARCH ANALYSE VISUALIZE

PROVISION MANAGE MONITOR

1 2 Skills

Leverage your existing skills: development, analytics, operations

Integration Interoperable with existing data center investments 3



APPLICAT

IONS

DATA

SYSTEM

REPOSITORIES

SOURC

ES


RDBMS EDW MPP


OPERATIONAL TOOLS

MANAGE & MONITOR

DEV & DATA TOOLS

BUILD & TEST

Business Analy4cs

Custom Applica4ons


Requirements for Enterprise Hadoop

1 2 Skills

Leverage your existing skills: development, analytics, operations


Integrate with Applications Business Intelligence, Developer IDEs, Data Integration

Systems Data Systems & Storage, Systems Management

Platforms Operating Systems, Virtualization, Cloud, Appliances

Integration Interoperable with existing data center investments 3


Splunk + Hortonworks

Big Data Comes From Machines Volume | Velocity | Variety | Variability

GPS, RFID,

Hypervisor, Web Servers,

Email, Messaging Clickstreams, Mobile,

Telephony, IVR, Databases, Sensors, Telematics, Storage,

Servers, Security Devices, Desktops

Machine Data Contains Powerful Insights

Delivering the 360-‐Degree Customer View

"   Screen new account applicaAons "   Improve customer service experience "   Reduce customer churn "   Recommend next product to buy "   Localize and personalize promoAons "   Track markeAng channel effecAveness "   Empower omni-‐channel retailing

Synthesize data from all customer touch points – 360° view

Why this is hard: data lives in separate silos with incompatible formats

Big Data Doesn’t Have to Be a Science Project

Get Started with Hunk + Hortonworks in < 1 Hour

HDFS and MapReduce

Immediately start exploring, analyzing and visualizing raw data in Hadoop

2

3

Point Hunk at Hadoop Cluster

Explore Analyze Visualize Dashboards Share

Download Hortonworks Sandbox and Hunk

1

Challenges of AlternaAve Approaches

360-‐Degree Customer View

"   Screen new account applicaAons

"   Improve customer service experience

"   Reduce customer churn

"   Recommend next product to buy

"   Localize and personalize promoAons

"   Empower omni-‐channel retailing

Synthesize data from all customer touch points – 360° view

More Complete Customer View

Store data in

Hadoop: Apache web

logs, ecommerce site

activity, Akamai

image hosting logs,

Squid proxy logs

Analyze these

massive,

diverse data

sets in Hadoop Enhance structured

information with

analysis of this raw

Hadoop data for a

more complete view

of customer behavior

Deep Insight into Customer Behavior and SenAment

Rapidly interact with data

•  Powerful Search Processing Language (SPL™)

•  Ad hoc exploratory analyAcs across massive datasets

•  Preview results •  No fixed schema •  No requirement to “understand” data upfront Drill down

to raw data

Search interface

Pause or stop MapReduce jobs

Preview results

Search, Explore and Analyze

Inform, Upsell and Cross-‐sell

Measure a_enAon to specific content

Analyze click-‐through and how consumers navigate

Compare product bundle promoAons with mulAvariate tesAng

PrioriAze PromoAons with Customer Influence Scores

Understand Web OperaAons and AdverAsing

Weblog Traffic Data 750 million

Web User Clickstreams

12 million monthly visits

queries per month

Maintain high performance

Protect content against malicious bots

Track traffic sources for advertisers

Analyze Mobile App Performance and Usage •  Product adoption trend •  Users and clients •  Feature adoption •  User engagement •  Usage patterns •  Mobile devices •  Client dashboard

Easy and Fast to Get Started, Learn and Use

Configure Hortonworks Sandbox with Hunk: Splunk Analytics for Hadoop bit.ly/1cJqbCu

Configure Hunk with Hortonworks Sandbox 1.3

bit.ly/MYF36g

Hortonworks + Hunk = Business Value

“Splunk’s Hunk is perhaps the most promising technology to deliver a true interactive experience. Especially powerful are Splunk’s capabilities for

discovering the structure of machine data and other unstructured data on the fly.”

Question & Answer session will be conducted electronically, using the panel to the right of your screen

About Splunk and Hortonworks hortonworks.com/partner/splunk/

Get started with Hortonworks Sandbox hortonworks.com/hadoop-tutorial/splunk-hunk/

Follow us: @hortonworks @splunk

Get started with Hunk: Splunk Analytics for Hadoop splunk.com/download/hunk

Try Now

enrich a 360-degree customer view with splunk and apache hadoop

Technology