lynn wong: make a difference with big data - hp

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Make a difference with Big Data

Lynn Wong

APJ Regional Business Lead

Big Data Category

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP CONFIDENTIAL 2

Top Technology Trends for 2014

• Emergence of the Mobile Cloud Mobile distributed computing paradigm will lead to explosion of new services.

• From Internet of Things to Web of Things Need connectivity, internetworking to link physical and digital.

• From Big Data to Extreme Data Simpler analytics tools needed to leverage the data deluge.

• The Revolution Will Be 3D New tools, techniques bring 3D printing power to masses.

• Supporting New Learning Styles Online courses demand seamless, ubiquitous approach.

• Next-generation mobile networks Mobile infrastructure must catch up with user needs.

• Balancing Identity and Privacy Growing risks and concerns about social networks.

• Smart and Connected Healthcare Intelligent systems, assistive devices will improve health.

• E-Government Interoperability a big challenge to delivering information.

• Scientific Cloud Computing Key to solving grand challenges, pursuing breakthroughs.


Big Data Market Dynamics


Deliver results with Analytics

Capitalizing on Big Data


changing the art of management”

“


Don’t CxO’s already have reports and warehouses ?

So what’s new?

Volume Velocity Variety

Vulnerability


How do you benefit from the 2.5 exabytes humanity creates every day?

Volume

“We don’t have better algorithms, we just have more data”

Peter Norvig, Director of Research, Google


Variety

Business Data

10% of Information

Annual Growth

~100%

~10%


Variety

Human Information Machine Data

Business Data

10% of Information

90% of Information

Annual Growth

~100%

~10%


Velocity

Anticipate energy needed

No time for an overnight batch run!

Estimate flight arrivals

Detect card fraud

Predict equipment failure

Repel criminal attacks

Find the best markets

Beat the crowds


Then there’s “V” for vulnerability

Data theft Legal search, fraud

100+ million credit cards stolen – after 5 audits declared all “okay”

70%+ of the cost of litigation is classification


Accelerating Innovation & Time to Value

695,000 status updates

98,000+ tweets

698,445 Google searches

1,820TB of data created

11million instant messages

168 million+ emails sent

YouTube

Viber

Qzone

Amazon Web Services

GoGrid

Rackspace

LimeLight

Jive Software

salesforce.com

Xactly

Paint.NET

Business

Education

Entertainment

Games

Lifestyle

Music

Navigation

News

Photo & Video

Productivity

Reference

Social Networking

Sport

Travel

Utilities

Workbrain

SuccessFactors

Taleo

Workday

Finance

box.net

Facebook

LinkedIn

TripIt

Pinterest

Zynga

Zynga

Baidu

Twitter

Twitter

Yammer

Atlassian

Atlassian

MobilieIron SmugMug

SmugMug

Atlassian

Amazon

Amazon iHandy

PingMe

PingMe

Associatedcontent

Flickr

Snapfish

Answers.com

Tumblr.

Urban

Scribd. Pandora

MobileFrame.com

Mixi

CYworld

Renren

Xing

Yandex

Yandex

Heroku

RightScale

New Relic

AppFog

Bromium Splunk

CloudSigma

cloudability

kaggle

nebula

Parse

ScaleXtreme

SolidFire

Zillabyte

dotCloud

BeyondCore

Mozy

Fring Toggl

MailChimp

Hootsuite

Foursquare

buzzd

Dragon Diction

SuperCam

UPS Mobile

Fed Ex Mobile

Scanner Pro

DocuSign

HP ePrint

iSchedule

Khan Academy

BrainPOP

myHomework

Cookie Doodle

Ah! Fasion Girl

PaperHost

SLI Systems

NetSuite

OpSource

Joyent

Hosting.com

Tata Communications

Datapipe

PPM

Alterian

Hyland

NetDocuments

NetReach

OpenText

Xerox

Google

Microsoft

IntraLinks

Qvidian

Sage

SugarCRM

Volusion

Zoho

Adobe

Avid

Corel

Microsoft

Serif

Yahoo

CyberShift

Saba

Softscape

Sonar6

Ariba

Yahoo!

Quadrem

Elemica

Kinaxis

CCC

DCC

SCM ADP VirtualEdge

Cornerstone onDemand

CyberShift

Kenexa Saba

Softscape

Sonar6

Workscape

Exact Online

FinancialForce.com

Intacct NetSuite

Plex Systems

Quickbooks

eBay

MRM

Claim Processing

Payroll

Sales tracking & Marketing

Commissions Database

ERP

CRM

SCM

HCM

HCM

PLM

HP

EMC

Cost Management

Order Entry

Product Configurator

Bills of Material Engineering

Inventory

Manufacturing Projects

Quality Control

SAP

Cash Management

Accounts Receivable Fixed Assets Costing

Billing

Time and Expense

Activity Management Training

Time & Attendance

Rostering

Service

Data Warehousing

The Internet Gigabytes

Client/Server Megabytes

Every 60 seconds

IBM

Unisys

Burroughs

Hitachi

NEC Bull

Fijitsu

Mainframe Kilobytes

Mobile, Social, Big Data & The Cloud

Zettabytes

217 new mobile web users

Yottabytes


Big Data: Where is coming from Today based on transactions

Business Model

Busi

ness

D

ecis

ions

Business Intelligence

Data Model

Web pages Blogs

--------- ---------

-------

--------- ---------

Tomorrow better decision

Transaction Collection

Transactional Data

$ € ¥

Email

Social Media

Images

Search Engine Audio Video

Instant messaging

IM


Type Pressures Example Big shift

Structured Inefficient scaling Reporting From rows to columnar

Semi-structured

Trillions of events Security logs See, understand, act

Human Video, audio, images Call center recordings

Meaning based computing

Streaming Real-time patterns & prediction

Online sales All three plus real-time

Changing paradigms of data processing


Market dynamics of Big Data and Analytics

Data explosion • Human and machine information growth can’t be contained

• Data growth over past 5-10 years surpasses all data from last 5 decades

Evolving delivery models • Every phone call, tweet, Facebook update, and e-mail

generates new information

• Consumerization of IT blurs line between employees and consumers

Real time decision making • Organizations can’t react and respond quickly enough

• Only 20% of CIOs have real-time access to customer data


Datasets whose volume, velocity, variety and complexity exceed ability of commonly used software tools to capture, process, store, manage, and analyze them.

*Gartner, Inc., “Big Data” Is Only the Beginning of Extreme Information Management”, Mark A. Beyer, Anne Lapkin, Nicholas Gall, Donald Feinberg, Valentin T. Sribar, Published 7 April 2011

What exactly is Big Data?

Information Sources

Mobile Transactional Data Search Texts CRM, SCM, ERP

$ € ¥

Images Email Social Media IT Ops Audio Video

Velocity Volume

Variety Complexity

Big Data

• Both structured and unstructured data present new opportunities for buyers, service providers, researchers and consumers as a means for better understanding the world.


Data “Lake”

Imagine if you were able to…

Transactional data

Ingest Understand Analyse

Social media Images Audio Video Mobile Email Texts Documents

ERP Business intelligence … Your apps

… Analytics …

Machine Data Business Data Human Information

In-memory Hadoop

Standard APIs & tools

100% of your data Seamlessly

Interact with and process

…

Open connectors


Virtualization Cloud

SAP HANA Big Data Gartner Hype Cycle

Technology Adoption Cycle

HP Big Data solutions are poised to cross “the chasm” Mapping Converged Systems to the hype cycle


Customers plan to invest in Big Data 40% CAGR – The “Moneyball” Effect


Big Data Impacts all of us


Big Data Innovation changes the game of IT operation


HP HAVEn


HP HAVEn

HAVEn

Social media IT/OT Images Audio Video Transactional

data Mobile Search engine Email Texts

Catalogue massive volumes of distributed data

Hadoop/ HDFS

Process and index all information

Autonomy IDOL

Analyze at extreme scale in real-time

Vertica

Collect & unify machine data

Enterprise Security

Powering HP Software + your apps

nApps

Documents

Approaching Big Data with an intelligent software architecture


Horizontal Use Cases

n-Apps Big Data opportunity Big data use cases are business-driven and cut across a wide range of industries & functions

Sources: IDC: 2012 “Worldwide Big Data Technology and Services Forecast: 2011-2015, Gartner: 2012 “Big Data Drives Rapid Changes in Infrastructure and $232 Billion in IT Spending Through 2016

Healthcare • Drug development • Scientific research • Evidence based

medicine • Healthcare outcomes

analysis

Manufacturing • Supply chain

optimization • Defect tracking • RFID Correlation • Warranty management

Telecom • Broadcast monitoring • Churn prevention • Advertising

optimization

Government • Law enforcement • Counter terrorism • Traffic flow

optimization

Finance • Fraud detection • Anti-money laundering • Risk management

Energy • Weather forecasting • Natural resource

exploration

• Sentiment analysis • Security analytics • Social CRM / network analysis • Churn mitigation • Brand monitoring • Cross and Up sell • Loyalty & promotion analysis • Web application optimization

• Operations analytics • Marketing campaign optimization • Brand management • Social media analytics • Pricing optimization • Internal risk assessment • Customer behavior analysis • Revenue assurance

• Logistics optimization • Clickstream analysis • Influencer analysis • IT infrastructure analysis • Legal discovery • Equipment monitoring • Enterprise search


• Finance • Product managers • Marketing • Sales • Supply chain • Security • IT

Every customer role will benefit from Big Data

1IDC, The Digital Universe Decade , May 2010. IDG Research Services, IT Executives Vision, conducted for HP, for 2020, May 2012


Only HAVEn delivers

100% of your data 1000x faster answers

50-90% less hardware

1,000,000+ machine events per second

• Create better products/services • Improve customer experience • Defeat cybercriminals

• Protect your IT investments • Protect your information • Protect your applications

30x data per server

• Machine data • Business data • Human information

700+ connectors


Start by asking questions about business objectives

Growth

Resource planning

Cost reduction

Operational efficiencies

New business models

Risk reduction

Talent management

Customer satisfaction

Planning

• How would the ability to ingest and react to data in real-time help you?

• What “human information” would you like to better leverage?

• What’s inhibiting you from effectively combining machine, structured, semi-structured and streaming data today?

• How would theft of your data impact your operations?

• What would it cost to manage 100x the amount of data you retain today?

And drill into how data matters in each case


The lines between traditional OLTP, OLAP and Analytics workloads have blurred

The Changing Nature of Big Data systems

Sales

Finance

Marketing

Operations

CRM

HR

Sales Targets, Finance Targets,

etc.

Transactional Data ETL and Information Management

Analytics and Reports/Dashboards

Data extract, cleanse, integrate

Homogenized grain, time relationships, dimensionality

Business calculations, Summarization, hierarchies

Data warehouse Analytics cubes Reporting, analysis & visualization tools


Shortcomings of legacy techniques for big data

Stale technologies Talent shortage

³Source: Coleman Parkes Survey Nov 2012 IT frustration

Lack of insight

… preventing organizations from:

•Developing disruptive business models

•Creating better products and services

•Enhancing customer experience

•Driving sustained competitive advantage

Proprietary vs Open Monolithic vs Scalable

Batch vs Real-time

Summary vs Granular


HAVEn


HAVEn - Ecosystem

Cloud

Integrators Converged Infrastructure

Enterprise Services

Business Intelligence Partners

Data Partners

Social media IT/OT Images Audio Video Transactional

data Mobile Search engine Email Texts Documents

HAVEn

Resellers

hp.com/haven

Technology Services

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP CONFIDENTIAL 33 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 33

Call center

Suppliers

Point of sale

Machine data

Customer sentiment (emails, social media, forums)

Data sources Legacy data warehouse

Bringing big data analytics to legacy data warehouses and dark data

Use case focus – EDW and Hadoop augmentation

Operational analytics

Reporting and analytics

Dashboards

Lines of business Finance

Human resources

Sales/CRM

Operations

Dark data/not captured

• Expensive

• Not “right time”

• Not detailed enough

• Slow


HP ConvergedSystems for Big Data Ready for HAVEn

Lego:

Lego Blocks: ConvergedSystems built from core building blocks

Lego Kit: Specific use case, target market and market segment

addressed by a ConvergedSystem solution

Lego City: Scalable, expandable, extensible HAVEn components


HP Converged Systems for Big Data Product Description

– HP Converged Systems address the growing needs of customers looking for enterprise ready & secure solutions to analyze big data cost effectively

Ideal Customer

– Medium to Large Enterprises and Lines of Business that are considering alternatives to expensive big data technologies for targeted use cases

– Provide customers the choice and flexibility they need to address a variety of Big Data workloads

Customer Value and Metrics

– Easy to procure, build and manage

Development Goals

– Deliver a portfolio of HP Converged Systems to support leading and innovative big data use cases

– Establish HP as a market leader and trusted advisor for big data

Related Services

– Technology services for installation, deployment and support

– Technology consulting for big data portfolio and business transformation


Blazingly fast big data analytics at massive scale and low TCO

Introducing HP ConvergedSystem 300 for Vertica

50x-1,000x faster performance

70% less per Terabyte than legacy data warehouses*

Performance • Accelerate your existing EDW with near real-time

analytics

Cost • More Data for Less • Competitively priced against EMC and Teradata

Simplified, rapid deployment

Time to value • Up and running in days vs. months • Modular building block for HAVEn architecture

* Based on large US bank customer success story

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP CONFIDENTIAL 37 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 37

HAVEn

Call center

Suppliers

Point of sale

Machine data

Customer sentiment (emails, social media forums)

Data sources Legacy data warehouse

Bringing big data analytics to legacy data warehouses and dark data

Use case focus - EDW and Hadoop augmentation

Operational analytics

Reporting and analytics

Dashboards

Hadoop data Analytics makes up ~40% of the workload in a data warehouse environment

Lines of business Finance

Human resources

Sales/CRM

Operations

• Cost effective

• Real-time

• Deep granularity

• Fast


Converged System 300 for Vertica

Improved efficiencies and lower TCO

• 70 percent savings in software and hardware costs4

• Cost-effective design saves up to 66 percent in DBA resources5

• Single system for structured and unstructured data

• 2X to 15X deeper compression than legacy data ware houses translate into lower infrastructure and admin costs6

Blazingly fast, near real-time analytics

• 50–1,000X faster performing queries7

• ETL delays can be avoided for many types of analytics leading to faster, more reliable insights

• Plug and play connectors can rapidly ingest unstructured data from multiple data sources simultaneously

• Compression significantly lowers query latency

• Seamless scaling based on workload demands

Simplified deployment

• Pre-tested, preconfigured, turnkey solution to deploy in days vs. weeks or months8

• Unified deployment, management and monitoring experience

• Integrated solution level support and deployment services

• Plug and play connectors can move data into and out of Vertica rapidly

Key Value Proposition


HP Vertica Analytics Platform

What does HP ConvergedSystem 300 for Vertica do

What Goes In Raw data

What Comes Out Meaningful Information

• Call Center Data • Customer Transactions/Lists • Inventories • Purchases • Web Logs • Sensor Information • Data from Web • Etc.

• Web Analytics • Reports and Information

• Best Selling Product • Best Customer • Most Returned Product • Path to Purchase

• Forward-looking Information • Where are the markets • Many other questions

Information Consumers

Connected Systems

HP DL380p Rack


Product details

HP ConvergedSystem 300

Integrated, configurable, pretested turnkey solution for managing and analyzing big data including • Baseline 3-node compute block

– Expandable to 16 nodes

– HP Vertica 7 Analytics software and licenses • Dedicated management block

– HP Insight CMU, Insight Control and CS300-specific management software

• HP Networking switches • Rack and infrastructure block • Factory Express installation and configuration • Solution-level (single call) 3-year support • Installation and Deployment service • Optional Big Data consulting services


Standard Configuration Performance Configuration

Starter Block (3-16)

Compute Servers (3-16)

DL380p, 2 x E5-2690v2 processors, 10GigE 2-port 561FLR adapter, 128

GB RAM

DL380p, 2 x E5-2690v2 processors, 10GigE 2-port 561FLR adapter, 256 GB

RAM

Storage 23 x 600 GB10k SAS SFF 2 x 300GB 10K system disks

23 x 1.2 TB 10k SAS SFF 2 x 300GB 10K system disks

Mgmt Servers (2) DL360p, 2 x E5-2660v2 processors / 128 GB RAM / 8 x 600 GB disks / 10GigE 2-port 561 FLR Adapter

Networking Block (2) HP 5900AF-48XGT-4QSFP+ Switches

OS Software RHEL 6.4 (or latest), Windows 2012 SE

Big Data SW HP Vertica 7 Analytics Platforms

Management Stack

HP Insight Control, Insight CMU, Vertica Console, Insight Diagnostics, Insight Remote Support, HP SUM, CS developed-appliance-specific console, Windows

Server HAVEn ready

HP Restricted: For HP Internal Use Only

HP ConvergedSystem 300 for Vertica - Product Details


The Vertica Technology

Achieve best data query performance with unique

Vertica column store

Linear scaling by adding more resources on the fly

Store more data, provide more views, use less

hardware

Query and load 24x7 with zero administration

Columnar storage and execution Clustering Compression Continuous

performance

Automated Performance Tuning

Database Design Time-series, geospatial, click-stream

and an SDK for more

Advanced Analytics


HP Vertica Analytics Platform

• Big data analytics for real time business intelligence

• Limitless scaling - add nodes for capacity and performance

• Extreme compression - up to 90% space reduction

• Load and go simplicity, MPP, no single point of failure

• Bi-directional Hadoop data connectors

Real-time and ad hoc analytics


HP intelligent end-to-end approach helps deliver right-time business decisions

Information Infrastructure

• Intelligent Design for performance and high availability

• Intelligent Scaling by Workload for high efficiency scale up or seamless scale out

• Intelligent delivery models for on-premise, cloud, or hybrid cloud

Information Management Information Insight

• Intelligent Management for a holistic management, governance and security

• Intelligent Culling (Prioritization) for separating signal from noise and optimizing resources to act on useful data

• Intelligent Workload Characterization for optimizing for transactional, warehouse or analytic workloads which can significantly vary in memory, processor, storage and networking requirements

• Intelligent pan-enterprise functionality for multichannel management and delivery

• Intelligent purpose-built scalable analytic platforms for advanced, predictive and real-time analytics


HP offers a full suite of packaged services for Big Data

• Complete lifecycle solution services to accelerate time to business results

– Strategy Services

– Design and Transition

– Start Services

– HP Solution Support

• Application transformation and hosting options

• Flexible leasing and financing to accelerate business priorities


Make Data Matter

Why HP for Big Data Unmatched HAVEn platform

100% data covered: structured, unstructured, semi-structured

Complete hardware, software, and services portfolio

#1 HPC vendor for 8 years

No vendor lock-in

Largest partner ecosystem

Unprecedented success designing & building scale-out projects


Our partner ecosystem


hp.com/go/bigdata


“Autonomy, with the power of its IDOL engine, takes fan data, collects, it stores, and stitches it together…that helps us understand what is being talked about across the ecosystem of the sport.” - Senior Director of IT, NASCAR

HP Autonomy Solution: • Autonomy IDOL + Autonomy Explore

Results: • Wildly successful NASCAR Fan and Media Engagement Center

collects and aggregates fan information

• Understands sentiment, identifies emerging issues, and uncovers trends that help the NASCAR team share and enrich the fan and broadcast experience

Powering real-time, multichannel fan engagement

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=8ZfdHx2ys6aDnM&tbnid=vNFwQaFywEusqM:&ved=0CAUQjRw&url=http://commons.wikimedia.org/wiki/File:NASCAR_practice.jpg&ei=NyuxUa6wEMW5qQHqjIGgCA&bvm=bv.47534661,d.aWM&psig=AFQjCNGHEpRyF6RxYvn4Z3UytvEryP2neA&ust=1370651822520250

lynn wong: make a difference with big data - hp

Technology

data deluge

big data market dynamics

big challenge

d new tools

whats new

explosion of new services

informationannual growth

social networks