infrastructure transformation for intelligent data centers of the future
DESCRIPTION
The future IT infrastructure for Big Data requires the introduction of specific technologies that need to span from Big Data processing to Big Data management and Big Data integration into current data center environments and IT strategy. This session will explore the Big Data infrastructure transformation required for intelligent data centers of the future.TRANSCRIPT
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Intelligent Data Center Infrastructure Big Data IT Transformation
Donald Livengood on behalf of Amos Ferrari June 10, 2013
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3
Forward-looking statements
This document contains forward looking statements regarding future operations, product development, product capabilities and availability dates. This information is subject to substantial uncertainties and is subject to change at any time without prior notification. Statements contained in this document concerning these matters only reflect Hewlett Packard's predictions and / or expectations as of the date of this document and actual results and future plans of Hewlett-Packard may differ significantly as a result of, among other things, changes in product strategy resulting from technological, internal corporate, market and other changes. This is not a commitment to deliver any material, code or functionality and should not be relied upon in making purchasing decisions.
This is a rolling (up to three year) roadmap and is subject to change without notice.
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 4
My background
Title
VP, Big Data Services
IT industry experience • Application Services Manager – UPS USA
• Director, Technical Support – UPS USA
• Technical Services Manager – Spain
Professional information • Computer Science engineer
• University of Minnesota
Years at HP
13
Current responsibilities VP, Global Technology Services Consulting Big Data Practice – leads the envisioning, development and implementation of Big Data related services globally
Name: Tom Norton E-mail: [email protected]
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 5
My background
Title
Global Consulting Strategist
TS Consulting
IT industry experience • Big Data
• Unified Communications & Collaboration
• Contact Center
• Telco BSS
Professional information • Certified PM
• IT Architect
Years at HP
27
Current responsibilities Responsible for the development, pursuit, and deployment of HP’s Big Data Infrastructure Services for the Strategic & Planning area
Name: Amos Ferrari
E-mail: [email protected]
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Big Data & IT
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 7
Big Data
Main value for you:
Additional business value
New customer services creation
Differentiator and value creation
Business control
“Big Data is a class of data challenges, due to increasing Volume, Velocity, Variety, and Vulnerability of data being processed and consumed, that are beyond the capabilities of the traditional software, architecture, and processes to effectively manage and utilize.”
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 8
Big Data Needs
Scale
Source Speed Security
Volume Variety Velocity Vulnerability
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9
Enterprises can’t keep up
Time consuming Data integration
Real-time Insight needed
Unable to secure
Flash flooding of legacy data bases
Magnitude of the data
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 10
New Data Management and analytics technologies
Variety Volume
Velocity
SAP HANA
PDW
Vertica
Hadoop
Autonomy
Semi-Structured
Structured
Unstructured
100% of Data Enable me to: on
Store
Capture
Manage
Analyze
Optimize
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 11
To energy and algorithm optimized:
HP Moonshot
From General Purpose architecture
With new Infrastructure technologies
To Software Defined Networks
From hardware networks
From disk
To Virtualized Storage and persistent memory
Today
Physical High performance network
High Performance attached Storage
Today -
Today
Scale Out Architecture
Enabling Real Time Services
Scalable and Cost effective
Managed and Secure
Infrastructure Enable me to: on
Store
Capture
Manage
Analyze
Optimize
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Big Data requires Infrastructure Transformation
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 13
Today Data Context
Op
era
tio
n, G
ov
ern
an
ce, S
ecu
rity
Op
era
tio
n, G
ov
ern
an
ce, S
ecu
rity
Op
era
tio
n, G
ov
ern
an
ce, S
ecu
rity
Op
era
tio
n, G
ov
ern
an
ce, S
ecu
rity
Social Media Data
Forum
Blog
Feeds
Web
Clicks
Multi-Media
Audio
Video
Images
Document Management
Content Management
File Sharing
File Hosting
Collaboration
Search
Message Data
IM and VOIP
Messaging System
Sensors Data
GPS
Sensors devices
RFID
Other events
Op
erat
ion
, Go
ver
nan
ce, S
ecu
rity
O
per
atio
n, G
ov
ern
ance
, Sec
uri
ty
Classic ETL Processing
Business Transactions and Interactions
Business Transactions and Interactions
CRM – ERM – SCM FMS – HRM
$ € ¥ Transaction Data
Analytical, Dashboards, Reports, Visualization
Enterprise Data Warehouse
Business Intelligence & Analytics
Internal Content
External Content or Discarded Content
Managed Content Unmanaged Content
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 14
Expanding today Data Management Technologies is expensive and in most cases will not solve
Finding out useful information requires powerful and massive processing
Sto
rag
e O
nly
Gri
d (
SA
N/N
AS
)
Moving data to compute
doesn’t scale
Can’t explore original, High
fidelity raw data
Archiving Cheap storage Expensive restore
Today's Data Architecture
Classic ETL Processing
Business Transactions and Interactions
Business Transactions and Interactions
CRM – ERM – SCM FMS – HRM
$ € ¥ Transaction Data
Analytical, Dashboards, Reports, Visualization
Enterprise Data Warehouse
Business Intelligence & Analytics
Gaps in
Message Data
Document Management
Social Media Data
Multi-Media
Sensors Data
Time consuming Data integration
Flash flooding of legacy data bases
Magnitude of the data
Real-time Insight needed
Unable to secure
Discarded Content, too expensive to retain store manage and analyze
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 15
Big Data Transformation Architecture
Big Data infrastructure
Insight Processing
Protection & Compliance Management
Infrastructure Integration
Enterprise Data Warehouse
Analytical, Dashboards Reports, Visualization
Business Intelligence
Business Transactions and Interactions
Web, Mobile
CRM – ERM – SCM FMS – HRM Value
Creation
share, refine & development
Message Data
Document Management
Social Media Data
Multi-Media
Sensors Data
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Big Data infrastructure Architecture components
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 17
Building the Intelligent Data Center
Big Data infrastructure
Insight Processing – For fast, flexible and predictable data refinement
Protection & Compliance Management – For secure, compliant and reduced risk data management
Infrastructure Integration – For full integration into the data center, traditional data
platforms and the holistic IT environment
Social Media Data
Multi-Media
Document Management
Message Data
Sensors Data
Cloud
Enterprise Data Warehouse
Cloud
Enterprise Data Warehouse
Analytical Dashboards and Reporting
Organizational Applications
Operations
Sales and Marketing
Legal
Web and Mobile Interaction Data
CRM – ERM – SCM – FMS - HRM
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 18
Components
Big Data infrastructure
Big Data Protection
Information Insight Processing
Compute Consumption Collection
Protection and Compliance Management
Information Infrastructure Integration
Security Governance Technology and
Operations Integration
Compliance Protection
Destruction
Security Backup &
Restoration
Governance
Archival Retention
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 19
Big Data infrastructure functional architecture
Technology Integration
Se
curi
ty
Op
era
tion
s
Big Data Converged, Automated, Energy efficient infrastructure
Activity logging Intrusion Prevention
Switch virtualization Virtual application network
SSL/VPN Networks Storage replication
Server scale-out management
Collection Compute Consumption
Protection
Big Data Management
Compliance
Big Data Storage
Big Data Processing On Line / Batch Analysis
Internal / External Data
Structured / Unstructured
Real Time
Backup and Recovery
Governance
Privacy and Security
Destruction Archival Retention
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 20
Big Data Compute key capabilities
Vertica
…
Private
Hadoop
Cloud
Machine 1
RAM
Processors
Memory
Scale-Out
In-Memory
CPU
CPU
CPU
Rack 1
Se
rve
r S
erv
er
Se
rve
r S
erv
er
Se
rve
r S
erv
er
Se
rve
r S
erv
er
Se
rve
r S
erv
er
serv
er
Se
rve
r
Rack 2
Se
rve
r S
erv
er
Se
rve
r S
erv
er
Se
rve
r S
erv
er
Se
rve
r S
erv
er
Se
rve
r S
erv
er
serv
er
Se
rve
r
Rack n
Se
rve
r S
erv
er
Se
rve
r S
erv
er
Se
rve
r S
erv
er
Se
rve
r S
erv
er
Se
rve
r S
erv
er
serv
er
Se
rve
r
Hybrid
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 21
Big Data Management & Integration key capabilities
• End 2 End Big Data SLA Services • Data in Transit Protection • Platform Management • …
Computation Collection Consumption
Security & Compliance
Converged, Automated, Energy Efficient Infrastructure
Level of service
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 22
HP HAVEn Big Data technologies mapping
Information Infrastructure integration
Sec
uri
ty
Op
era
tion
s
Big Data Converged, Automated, Energy efficient infrastructure
Activity logging
Intrusion Prevention
Switch virtualization Virtual application network
SSL/VPN Networks Storage replication
Server scale-out
management
Collection Computation Consumption
Protection
Information Management
Compliance
Information Insight processing
Open Source
Internal / External Data
Structured / Unstructured
Partners
Backup and Recovery
Governance
Privacy and Security
Destruction Archival Retention
Consumption
HP StoreEver HP StoreOnce HP StoreOnce HP StoreAll
OPERATIONS ORCHESTRATION HP DataProtector7
HP FlexFabric
SERVER AUTOMATION
HP
Big Data Governance
SAP- HANA Microsoft PDW
Information Governance
Content Management
eDiscovery
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Big Data IT Infrastructure to enable Intelligent Data Center of the future Examples
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 24
Transforming Today’s Architecture
Big Data infrastructure example with Hadoop
Insight Processing
Protection & Compliance Management
Kerberos access. Identity management, Network optimization.
Centralized Management
Enterprise Data Warehouse
Reports and Interactive Apps
RDBMS (Aggregated data)
Business Transactions and Interactions
Web, Mobile
CRM – ERM – SCM FMS – HRM
Message Data
Document Management
Social Media Data
Multi-Media
Sensors Data
Hadoop Explore all original, High fidelity
raw data
Replicate data in Hadoop Archive only value data
Move some data to BI systems and also use
Hadoop MapReduce
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 25
Expanding capabilities
Big Data infrastructure example with Hadoop and Vertica
Kerberos access. Identity management, Network optimization. Centralized
Management
Enterprise Data Warehouse
RDBMS (Aggregated data)
Business Transactions and Interactions
Web, Mobile
CRM – ERM – SCM FMS – HRM
Message Data
Document Management
Social Media Data
Multi-Media
Sensors Data
Hadoop Explore all
original, High fidelity raw data
Keep data in Hadoop
Cheap store & restore
Vertica Real Time structured Analytics
Move Refined Info to Vertica
Easy protect and comply
Vertica Real Time Analytics
Move some data to BI systems and also use
Hadoop MapReduce
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 26
Inserting Intelligence
Big Data infrastructure example with Hadoop and Autonomy
Kerberos access. Identity management, Network optimization. Centralized
Management
Enterprise Data Warehouse
RDBMS (Aggregated data)
Business Transactions and Interactions
Web, Mobile
CRM – ERM – SCM FMS – HRM
Message Data
Document Management
Social Media Data
Multi-Media
Sensors Data
Hadoop Explore all
original, High fidelity raw data
Keep data in Hadoop
Cheap store & restore
Au
ton
om
y Id
ol
Autonomy Real Time Search
Access and search all Data
Easy protect and comply
Move some data to BI systems and
also use Hadoop MapReduce Autonomy
Real Time Universal Access
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Big Data transformation IT Infrastructure to enable Intelligent Data Center of the future Where to start
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 28
Suggested actions
Identify Critical DW systems for potential initial target implementation
Outline today’s data bottlenecks related to Big Data
Evaluate areas where today’s raw data is not used or deleted
! Determine demand for archived data from the business and potential services that you can offer
Determine your IT Big Data Architecture Strategy.
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 29
Start today
Where to start
Implement Big Data infrastructure
Determine readiness ,Storage, Governance, Security …
Derive IT Architecture & Roadmap
Derive IT Strategy & define initiative
Identify Big Data Strategy
Information sources
Business and Technical feasibility
!
Business Case
People
Link with BI and Analytics and other business cases
Expand infrastructure
Define Solutions Standards & Design
Plan for Big Data Services
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 30
HP Big Data Solutions
Big Data as a Service
HP Big Data Analytics for IT
HP Actionable Intelligence Solutions
HAVEn – HP’s Big data analytics platform
Hadoop • Autonomy • Vertica • Enterprise Security • n of solutions
Big Data Solutions
Vertica Community Edition
Autonomy Legacy Data Cleanup
HP IT Management
HP Big Data Architecture Strategy Services, HP Big Data System Infrastructure services, HP Big Data Protection Services
Operate yourself Consume as a service
Professional Services
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Take the lead Providing IT Big Data Services by Transforming data Infrastructure to enable Intelligent Data Center of the future
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 32
Learn more about this topic
Use HP Autonomy’s Augmented Reality (AR) to access more content
1. Launch the HP Autonomy AR app*
2. View this slide through the app
3. Unlock additional information!
*Available on the App Store and Google Play
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Thank you