evolution from apache hadoop to the enterprise data hub by cloudera - arabnet digital summit 2014

14

Click here to load reader

Upload: arabnet-me

Post on 11-Aug-2014

565 views

Category:

Data & Analytics


4 download

DESCRIPTION

A new foundation for the Modern Information Architecture. Speaker: Amr Awadallah, CTO & Cofounder, Cloudera Our legacy information architecture is not able to cope with the realities of today's business. This is because it is not able to scale to meet our SLAs due to separation of storage and compute, economically store the volumes and types of data we currently confront, provide the agility necessary for innovation, and most importantly, provide a full 360 degree view of our customers, products, and business. In this talk Dr. Amr Awadallah will present the Enterprise Data Hub (EDH) as the new foundation for the modern information architecture. Built with Apache Hadoop at the core, the EDH is an extremely scalable, flexible, and fault-tolerant, data processing system designed to put data at the center of your business.

TRANSCRIPT

Page 1: Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet Digital Summit 2014

CONFIDENTIAL

The Future of Data Management: The Enterprise Data HubAmr Awadallah (@awadallah) | Co-Founder & CTO

Page 2: Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet Digital Summit 2014

©2014 Cloudera, Inc. All rights reserved. 2

Cloudera SnapshotFounded 2008, by former employees ofEmployees Today ~ 600World Class Support 24x7 Global Staff

Pro-active & Predictive Support ProgramsMission Critical Thousands of Enterprise Users

Over 350 Paying Subscription CustomersThe Largest Ecosystem Over 1000 PartnersCloudera University Over 40,000 TrainedOpen Source Leaders Cloudera Employees are Leading Developers & ContributorsTotal Capital Raised A lot! (from Intel, Google, Dell, T. Rowe Price, Accel, Greylock)Mission Help Organizations Leverage the Power of All Their Data to

Ask Bigger Questions.

Page 3: Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet Digital Summit 2014

©2014 Cloudera, Inc. All rights reserved. 3

An Environment of Change

Page 4: Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet Digital Summit 2014

©2014 Cloudera, Inc. All rights reserved. 4

Expanding Data Requires A New ApproachWhat we doCopy Data to Applications

What we should doBring Applications to Data

DataInformation-centric

businesses use all Data:

Multi-structured, Internal & external data

of all types

App

App

App

Process-centric businesses use:

• Structured data mainly• Internal data only• “Important” data only•Multiple copies of data

App

App

App

Data

Data

Data

Data

Page 5: Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet Digital Summit 2014

©2014 Cloudera, Inc. All rights reserved.

The Power of the EDH

5

THE OLD WAY EDH

Page 6: Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet Digital Summit 2014

Hadoop Changes the Game: Storage and Compute on One Platform

©2014 Cloudera, Inc. All rights reserved. 6

The Hadoop WayThe Old Way

$30,000+ per TBExpensive & Unattainable

• Hard to scale• Network is a bottleneck• Only handles relational data• Difficult to add new fields & data types

Expensive, Special purpose, “Reliable” ServersExpensive Licensed Software

Network

Data Storage(SAN, NAS)

Compute(RDBMS, EDW)

$300-$1,000 per TBAffordable & Attainable

• Scales out forever• No bottlenecks• Easy to ingest any data• Agile data access

Commodity “Unreliable” ServersHybrid Open Source Software

Compute(CPU)

Memory Storage(Disk)

zz

Page 7: Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet Digital Summit 2014

©2014 Cloudera, Inc. All rights reserved. 7

Hadoop and The Enterprise Data HubOpen SourceScalableFlexibleCost-Effective

✔Managed ✖Open Architecture ✖Secure and Governed ✖

✔✔✔

3RD PARTYAPPS

STORAGE FOR ANY TYPE OF DATAUNIFIED, ELASTIC, RESILIENT, SECURE

CLOUDERA’S ENTERPRISE DATA HUB

BATCHPROCESSING

ANALYTICSQL

SEARCHENGINE

MACHINELEARNING

STREAMPROCESSING

WORKLOAD MANAGEMENT

FILESYSTEM ONLINE NOSQL

DATAM

ANAGEM

ENT

SYSTEMM

ANAGEM

ENT

, SECURE

Page 8: Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet Digital Summit 2014

Enterprise Data Hub: A Complete Big Data Solution

©2014 Cloudera, Inc. All rights reserved.

• Full-Fidelity Active Compliance Archive• Accelerate Time to Insight• Unlock Agility and Innovation• Consolidate Silos for 360o View• Enable Converged Analytics

Page 9: Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet Digital Summit 2014

BI and Analytics Partners

Enabling The App Store of Big Data

SI, Cloud, MSP Partners

Database PartnersResellers

Data Integration PartnersHardware Partners

©2014 Cloudera, Inc. All rights reserved.

Page 10: Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet Digital Summit 2014

Customer Success Across IndustriesFinancial &Business Services

Telecom & Technology

Healthcare &Life Sciences

Media &Information

Retail &Consumer

Energy & Public Sector

©2014 Cloudera, Inc. All rights reserved.

Page 11: Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet Digital Summit 2014

How many cars are there in each Walmart parking lot?Can we use that information to refine our prediction?

How will seasonality affect Walmart’s earnings next quarter?

©2014 Cloudera, Inc. All rights reserved. 11

Page 12: Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet Digital Summit 2014

12

How do seed selection, planting density, ground temperature, soil composition & weather impact yields?

How much corn did my farm produce last year?

Page 13: Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet Digital Summit 2014

13

Thank You!

©2014 Cloudera, Inc. All rights reserved.

Page 14: Evolution from Apache Hadoop to the Enterprise Data Hub by Cloudera - ArabNet Digital Summit 2014

WEB/MOBILE APPLICATIONS

ONLINE SERVING SYSTEM

ENTERPRISE DATA WAREHOUSE

ENTERPRISE REPORTINGBI / ANALYTICSMACHINE

LEARNINGCONVERGED

APPLICATIONSCLOUDERA MANAGER

META DATA / ETL TOOLS

ENTERPRISE DATA HUB

©2014 Cloudera, Inc. All Rights Reserved.

The Modern Information ArchitectureData Architects System Operators Engineers Data Scientists Analysts Business Users

Customers & End Users

SYS LOGS WEB LOGS FILES RDBMS