Modernizing Your Data Warehouse for Hadoop

Christian Coté

Big data. Small data. All data.

The traditional data warehouse

“…data warehousing has reached the most significant

tipping point since its inception.

The biggest, possibly most elaborate data

management system in IT is changing.”

– Gartner, “The State of Data Warehousing in 2012”

The traditional data warehouse

Real time data2

Increasing datavolumes

1

Cloud-borndata

4

Increasing datavolumes

1 New data sourcesand types

3

The modern data warehouse

Microsoft’s modern data warehouse

Data Platform

PDW

SQL Server 2014

Microsoft Azure HDInsight

Scale out technologies

in Parallel Data Warehouse

0TB 6PB

APS /

HDInsight

APS

APS /

HDInsight

APS /

HDInsight

APS /

HDInsight

APS /

HDInsight

APS /

HDInsight

From terabytes to multi-petabytesScale out relational data to petabytes

In-memory performanceIn-memory Columnstore for next-generation performance

Columnstore

index representation

Concurrency and mixed workloadsGreat performance for mixed workloads

Query

Results

Data complexity: variety and velocity

Petabytes

What is big data?

Hadoop Cluster

What is Hadoop?

Hive

Distributed, scalable system on commodity HW

Core Services

Operational services Data services

HDFS

SQOOP

FLUME

NFS

LOAD & EXTRACT

WebHDFS

OOZIE

AMBARI

YARN

MAP REDUCE

HIVE &HCATALOG

PIG

HBASEFALCON

compute

&

storage

. . .

. . .

. . compute

&

storage

.

.

Hadoop clusters provide scale-out storage and distributed data processing on commodity hardware

Web app

optimization

Smart meter

monitoring

Equipment

monitoring

Advertising

analysis

Life sciences

research

Fraud

detection

Healthcare

outcomesWeather forecasting

Social network

analysis

Churn

analysis

Traffic flow

optimization

IT infrastructure

optimization

Legal

discovery

Natural resource

exploration

Hadoop offerings on-premise and cloudReal-time with complex event processing

Microsoft Azure

Architecture

Analyze unstructured data

in Excel

Combine different types of data with Power

Query

Analyze your data with Power Pivot and

Power View and perform analysis

Features and benefits

Build a cluster in minutes and

tear it down when you’re done

Optimize cluster-size for time to

insight or cost-savings

Features and benefits

Try HDInsight at www.windowsazure.com/bigdata

Try SQL Server for data warehousing in Microsoft Azure VMs atwww.windowsazure.com

Try Hortonworks Data Platform for Windows at www. hortonworks.com/products/hdp-windows/

Try SQL Server 2014 CTP1 at http://www.microsoft.com/en-us/sqlserver/sql-server-2014.aspx