ssp aps laurent marzouk data insight & cloud architecth41382. · 100-percent apache hadoop sql...

33
Aurélie Pericchi SSP APS Laurent Marzouk Data Insight & Cloud Architect

Upload: others

Post on 22-May-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Aurélie Pericchi

SSP APS

Laurent Marzouk

Data Insight & Cloud Architect

Page 2: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

2005Concert de Coldplay

Page 3: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Concert de Coldplay

2014

Page 4: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

of the world’s

data has been

created over the

last two

years alone1

90%

1. Source IDC

Page 5: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

5

“Digital business is the

creation of new business

designs by blurring the

digital and physical worlds.”Source: Putting Digital Business to Work in 2015. Gartner.

crowdsourcing and data intelligent

operations

smart machines

anticipating what’s next

customers with richer experiences

Big Data is changing how business gets done…

Page 6: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Why leadings companies use Big Data and analytics ?

6

Cost reductions

Time reductions

New product development and optimized offerings

Smarter business decision making

42% of businesses are developing new products and

business models using big data

Page 7: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Leading enterprises are reimagining their businesswith Big Data

Transforming

healthcare

Creating engaging

customer

experiences

Automating

manufacturing

operations

Identifying and

retaining top

talent

Targeting

customers better

Customer

stories

City of BarcelonaKimberly ClarkAscribeKUKA SystemsGroup

PageUp People

Page 8: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Reimagining

the customer

journey

Creatingbusiness valuewith Big Data

Redesigning

operational

processes

Challenging

business

models

Page 9: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Endless

possibilities

for your

business

Web app

optimization

Smart meter

monitoring

Equipment

monitoring

Advertising

analysis

Life sciences

research

Fraud

detection

Healthcare

outcomes

Weather

forecasting

Natural resource

exploration

Social network

analysis

Churn

analysis

Traffic flow

optimization

IT infrastructure

optimization

Legal

discovery

Big data is everywhere. Be first and move fast in your industry.

9

Page 10: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

How can Microsoft help you makesmarter, faster, data-driven decisions?

10

Discover and manage The power to discover, explore, and combine any type or size

of data from anywhere

Visualize and analyze Easy, familiar ways to analyze, visualize, predict and ask

questions of your data

Share and decideThe speed to share across the enterprise and access data

anywhere, on any device

Key messaging

and scenarios

Page 11: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

How can Microsoft help you makesmarter, faster, data-driven decisions?

11

Harness big dataacross the business with a modern data warehouse and cloud-based analytics solutions

Empower business leaders with powerful self-service business analytics

Build mission critical database solutionswith a trusted platform and cloud-based testing infrastructure

Key messaging

and scenarios

Unleash the full potential of the Internet of Thingswith intelligent systems

Page 12: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

“ ”

Empoweringthe digital business

12

Trusted

platform

Integrated

data

Advanced

analytics

Modern

productivity

Connected

things

Page 13: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Connectingdigital platforms…and protecting assets in amobile-first, cloud-first world.

Reimagining the enterprise with Microsoft

13

Empowering people…to be more responsive, agile, and customer-obsessed in the digital age.

…comprehensive, cross-platform and connected solutions…

…culture of reliability, security and privacy…

Global excellence in hybrid cloud services…

…spanning consumer and business

…familiar and fluid experiences…

Page 14: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

14

Page 15: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready
Page 16: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready
Page 17: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Breaking points of traditional approach

Page 18: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Hardware and software engineered togetherThe ease of an appliance

Pre-built hardware + software appliance

• Co-engineered with Dell, HP, and Quanta

• Pre-built hardware

• Pre-installed software

Plug and play Built-in best practices

Time savings Built for Big Data

Analytics Platform System

SQL Server

Parallel Data

Warehouse

Microsoft

HDInsight

(Hadoop)

PolyBase

Page 19: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Microsoft Analytics Platform SystemThe turnkey modern data warehouse appliance

Page 20: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

High performance and tuned within the appliance

End-user authentication with Active Directory

Accessible insightsfor everyone with Microsoft BI tools

Managed and monitored using System Center

100-percent Apache Hadoop

SQL Server

Parallel Data

Warehouse

Microsoft

HDInsight

PolyBase

APS delivers enterprise-ready Hadoop with HDInsightManageable, secured, and highly available Hadoop integrated into the appliance

At rest data encryption of data pages on disk, transactional logs, and database backups

Page 21: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Provides a single T-SQL query model for PDW and Hadoop with rich features of T-SQL, including joins without ETL

Uses the power of MPP to enhance query execution performance

Supports Windows Azure HDInsight to enable new hybrid cloud scenarios

Provides the ability to query non-Microsoft Hadoop distributions, such as Hortonworks and Cloudera

SQL Server

Parallel Data

WarehouseMicrosoft Azure

HDInsight

PolyBase

Microsoft

HDInsight

Hortonworks for

Windows and Linux

Cloudera

Connecting islands of data with PolyBaseBringing Hadoop point solutions and the data warehouse together for users and IT

Result setSelect…

Page 22: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Enhanced compatibility and ease of use with PolyBaseImprove APS operations by extending PolyBase

HDFS file formats

Textfile and

RCFile support

• Microsoft Azure HDInsight

• HDInsight on APS

• Hortonworks Data Platform

1.3 and 2.0 (Linux/Windows

Server)

• Cloudera Linux 4.3

External table

source and file

format syntax

Security and

permission model

Microsoft

Azure

Storage

Blobs

Analytics Platform

System(powered by PolyBase)

PolyBase

Page 23: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Big Data insights for anyoneNew insights with familiar tools through native Microsoft BI integration

Minimizes ITintervention for discovering data with tools such as Microsoft Excel

Enables DBA and power users to join relational and Hadoop data with T-SQL

Offers Hadoop tools like MapReduce, Hive, and Pig for data scientists

Takes advantage of high adoptionof Excel, Power View, PowerPivot, and SQL Server Analysis Services

Power users

Data scientist

Everyone else using

Microsoft BI tools

Page 24: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Microsoft Analytics Platform SystemThe turnkey modern data warehouse appliance

Page 25: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Scale out Multiple nodes with dedicated CPU,

memory, and storage

Ability to incrementally add hardware

for near-linear scale to multiple

petabytes

Ability to handle query complexity and

concurrency at scale

No “forklift” of prior warehouse to

increase capacity

Ability to scale out HDInsight and PDW

Scale-out technologies in the Analytics Platform System

25

PDW

0 terabytes 6 petabytes

PDW /

HDInsight

PDW /

HDInsight

PDW /

HDInsight

PDW /

HDInsight

PDW /

HDInsight

PDW /

HDInsight

Page 26: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

• Store data in columnar format for massive compression

• Load data into or out of memory for next-generation performance with up to 60% improvement in data loading speed

• Updateable and clustered for real-time trickle loading

26

Up to 100xfaster queries

Updateable clustered columnstore vs. table with customary indexing

Up to 15xmore compression

Columnstore index representation

Parallel query execution

Query

Results

Page 27: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

BI Tools

Reporting and cubes

SQL Server SMP

Concurrency that fuels rapid adoptionGreat performance with mixed workloads

Analytics Platform SystemETL/ELT with SSIS, DQS, MDS

ERP CRM LOB APPS

ETL/ELT with DWLoader

Hadoop / Big Data

PDW

HDInsight

(Hadoop)

PolyBase

Ad hoc queries

Intra-Day

Near real-time

Fast ad hoc

Columnstore

Polybase

CRTAS

Link Table

Real-Time

ROLAP / MOLAP

DirectQuery

SNAC

Page 28: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Microsoft Analytics Platform SystemThe turnkey modern data warehouse appliance

Page 29: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Hardware and software engineered togetherThe ease of an appliance

Co-engineered

with HP, Dell, and

Quanta best

practices

Leading

performance with

commodity

hardware

Pre-configured,

built, and tuned

software and

hardware

Integrated

support plan with

a single Microsoft

contactPDW

HDInsight

PolyBase

Page 30: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Compelling Events For APS

Significant

data growth

Daily

information

available too

late

APS = Speed and Scale

Analyzing new

data types and

sources = Big

Data

Poor query or

report

performance

Right-time

analytics

Mixed

workloads /

high

concurrency

Current DWH

is burden on

infrastructure

Current

environment

too expensive

Analytics

Platform

System

HP ConvergedSystem 300

for Microsoft Analytics Platform

Page 31: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Top 10 features and benefits of APS

Great Price /

Performance

with low TCO

Reduce risk

through

appliances and

existing

skillsets

Appliance

simplicity and

ease of

administration

for DBA and

Sys admin

Big Data ready

through

Hadoop

integration

Linear

scalability with

true MPP

architecture

Tier 1 Data

Warehouse

Appliance

Offering

Analytics

Platform

SystemFlexibility

Distributed

Data

Warehouse

Architecture

Complete

solution

through

integration with

MS BI

Blazing fast

performance

with shared

nothing

architecture

1 2 3 4

5 6 7 8

9 10

Page 32: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready

Get connected• Industry, business & IT perspectives

• Events & webcasts

www.microsoft.com/aps

Reimagining the enterprise with MicrosoftNext steps

Please visit us at our stand today to ask questions and see demos

Envisionning workshop

Page 33: SSP APS Laurent Marzouk Data Insight & Cloud Architecth41382. · 100-percent Apache Hadoop SQL Server Parallel Data Warehouse Microsoft HDInsight PolyBase APS delivers enterprise-ready