Analytics on AzureWhat to Use When

Cloud Solution Architect | Data & AI | One Commercial Partner | United Kingdom

Christina E. Leo

christina.leo@microsoft.com

@christinaleo

The Azure Data Landscape

Big Data & Analytics

HDInsight Stream Analytics

Analysis

Services

People

Automated Systems

Apps

Web

Mobile

Bots

Data Sources

Apps

Sensors and devices

IoT

IoT Hub

Prebuild Models

Cognitive

Services

Dashboards &

Visualizations

Power BI

Orchestration

Data Catalog

Data Factory

Event Hubs

Storage Blob

Table Storage

Integrations

Services

DataBricks

Operationalization

Bot

Framework

Functions &

Serverless

Azure Search

Machine Learning

Data Science &

Deep Learning VM

Studio

Azure ML Services

ML Development

Environments

Server

Data Storage

Data Lake Store

CosmosDB

SQL Server 2017

SQL Data

Warehouse

SQL Azure

SQL Managed

Instance

D A T A → S T O R A G E → I N T E L L I G E N C E → A C T I O N

Solution ScenariosBig Data & Advanced Analytics

Modern Data Warehousing

“We want to integrate all our data including ‘big data” with our data warehouse”

Advanced Analytics

“We are trying to predict when our customers churn.”

Real-Time Analytics

“We are trying to get insights from our devices in real-time, etc.”

Value

Uses of Data

BI AA AIDescriptive Diagnostic Predictive Prescriptive

Data Engineering, Warehousing & Preparation

Dashboards and Reports

Detailed Statistics

Machine Learning

Deep Learning

Analyst Data Scientist

ML EngineerData Engineer

AI Engineer

Modern Data Warehousing

Modern Data Warehousing

“We want to integrate all our data including ‘big data” with our data warehouse”

Advanced Analytics

“We are trying to predict when our customers churn.”

Real-Time Analytics

“We are trying to get insights from our devices in real-time, etc.”

The Modern Data Warehouse extends the scope of the data warehouse to serve “big data” that is prepared with techniques beyond relational ETL.

M O D E R N D A T A W A R E H O U S I N G C A N O N I C A L O P E R A T I O N S

D A T A W A R E H O U S I N G P A T T E R N I N A Z U R E

APPLICATIONS

DASHBOARDSBUSINESS / CUSTOM

APPS

(STRUCTURED)

LOGS, FILES AND MEDIA

(UNSTRUCTURED)

rAZURE DATABRICKS

DATA LAKE

STOREAZURE

STORAGE

HDINSIGHT

AZURE SQL DW

AAS

Loading and preparing data for analysis with a data warehouse

DATA

FACTORY

Azure

Import/Export

Service

API’s, CLI &

GUI Tools

COSMOS DB

COSMOS DB SQL DB

Real-Time AnalyticsReal-time Analytics (aka Stream Analytics) is the phenomenon of processing data as soon as it is generated, to derive very quick analysis/insight for timely action.

Modern Data Warehousing

“We want to integrate all our data including ‘big data” with our data warehouse”

Advanced Analytics

“We are trying to predict when our customers churn.”

Real-Time Analytics

“We are trying to get insights from our devices in real-time, etc.”

S T R E A M I N G - C A N O N I C A L O P E R A T I O N S

B I G D A T A S T R E A M I N G P A T T E R N W I T H A Z U R E

REAL-TIME

APPLICATIONS

REAL-TIME

DASHBOARDSBUSINESS / CUSTOM

APPS

(STRUCTURED)

LOGS, FILES AND MEDIA

(UNSTRUCTURED)

r

SENSORS AND IOT

(UNSTRUCTURED)

EVENT HUBS IoT HUB KAFKA on HDINSIGHT STREAM

ANALYTICS

STORM on

HDINSIGHT AZURE DATABRICKS

(Spark Streaming)

AZURE ML STUDIO

R SERVER AZURE DATABRICKS

(Spark ML)

D A T A W A R E H O U S I N G P A T T E R N I N A Z U R E

Big Data Lambda Architecture

AZURE STORAGE

PolyBase

AZURE SQL DATA WAREHOUSE

AZURE DATA FACTORY

ANALYTICAL DASHBOARDS

WEB & MOBILE APPS

AZURE DATA FACTORY

AZURE DATABRICKS

(Spark ML, SparkR, SparklyR)

AZURE DATABRICKS

(Spark)

AZURE HDINSIGHT

(Kafka)

AZURE COSMOS DB

BUSINESS / CUSTOM

APPS

(STRUCTURED)

LOGS, FILES AND MEDIA

(UNSTRUCTURED)

r

event

AZURE EventHub

Stream Analytics

Advanced AnalyticsAdvanced Analytics is the process of applying machine learning and/or deep learning techniques to data for the purpose of creating predictive/prescriptive insights.

Modern Data Warehousing

“We want to integrate all our data including ‘big data” with our data warehouse”

Advanced Analytics

“We are trying to predict when our customers churn.”

Real-Time Analytics

“We are trying to get insights from our devices in real-time, etc.”

A D V A N C E D A N A L Y T I C S - C A N O N I C A L O P E R A T I O N S

A D V A N C E D A N A L Y T I C S P A T T E R N I N A Z U R E

APPLICATIONS

DASHBOARDSBUSINESS / CUSTOM

APPS

(STRUCTURED)

LOGS, FILES AND MEDIA

(UNSTRUCTURED)

r

SENSORS AND IOT

(UNSTRUCTURED)

DATA LAKE

STOREAZURE

STORAGE

HDINSIGHT AZURE DATABRICKS

AZURE ML ML SERVER

AZURE MLSTUDIO

SQL Server (In-database ML)

AZURE DATABRICKS(Spark ML)

DATA SCIENCE VM

COSMOS DB

SQL DB

SQL DW

AZURE ANALYSIS SERVICES

COSMOS DB

BATCH AI

SQL DB

Performing data collection/understanding, modeling and deployment

DATA

FACTORY

AZURE CONTAINER

SERVICE

SQL Server (In-database ML)

SQL DW

DEMO

TH

ING

S T

O N

OTE

There are no right or wrong solutions,

only optimal solutions

We lead with certain solutions and

customise based on customer scenarios

Customer voice and product and service

maturity govern lead solutions

Consider price and performance, ease of

use, and ecosystem acceptance as factors

Everything is fluid - a lead solution today

might be non-optimal tomorrow, based on

the factors above and new releases

Decision Points

IaaS PaaS SaaS

Customer:

❑ Regionality

❑ VNET requirements

❑ Speed of business

❑ Data Volumes

❑ Data Types

❑ Governance

❑ Team

❑ Skills

AI Decision Tree