hvorfor migrere til cloud? - visma · 2019-11-29 · lab architecture synapsedatafactory-suffix...

46
Hvorfor migrere til Cloud? Jon Jahren, Microsoft Norge

Upload: others

Post on 22-May-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Hvorfor migrere

til Cloud?Jon Jahren, Microsoft Norge

Page 2: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory
Page 3: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory
Page 4: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

| |

Page 5: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Why move to the cloud?

Page 6: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Why move to the cloud?

Page 7: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Why move to the cloud?

Page 8: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Why move to the cloud?

Page 9: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Why move to the cloud?

Page 10: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Photo: Iwan Baan/New York Magazine

Page 11: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Why move to the cloud?

Page 12: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Business Returns

IT must rapidly produce measurable

business returns to stay relevant

Transformation

Evolving how businesses operate and

interact with the market

Modernization

Improving customer and employee

experiences

Business Transformation enabled by Cloud Technologies

Key Business Drivers

Growth

Scaling products and services to meet

ever growing business needs

Page 13: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

The major drivers for

IT Governance

Keep risk at acceptable levels

Maintain availability to systems

and services

Consistently apply policy and

audit compliance

Protect customer data

Page 14: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Objective of this model: Create balance

Control &

Stability

Speed &

Results

Page 15: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Organisations are Built for Yesterdays Market

M2: Speed, Agility & Sensing

M1: Todays Operational Reality

12-18 months

Mode 2: Built to Evolve Dynamic marketplace with increasing frequency of disruptive innovations –requiring a rapidly agile and adaptive operating model

Mode 1: Built to Run Market stability – multi year\decade innovation cycles – requiring an operating model that captures markets and extracts value

… Transformation is a Bimodal Challenge

Page 16: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Digital Transformation

One story two entry points

Mode 1Mode 2

Real-time Brand

Experience Challenge

Digital Customer

Dialogue

Systems of

Engagement

Systems of

Intelligence

Turn data into Bus

Model insight

Agile

M2: IT Ops Model

- Low cost & low change risk

- On demand – agile cycle times

- Distributed empowerment

- Zero based infrastructure

M1: IT Ops Model

- High cost & high change risk

- Waterfall cycle times

- Centralised delivery

- Infrastructure friction

Business Led IT Led

COGS – Customer Value &

Revenue Focus

BLUE OCEAN RED OCEAN

SG&A – Cost reduction strategy –

DT funding challenge

Digital Platform Adoption

Shifting complexity, integration &

lifecycle risk to the platform provider

Managing Complexity

Best of breed complexity, integration &

lifecycle risk borne by Enterprise IT Model

Systems of Record – Years of

accumulated technical debt

- 20% Growth Assets

- 80% Cost Reduction Directive

- Untapped M1 data

Outsourcing Model:

Underwrite the OpEx Savings

with 5 yr contract – your mess

locked in, for less

Cannot get to M2 with outsourcing

model – Direct to Vendor

Customer

Service

Revolution

Digital

Workforce

Revolution

Page 17: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Microsoft Cloud Adoption Framework for Azure

Ready

• Azure readiness guide

• First landing zone

• Expand the blueprint

• Best practice Validation

Plan

• Digital estate

• Initial organization alignment

• Skills readiness plan

• Cloud adoption plan

AdoptDefine Strategy

• Understand motivations

• Business outcomes

• Business justification

• Prioritize project

ManageBusiness commitmentsoperations baseline •

Ops maturity

GovernMethodology • Benchmark

initial best practice • Governance maturity

Migrate• First workload migration

• Expanded scenarios

• Best practice validation

• Process improvements

Innovate• Innovation guide

• Expanded scenarios

• Best practice validation

• Process improvements

Page 18: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Assess current state and future state to

establish a vision for applying the frameworkAssess2

Establish a Minimally Viable Product (MVP) to

serve as a foundation for governanceMVP3

How do I get started?

Frame the conversation to mitigate tangible

business risks through consistent governanceFramework1

Mature with each release to align Cloud

Adoption and existing IT functionsEvolve4

Page 19: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Traditional Approach to Control

Block Dev/Ops from directly accessing the cloud (portal/api/cli) to attain control

Developers

Operations

Cloud Engineers responsible for Cloud environment

Page 20: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Speed + Control

Cloud Engineers

Developers

Operations

Management

Groups

Templates RBAC

Blueprints

Policies

Policy

Cloud-native governance -> removing barriers to compliance and enabling velocity

Cost

Management

Page 21: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory
Page 22: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Assess current state and future state to

establish a vision for applying the frameworkAssess2

Establish a Minimally Viable Product (MVP) to

serve as a foundation for governanceMVP3

How do I get started?

Frame the conversation to mitigate tangible

business risks through consistent governanceFramework1

Mature with each release to align Cloud

Adoption and existing IT functionsEvolve4

Page 23: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

http://aka.ms/adopt/gov/assess

Page 24: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Understand the business vision driving cloud adoption

Page 25: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Evaluating current state

Security managementappears to be an important area of focus for this customer.

Page 26: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Assess current state and future state to

establish a vision for applying the frameworkAssess2

Establish a Minimally Viable Product (MVP) to

serve as a foundation for governanceMVP3

How do I get started?

Frame the conversation to mitigate tangible

business risks through consistent governanceFramework1

Mature with each release to align Cloud

Adoption and existing IT functionsEvolve4

Page 27: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

2. Subscriptions: To group similar

resources into logical collections

3. Resource Groups: To further group

applications or workloads into

deployment and operations units

1. Management Groups:

To reflect security,

operations and

business/accounting

hierarchies

Resource OrganizationThe basic foundation of all governance practices

Page 28: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Assess current state and future state to

establish a vision for applying the frameworkAssess2

Establish a Minimally Viable Product (MVP) to

serve as a foundation for governanceMVP3

How do I get started?

Frame the conversation to mitigate tangible

business risks through consistent governanceFramework1

Mature with each release to align Cloud

Adoption and existing IT functionsEvolve4

Page 29: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

10s - 1,000s of apps Diverse infrastructure Multi-cloud

IoT devices Edge

Datacenters

Branch offices

Hosters

OEM hardware

VMs

Containers

Databases

Serverless

Customer environments are increasingly complex

Page 30: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Microsoft Azure

Page 31: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory
Page 32: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Always current Cloud billingElastic scale Unmatched securityUnified management

• Deploy in seconds

• Scale up/down, scale out

based on capacity

• Automation of DB tasks at

scale, e.g. high availability

• Access to Azure Advanced

Data Security

• Consistent governance with

Azure Policy & Role-based

Access Control

• Modern cloud billing

model across hybrid

infrastructure

• Leverage existing licensing

investments with Microsoft

• Single view across on-

prem and public cloud

• Integrated management

across data infrastructure

and operations

Azure data services anywhere

Any hardware, any Kubernetes

Bring Azure data services to any infrastructure with Azure Arc

• Automated patching/updates

based on customer policy

• Hyperscale innovation now

available on-premises

• Evergreen SQL with no end-

of-support

Page 33: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory
Page 34: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory
Page 35: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Apps and BICustom

apps AnalyticsBI

AKS

Any Kubernetes

Any hardware

Azure data services

Microsoft Azure

Site Recovery

OEM hardware

Azure Arc management

GKE EKSKubernetes OPENSHIFT

Azure Site RecoveryMonitoringAzure Security

Provisioning

HA/DR

Scaling

Updates

Backup

Diagnostics

Azure data services anywhere at a glance

Page 36: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory
Page 37: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

© Microsoft Corporation

Modern Data Platform Reference Architecture

Semi-StructuredV=Volume

csv, logs, json, xml(loosely-typed)

Non-structuredV=Variety

images, video, audio, free text(no structure)

StreamV=Velocity

IoT devices, sensors, gadgets(loosely-typed)

Load and Ingest

Store

Process

Serve

Data Factory

Event Hubs Stream Analytics

Relational Databases(strongly-typed, structured) Azure Synapse Analytics Power BI Premium

DatabricksAzure Data Lake Gen2 CosmosDB

Power BI Premium

Application

Analytics

Analytics

λ Lambda Architecture

Hot PathReal Time Analytics

Cold PathHistory and

Trend Analysis

Stream Datasets

and Real-time

Dashboards

Enterprise-grade

semantic model

Integrate big data

scenarios with

traditional data

warehouse

Cognitive ServicesAzure ML

Business User

Fast load

data with

Polybase/

ParquetDirect

Build and Score ML models

Scheduled / event-

triggered data ingestion

Page 38: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

© Microsoft Corporation

Azure Data PlatformResource Group

ADPDesktopVirtual Machine

ADPVirtualNetworkVirtual Network

MDWResourcesStorage Account

ADPLogicAppLogic App

SynapseDataLakesuffixAzure Data Lake Storage Gen2

ADPEventHubs-suffixEvent Hubs

SynapseStreamAnalytics-suffixEvent Hubs

SynapseDataFactory-suffixAzure Data Factory

ADPDatabricksAzure Databricks

ADPComputerVisionComputer Vision API

ADPCosmosDB-suffixAzure CosmosDB

PowerBIPower BI Desktop/Workspace

Azure Data Platform End2EndLab Architecture

SynapseDataFactory-suffixAzure Data Factory

Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory Pipelines

Lab 2: Transform Big Data using Azure Data Factory Mapping Data Flows

Lab 3: Explore Big Data with Azure Databricks

Lab 4: Add AI to your Big Data pipeline with Cognitive Services

Lab 5: Ingest and Analyse Real-Time Data with Event Hubs and Stream Analytics

1

2

3

4

1

1

2

2

3

4

5 6

1

1

2

3

3

4

Student s Computer

operationalsql-suffix\NYCDataSetsAzure SQL Database

2

synapsesql-suffix\SynapseDWAzure Synapse Analytics

5

4

RDP Connection or

Azure Bastion

Page 39: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

PAGE 39

Example Reference Architecture for Data & Analytics

Raw Data Enriched data

Valu

e A

dd

ed

IT

Serv

ices

Page 40: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Example Data Journey

Enterprise

Data

Warehouse

Federated BI

(Logical DW)

Self-service BI

Enterprise

Information

Management

Transformation:

Data &

Intelligence

Page 41: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

PAGE 41

Data Strategy Measures of Success

41

Data Foundation

• Modern Architecture

Usage

• Data Health

Scorecards

• Active Usage (View +

Go-to-Green plan)

• # of business issues/

customer metrics

moved from red to

green

Diagnostics

• Culture: # of

experiments

• Impact of Diagnostics

($$/ Satisfaction)

ML & AI

• Culture: # of AI/ ML

Models

• Impact of ML/ AI ($$/

Satisfaction)

Raise the Overall Data Quotient!

Page 42: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Example Approach to the Data Lake

o Get Executive Sponsorshipo Create architect/Engineer and COEo Agree Basic Tenets and data lake approacho Define standards & governance model

Run PoC Create infrastructure Socialize

Identify Key scenario Implement governanceRegular updates to

leadership team

Create initial lake Implement infrastructure

Working group socializes each department

Complete end-to-end scenario

Add telemetry, tagging etc.Expectation setting to

business

Page 43: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Data Management

Tenets

43

Create Agility by moving to API access

Reduce Cost & Risk

Minimize Data Duplication – Compute to storage

Production data governed with high quality

Deliver Data at the Right Time

Modernization

Elastic Scale out of storage & compute separately

Discover & Publish single source of truth data

Include ML into initial solution design

Page 44: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

Example Data Platform : Architecture

Page 45: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory

PAGE 45

Drilldown: Data Vision Guiding Principles

Focus on horizonal/strategic capabilitiesStrategic projects with higher impact across org

Business/ Customer driven approach

Driving the Culture with Change Leadership

Strategic outside-in approach (user view)

Quad Model Approach for Product Development

Drive Change Leadership with data everyday

Build on Connected capabilitiesCreate unique connected data sets

Standardize access requirements

One Data TeamUtilize the full-stack capabilities

Up the Data Quotient from Foundation to ML/AI capabilities

Keep and double down on what works

Data platform vision execution

Productized ML

Scorecard & Service Scorecards

Page 46: Hvorfor migrere til Cloud? - Visma · 2019-11-29 · Lab Architecture SynapseDataFactory-suffix Azure Data Factory Lab 1: Load Data into Azure Synapse Analytics using Azure Data Factory