cloud data warehouse modernization on azure workshop...text here big data pricing and title here...
TRANSCRIPT
Text here
Title HereBig Data Pricing and Packaging Overview
Cloud Data Warehouse Modernization on Azure Workshop
Daniel HeinCloud Ecosystem Solution Architect
Matt RogersPartner Alliances Manager
2 © Informatica. Proprietary and Confidential.
Agenda1.00 – Lunch served
1.30 – Welcome and Workshop overview
1.40 – EDC Demo
1.50 – EDC Lab
2.30 – Break
2.40 – IICS Demo
2.50 – IICS Lab
4.00 – Close
3 © Informatica. Proprietary and Confidential.3 © Informatica. Proprietary and Confidential.3 © Informatica. Proprietary and Confidential.
What are the barriers to Azure adoption?
• How do I get my data to Azure?• Where should I land it in Azure?
Connectivity
• What data should I put in Azure?• What data can I put in Azure?
Locating the right data
• Writing custom code is easy for a starter project, but how will I scale on Azure?
Azure Experiment vs Azure Strategy
• Which vendors should I work with on Azure to build a complete cloud data management strategy? • How will I ensure all the pieces work together well?
Patchwork of vendors/services
4 © Informatica. Proprietary and Confidential.
Putting you on the fast lane to Azure
100+Data sources
10xFaster to locate
the right data
17Microsoft product
integrations
A Leader in Five Gartner Magic Quadrants
Magic Quadrant
for Data
Integration Tools
Aug 2017
Mark A. Beyer , et al.,
3 August 2017
Magic Quadrant
for Metadata
Management Solutions
Aug 2017
Guido De Simoni, et al.,
10 August 2017
Magic Quadrant
for Data
Quality Tools
Oct 2017
Mei Yang Selvage, et al.,
24 October 2017
Magic Quadrant
for Enterprise Integration
Platform as a Service
Apr 2018
Keith Guttridge, et al.,
18 April 2018
Magic Quadrant
for Master Data
Management Solutions
Oct 2017
Bill O'Kane, et al.,
30 October 2017
These graphics were published by Gartner, Inc. as part of larger research documents and should be evaluated in the context of the entire document. The Gartner documents are
available upon request from Informatica. Gartner does not endorse any vendor, product or service depicted in its research pub lications, and does not advise technology users to
select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinion s of Gartner's research organization and should not be
construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, inclu ding any warranties of merchantability or fitness for a
particular purpose.
Lead with Informatica
Top 10 of Fortune 100
Lead with Informatica
85 of Fortune 100
88 © Informatica. Proprietary and Confidential.
Think BIGStart Small
Scale Fast
Our Approach to Driving Customer Success
MONITOR AND MANAGE
DATA ENGINE
CONNECTIVITY
DATAQUALITY
ENTERPRISEDATA CATALOG
DATASECURITY
Products
Solutions
MASTER DATAMANAGEMENT
Intelligent
Data Platform
CLOUDREAL TIME/
STREAMINGBIG DATA TRADITIONAL
DATAINTEGRATION
iPaaS
PRODUCT 360
ENTERPRISE DATA GOVERNANCE
SECURE@SOURCESUPPLIER 360
REFERENCE 360
ENTERPRISE DATA LAKE
CUSTOMER 360
BIG DATA MANAGEMENT
10 © Informatica. Proprietary and Confidential.
Informatica products for Azure
Power CenterInformatica
Intelligent Cloud Services
Big Data
Management
Enterprise Data
Catalog
Informatica
Intelligent Cloud
Services
Informatica
Data Quality
Enterprise Data
CatalogPower Center
Master Data
Management
Big Data
ManagementInformatica Data
Quality
Supported on:
Available on:
Supported Azure ConnectorsAzure Blob
Azure Data Lake
Store (ADLS)DocumentDBAzure SQL DW
SQL Server 2016 HD Insight
Dynamics365
(CRM, AX, GP,
NAV)
CosmosDB* Check PAM for specific product support
Axon Secure@Source
Informatica
Intelligent Streams
PaaS PAYG and BYOL
Azure SQL DB
11 © Informatica. Proprietary and Confidential.
Azure “On-Ramp”
Jumpstarting
the Cloud Data
Warehousing Journey
Informatica is the “On-ramp”to Azure SQL DW
• Connectivity to all on-premises data warehouse vendors
• Intelligent cataloging to make it easy to locate data to be moved to the cloud
• Best-in-class data integration capabilities
• Rapidly identify dependencies to develop a rock solid migration strategy
Solution Components
Secure Agent
Enterprise Data
Catalog (EDC)
Intelligent Cloud
Services (IICS)
Azure SQL DW
On-Premises
EDWAvailable via the Azure Marketplace
12 © Informatica. Proprietary and Confidential.
EnterpriseData Catalog
14 © Informatica. Proprietary and Confidential.
Understand your data landscape with machine learning-based,data asset discovery and visibility
© Informatica. Proprietary and Confidential.
Classify your
data
Know more
about your dataShare your data
knowledge
15 © Informatica. Proprietary and Confidential.15 © Informatica. Proprietary and Confidential.
How can a data catalog help?
Self-service Discovery
for Analytics
Find and locate data assets
quickly and make sense of the
data in business context
Data Governance
Get inventory of your data
assets and make it available
for business
IT Impact Analysis
Get a clear, complete of picture
of data environment
16 © Informatica. Proprietary and Confidential.16 © Informatica. Proprietary and Confidential.
Enterprise Data Catalog for Everyone
How can IT enable
business to discover data assets with verified
data quality and traceability?
Data Architect
How can I search,
explore, understand and trust data required for my
analysis?
Data ConsumerData Steward
How can I manage
metadata for key enterprise data assets?
How do I manage the data lifecycle?
How can I make the
extract, transform, and load data flow for my data
warehousing projects v isible to others?
ETL DeveloperTechnical Data
Analyst
How can I
understand how data moves through my
application portfolio to my
data warehouses for analytics?
EDC
17 © Informatica. Proprietary and Confidential.17 © Informatica. Proprietary and Confidential.
• Easily find and discover trusted data
• Explore 360-degree data relationships
• End-to-End data lineage &
impact analysis
• Integrated Business Glossary
• Crowd-sourced enrichment
and auto-tagging of data assets
• Automatic Classification for
data domains
• Machine-learning-based data similarity recommendations
(CLAIRE)
Comprehensive Discovery and Visibility
to all data assets
Enterprise Information Catalog
18 © Informatica. Proprietary and Confidential.18 © Informatica. Proprietary and Confidential.
Entity Recognition
Street City State Zip
Address
First
Name
Last
Name
Customer
PrID Product Name
Product
DateAmount
Order
AI-driven, machine learning based techniques to identify, cluster and match similar columns and provide
recommendations for similar data sets
19 © Informatica. Proprietary and Confidential.19 © Informatica. Proprietary and Confidential.
Enterprise Data CatalogEnhanced Column Similarity
Unsupervised clustering of similar columns based on names, lineage, values and patterns
Enhanced Smart Domain Discovery based on new column similarity clusters.
20 © Informatica. Proprietary and Confidential.20 © Informatica. Proprietary and Confidential.
Enterprise Data CatalogOpen Metadata API
No metadata lock-in; any
metadata can be ingested and
accessed from EIC
Programmatic curation of data
assets to deal with metadata at
scale
Integrate with third party
applications search, lineage and
asset relationship services
Analytics on Metadata Repository
Access metadata knowledge
graphs with Open Metadata API
21 © Informatica. Proprietary and Confidential.21 © Informatica. Proprietary and Confidential.
Enterprise Data CatalogEDC Plugin for Tableau
Identify, understand metadata
associated with a Tableau
Report
Complete, governed and trusted
view of data assets
Tableau plugin connects to an
existing EIC deployment
22 © Informatica. Proprietary and Confidential.22 © Informatica. Proprietary and Confidential.
Enterprise Data CatalogInformatica Axon Integration
Determine the technical lineage
of specific data and surface this
in a business relevant context
Import business glossary and
classifications from Informatica Axon.
This is a two-way integration with easy
navigation to associated technical and
business assets.
Business
Glossaries from Informatica Axon
Links to EIC from
Axon
23 © Informatica. Proprietary and Confidential.23 © Informatica. Proprietary and Confidential.
Cloudera | Hortonworks | MapR | AWS
EMR | Azure HD Insight
Big Data
PowerCenter | DQ | MDM | BDM |
MM | BG | ILM | Axon
Informatica Cloud | DIH
Informatica
IBM DataStage | Microsoft SSIS
Oracle Data Integrator | Talend
Other ETL
CSV | XML | JSON | Avro
Parquet | Excel | PDF | PPT | DOC
Zip Files | SharePoint | OneDrive | ADLS
Azure Blob | AWS S3 | HDFS
MapRFS | Local
Files and File Systems
Oracle | DB2 | SQL Server | Sybase
Teradata | Netezza | MySQL
Greenplum | Azure SQL DB/DW
SAP HANA | AWS Redshift
Google BigQuery | JDBC
Databases
AWS | Azure | Google
Cloud Platforms
SAP | Salesforce | Oracle
Applications
Tableau | IBM Cognos |
SAP | BusinessObjects |
MicroStrategy OBIEE | QlikView
Business Intelligence
Unified Metadata
ENTERPRISE
`
Enterprise Data Catalog
Hands on Lab Workshop
25 © Informatica. Proprietary and Confidential.
Duration: 10 minutes
In this lesson you will learn how to search for relevant data assets using Search and Dynamic Faceting capabilities in the Catalog. You will also learn to explore associated Data Profiling statistics to determine the quality of the assets.
Objectives
• Find Data Assets
• Explore Data Profiling Statistics
Lesson 1: Data Discovery
26 © Informatica. Proprietary and Confidential.
Duration: 10 minutes
In this lesson, you will learn about data domain assets in Enterprise Data Catalog.
Objectives
• Review Data Domain
Lesson 2: Data Domain Curation
27 © Informatica. Proprietary and Confidential.
Lesson 3: Lineage and Impact Analysis
Duration: 10 minutes
In this lesson, you will learn how to use the new drill down lineage views in the Catalog to visualize data provenance. You will also learn how to use the detailed impact analysis reports in the catalog to understand impact due to change in data assets or ETL flows.
Objectives
• Understand Drill Down Lineage Views in the Catalog
• Perform Impact Analysis on Data Assets
• Understand Relationships
28 © Informatica. Proprietary and Confidential.
Duration: 10 minutes
In this lesson, you will learn how the Catalog automatically classifies data based on known domains. You will also learn how you can annotate datasets to further classify data assets along multiple dimensions.
Objectives
• Work with Semantic-Search
• Understand Crowd-sourced curation
Lesson 4: Data Classification
Coffee/Tea Break10 minutes
30 © Informatica. Proprietary and Confidential.
`
Intelligent Cloud Services
Hands on Lab Workshop
31 © Informatica. Proprietary and Confidential.
Informatica Intelligent Cloud Services (IICS)
Data
IntegrationApplication
IntegrationB2B
Integration
Hub
API
Management
Integration,
transformation and orchestration for powering data
warehouses and analytic workloads
API-first integration
that orchestrates, governs and
manages data and
application services
Automate
secure data exchange
across partner
networks
Automate
integration at mixed latencies and eliminate
point-to-point integrations
Gain visibility
and control of integration
and data APIs
32 © Informatica. Proprietary and Confidential.
Informatica Intelligent Cloud Services Architecture
Your Corporate Network
Cloud Applications
No staging required Data transmission is secure Multiple security certifications
Cloud Hosted AgentAgent Groups for High Availability
Agent GroupCloud Agent
Secure Agent
f irewall
Intelligent
Cloud Services
33 © Informatica. Proprietary and Confidential.
Duration: 15 minutes
In this lesson you will learn how to mass ingest files from remote servers to a cloud storage
Objectives
• Create mass ingestion Task to read data from the Flat file and load into Azure Blob. In this lab, will learn how to move Flat files from Linux machine or Ftp server to Azure blob storage.
Lesson 1: Mass Ingestion
34 © Informatica. Proprietary and Confidential.
Duration: 20 minutes
In this lesson, you will learn how to easily synchronize data from on-premises database to a cloud data warehouse.
Objectives
• Create Synchronization Task to read data from the Flat file and load into Azure SQL DW.
Lesson 2: Data Synchronisation
35 © Informatica. Proprietary and Confidential.
Lesson 3: Working with semi-structured data
Duration: 20 minutes
In this lesson, you will learn how to load a JSON file to a cloud data warehouse using Informatica’s Intelligent Cloud Services.
Objectives
• Understand how to handle semi-unstructured data
• Data Transformations deep dive
36 © Informatica. Proprietary and Confidential.
Duration: 30 minutes
In this lesson, you will learn how build commonly known data warehouse patterns using cloud data integration
Objectives
• Create a slowly changing dimension mapping that reads data from Oracle source and load into Azure SQL DW.
Lesson 4: Common Data Warehouse Patterns
37 © Informatica. Proprietary and Confidential.
Duration: 30 minutes
In this lesson, you will learn how to control the execution sequence using taskflow.
Objectives
• Create Taskflow to execute previously created Mapping and Synchronization tasks.
Lesson 5: Task Orchestration
Q & A
Lunch
Thank You