migrating to snowflake blast out of the ......t3s2 - migrating to snowflake2-final created date...
TRANSCRIPT
© 2019 Snowflake Inc. All Rights Reserved
MIGRATING TO SNOWFLAKEBLAST OUT OF THE STARTING GATE
DANNY BRYANT, SENIOR SOLUTION ARCHITECT
© 2019 Snowflake Inc. All Rights Reserved
AGENDA
2
Migration Basics
Migration Strategy
How Snowflake Helps Our Customers Migrate
© 2019 Snowflake Inc. All Rights Reserved
Migration Basics
© 2019 Snowflake Inc. All Rights Reserved 4
How much does it cost?
TOP 3 QUESTIONS ABOUT MIGRATION TO SNOWFLAKE
© 2019 Snowflake Inc. All Rights Reserved 5
How much does it cost?
How long will it take?
TOP 3 QUESTIONS ABOUT MIGRATION TO SNOWFLAKE
© 2019 Snowflake Inc. All Rights Reserved 6
Have other customers done what we’re
trying to do?
How much does it cost?
How long will it take?
TOP 3 QUESTIONS ABOUT MIGRATION TO SNOWFLAKE
© 2019 Snowflake Inc. All Rights Reserved 7
Have other customers done what we’re
trying to do?
How much does it cost?
Costs depend on the size and scope of what is migrated from the
existing solution to Snowflake
How long will it take?
TOP 3 QUESTIONS ABOUT MIGRATION TO SNOWFLAKE
© 2019 Snowflake Inc. All Rights Reserved 8
Duration depends on the size and scope of what is migrated from the
existing solution to Snowflake
Have other customers done what we’re
trying to do?
How much does it cost?
Costs depend on the size and scope of what is migrated from the
existing solution to Snowflake
How long will it take?
TOP 3 QUESTIONS ABOUT MIGRATION TO SNOWFLAKE
© 2019 Snowflake Inc. All Rights Reserved 9
Duration depends on the size and scope of what is migrated from the
existing solution to Snowflake
Have other customers done what we’re
trying to do?
How much does it cost?
Costs depend on the size and scope of what is migrated from the
existing solution to Snowflake
How long will it take?
Customers have successfully migrated from other data warehouse platforms to
Snowflake, but each customer is different
TOP 3 QUESTIONS ABOUT MIGRATION TO SNOWFLAKE
© 2019 Snowflake Inc. All Rights Reserved
TOP 3 QUESTIONS ABOUT MIGRATION TO SNOWFLAKE
10
The answer is…it depends!
Duration depends on the size and scope of what is migrated from
the existing solution to Snowflake
Have other customers done what we’re trying to do?
How much does it cost?
Costs depend on the size and scope of what is migrated from the
existing solution to Snowflake
How long will it take?
Customers have successfully migrated from other data warehouse platforms to
Snowflake, but each customer is different
© 2019 Snowflake Inc. All Rights Reserved
Migration Strategy
© 2019 Snowflake Inc. All Rights Reserved
THE NEED FOR MIGRATION STRATEGY
12
Migration Planning A successful migration begins with planning that is led by Snowflake PS or a SI partner to accurately scope the migration.
Code Conversion DDL, procedures, and scripts must be translated to be valid in Snowflake. Code Conversion effort is typically the largest unknown in scoping migrations.
Data Migration Data migration is the movement of an initial dataset from the legacy system to Snowflake. Effort in this phase varies based upon the number of tables and the method for moving the data.
Data Ingestion Data ingestion is the movement of data changes from the source system to Snowflake. Effort in this phase varies based upon the number of data sources and the method for moving the data.
Data Validation Side by side comparison of the legacy data results and Snowflake results. Verifies data was moved successfully and that any translated logic is giving the intended results.
Reporting & Analytics Tools
Integrations for reporting and analytics tools must be modified and reconnected. This can be very involved for some systems where objects must be re-architected.
Security Implementation
Evaluate legacy security requirements, architect and implement the new security plan including SSO, roles, and data level security.
© 2019 Snowflake Inc. All Rights Reserved
3 QUESTIONS THAT SHOULD BE ASKED ABOUT MIGRATION TO SNOWFLAKE
13
What support is available to help with the migration to
Snowflake?
What really needs to be migrated to
Snowflake?
What are the common characteristics of successful
migrations to Snowflake?
© 2019 Snowflake Inc. All Rights Reserved
• List databases and database objects to migrate
• List processes and tools that populate and pull data from the existing data warehouse
• List security roles, users and permissions
• Documentation of the existing data warehouse solution into an as-is architecture diagram
14
WHAT NEEDS TO BE MIGRATED – DOCUMENT THE EXISTING SOLUTION
© 2019 Snowflake Inc. All Rights Reserved
• Categorize processes to migrate as-is, processes that need reengineering, processes that need fixing, and processes that are no longer needed
• Draft migration deliverables
• Create to-be architecture diagram
15
WHAT NEEDS TO BE MIGRATED – ESTABLISH A MIGRATION APPROACH
© 2019 Snowflake Inc. All Rights Reserved 16
WHAT NEEDS TO BE MIGRATED –MODERNIZATION CONSIDERATIONS
• Migrating to Snowflake provides an opportunity to reevaluate the design of the data warehouse solution
• Consider the tradeoffs between refactoring parts of the previous data warehouse solution and the downstream impacts of that refactoring
• Scalability limitations that existed in the previous data warehouse solution are mitigated in Snowflake which requires a new approach to capacity utilization
© 2019 Snowflake Inc. All Rights Reserved
• List data sets to migrate first
• Create method for identifying process dependencies for data sets
• Document process dependencies for data sets
17
WHAT NEEDS TO BE MIGRATED – PRIORITIZE DATA SETS FOR MIGRATION
© 2019 Snowflake Inc. All Rights Reserved
• Completing a migration requires time, money and energy, so you need a compelling reason to migrate
• Having a compelling reason to migrate keeps you moving forward if challenges arise with the migration
• Completing the migration brings a sense of accomplishment that something important has been achieved
18
CHARACTERISTICS OF SUCCESSFUL MIGRATIONS –MOTIVATION
© 2019 Snowflake Inc. All Rights Reserved
• Determine the commitment of decision makers to providing the necessary funding
• Document budget allocated for the migration project
• Work with the Snowflake sales team to estimate the costs to run Snowflake and the Snowflake Professional Services team to estimate the cost of the migration
19
CHARACTERISTICS OF SUCCESSFUL MIGRATIONS –FINANCIAL COMMITMENT
© 2019 Snowflake Inc. All Rights Reserved
• Need automated repeatable processes to minimize the amount of time required to migrate to Snowflake
• Snowflake has solution partners with migration tooling to help our customers with their migrations
20
CHARACTERISTICS OF SUCCESSFUL MIGRATIONS –MIGRATION TOOLING
© 2019 Snowflake Inc. All Rights Reserved
SUPPORT FOR MIGRATING TO SNOWFLAKE
21
Our Snowflake Solutions Partners have experience migrating customers from other Data
Warehouse platforms to Snowflake
Professional Services has a scalable, repeatable methodology and tools to
help our customers with their migration to Snowflake
Solutions PartnersDocumentation
Migration Reference Manuals available for Teradata, Oracle,
and Netezza on https://resources.snowflake.com/
Snowflake Migration Team
© 2019 Snowflake Inc. All Rights Reserved
How Snowflake Helps Our Customers Migrate
© 2019 Snowflake Inc. All Rights Reserved
IS THERE A MIGRATION EASY BUTTON?
23
© 2019 Snowflake Inc. All Rights Reserved
THERE IS A MIGRATION EASIER BUTTON
24
© 2019 Snowflake Inc. All Rights Reserved
SNOWFLAKE CONVERSION SERVICES
25
We are looking for customer partnerships to develop additional sources!
TablesViews
DML StatementsUser Defined Functions
Stored ProceduresBTEQ
TablesViews
DML StatementsUser Defined Functions
Stored Procedures
© 2019 Snowflake Inc. All Rights Reserved
ACCELERATE MIGRATIONS TO SNOWFLAKE!
26
Teradata, Netezza, and others
Automated Code Translation
Recent success for a national grocery chain:1.5M lines of SQL code 98% converted in one week
PARTNER HIGHLIGHT:
© 2019 Snowflake Inc. All Rights Reserved
HOW SHIFT ACCELERATES MIGRATIONS
27
X-Factor Cases (e.g. ETL, Scripting Frameworks, Proprietary Functions)
Stored Procs
SQL
DDL
LEGACY DATA WAREHOUSE ENVIRONMENT
Syntax-based Translation
(e.g. data type conversion, expression
conversion)
Semantic-based Translation
(e.g. PL/SQL, code optimization)
© 2019 Snowflake Inc. All Rights Reserved
AUTOMATED ANALYSIS AND CODE TRANSLATION
28
© 2019 Snowflake Inc. All Rights Reserved
MIGRATION READINESS ENGAGEMENTDelivery of a migration plan to a customer
29
• 1 or 2 week, depending on code conversion approach, onsite paid Professional Services engagement to provide a detailed migration plan for customers across all phases of the migration
• Cost - $20K (1 week) or Custom SOW (2 weeks)
What it is:
• Completed migration questionnaire
• Customer DDL and DML scripts
• Customer subject matter experts for the 1 or 2 week on-site engagement
What is Required:
• Customer receives a full migration plan including cost estimation, migration timeline, resource plan, and detailed list of migration steps for code conversion, data migration, data ingestion, data validation, and reporting & analytics tools
• Customer determines if the migration plan is based on a SI, Snowflake Professional Services, or a combination of both executing the migration
Output & Value
© 2019 Snowflake Inc. All Rights Reserved
SUMMARY
30
• The cost and timeline for migrating to Snowflake varies based on each customer’s requirements
• Migrating to Snowflake is most successful when using a planned out migration strategy that leverages as much automation as possible
• Snowflake has migration services to help our customers migrate successfully
© 2019 Snowflake Inc. All Rights Reserved
Thank You