Migrating to Drupal 8How to Migrate Your Content and Minimize the Risks

Acquia Webinar, 11-May-2015

The 5 phases of migration

→ Auditing the source site→ Planning the new data model→ Prepping for migration → Planning the migration→ Launching the migration

Phase 1: Source site audit

Taxes? Nope, understand your beginnings.

Know your source site

Primary for a smooth migration:→ What does my site contain? (data structures)→ What does my site use? (contrib and custom modules)→ What does my site look like? (contrib or custom

themes)

Simple or Complex

Questions to ask yourself: → Are you looking to essentially migrate everything?→ How complex do you believe your site to be?→ Do you have time to try a basic and direct migration?→ Do you have cleanup you wish to perform?

Data structure: bundles, fields, entities

→ Content types→ Fields (migrate_tools’ migrate-fields-source Drush

commands can help with this)→ Menus→ Vocabularies→ Taxonomy terms→ Users

Visualize your entities and fields

Features/Modules→ Advanced search

▪ i.e. Search API + Search API Solr + Acquia Search→ External Authentication

▪ SimpleSAMLphp, LDAP, OAuth→ Rebuilding views used in D6 or D7

▪ Manual process▪ Missing contrib modules?

→ Commonly used modules: ▪ Pathauto, Redirect, Purge, et cetera.

→ Simple Sitemap module to generate Sitemap.xml file

Are the contrib modules required for your site available yet on D8? → Check out Drupal 8 Contrib Porting Tracker

▪ https://www.drupal.org/project/contrib_tracker

More Questions

→ Are the skills available to rewrite custom modules?→ Are those custom modules still necessary or does D8

introduce better solutions in Core or Contrib?→ Is there a cutover window?→ Is new content being added and are site alterations

being made while the site content is being migrated?

Discuss some of the challenges

→ Learning D8▪ We started assembling module building lessons:

• https://docs.acquia.com/articles/building-drupal-8-modules

→ Find skilled assistance to fill skill gaps→ Identify impacted stakeholders

▪ Including stakeholders in the launch and post-launch process

▪ Insuring they are there to test.▪ Insuring continued buy-in and collaboration

Phase 2: Data model planningCongrats!You’ve chosen a complex migration/cleanup

Plan, consult, resolve, goto 10

10 Plan out what content you expect to see20 Consult with interested parties [time boxed]30 Resolve any issues in your model40 goto 10

What’s in a name?

When building it’s important to communicate as much as possible, as concisely as possible with structure.

→ i.e. Authors: field_article_author became field_authors→ i.e. Drupal Projects: field_terms_project became

field_projects_drupal

Often there isn’t a single right answer, consistency is key.

Issues we encountered

1. Typos and misspellings in source2. Merging fields3. Processing fields

Review Date (date-time)→ field_date_review→ field_review_date (content type A)

▪ date→ field_revew_date (content type B)

▪ date-time▪ required using hours:minutes + date▪ possible bug requiring fix?

Trouble merging two similar fields

Simple wins

1. Coordinating the machine names2. Type and spelling fixes in machine names3. Opportunity to change data-type

Phase 3 - Migration prep or:How We Learned to Stop Worrying and Love the Migrate API

What’s in core today?

A quick overview of contrib modulesModule Description

migrate_upgrade Drush commands for running upgrades from Drupal 6 or 7 to Drupal 8

migrate_tools General-purpose Drush commands and basic UI for managing migrations

migrate_plus Extends the core migration framework with additional functionality (e.g. migration from XML and JSON sources)

migrate_ui UI on top of Migration entities in core

migrate_manifest Drush command for running template-based SQL migrations from a manifest file

Dataset and data model, please.→ Are we migrating one thousand or one million items?→ Do you have any data model change?→ Migrating via the UI is almost never a good solution

▪ Doesn’t allow for custom migrations▪ Drush scales better▪ Contrib has powerful and flexible Drush commands

Get started with migration modules$ drush @site.env en -y migrate_drupal, migrate_upgrade, migrate_toolsThe following extensions will be enabled: migrate_drupal, migrate_upgrade, migrate_tools, migrate, migrate_plus

→ Use branch 8.x-2.x for all migration modules→ Get the DB connection settings for the source site→ Get familiar with the new Drush migration commands

$ drush | grep migrate

Configure your initial migration$ drush @site.env migrate-upgrade --legacy-db-url=mysql://{USER}:{PASSWORD}@{HOST}:{PORT}/{DB} --legacy-root={URI} --configure-only

→ All necessary YAML files should now be exported→ Audit with Drush Configuration Management commands

$ drush config-list | grep -c migrate62

Confirm your migration key is set>>> \Drupal::>>> \Drupal::state()->get('migrate_drupal_6');=> [ "key" => "upgrade", "database" => [ "driver" => "mysql", "username" => "{USER}", "password" => "{PASSWORD}", "host" => "{HOST}", "port" => "{PORT}", "database" => "{DB}", "prefix" => null, ], ]

Run a full migration

$ drush @site.env migrate-upgrade --legacy-db-url=mysql://{USER}:{PASSWORD}@{HOST}:{PORT}/{DB} --legacy-root={URI}

→ Straightforward migration...but not extremely flexible→ You probably need control at all levels of the migration→ Enter the migrate-import Drush command

Create a custom module (it’s easy!)→ Create your *.info.yml file→ Add your exported migration files to config/install→ Enable your module. That’s it.

$ drush @site.env config-export -y sync ; cp /path/to/config_dir/sync/migrate_plus.migration* /path/to/custom/module/config/install/

Run a custom migration$ drush @site.env migrate-import --group=migrate_drupal_6 --limit=10000 --feedback=1000

$ drush @site.env migrate-import d6_url_alias,d6_block, d6_file,d6_user,{...} --limit=10000 --feedback=1000

→ Use --limit to limit on the number of items to process→ Use --feedback to be notified of progress→ You can even arbitrarily restrict by source plugin

Migration status and messages$ drush @site.env migrate-status

$ for i in upgrade_d6_node_type upgrade_d6_user {...} ; do drush @site.env migrate-messages $i ; done

→ Use migrate-status to have an overview of (last) imported, unprocessed and total number of items, etc.

→ Use migrate-messages to catch migration errors

What about really custom migrations?→ It still mostly is the Migrate you know and like

▪ prepareRow()▪ query()▪ ...

→ Extend default core Migrate source plugins→ Write your own source Plugins

Sample custom source Plugin<?php

namespace Drupal\migrate_webinar\Plugin\migrate\source;

use Drupal\path\Plugin\migrate\source\d6\UrlAlias;

/** * @MigrateSource( * id = "webinar_d6_url_alias" * ) */class WebinarUrlAlias extends UrlAlias {

public function query() { $query = parent::query(); $query->condition('src', 'user/%', 'NOT LIKE'); return $query; }

}

Core source Plugin

Plugin annotation

Custom logic via the query() method

Migration tips and tricks→ The Migration API is still in flux. Check drupal.org!

▪ Major changes in Drupal 8.1.x - Check http://mikeryan.name/blog/mikeryan/migration-update-for-drupal-8-1

→ See a d6 or d7 string in a migration filename?▪ It’s specific to this Drupal version▪ If you don’t see anything, it works for both

→ Start backwards with custom migrations▪ Use a minimalistic set of migration files first▪ Add the required dependencies as you go

Phase 4: Migration planning & strategy

The devil is in the...

Stages of migration

1. Initial development - Used Dev and Test2. Working through the migration bugs3. Prepping prod (check and triple check)4. Code “almost ready” - test migration manually5. Switch to continuous content migration (failed)

▪ dev for active migration work▪ stage for QA▪ prod for lead up to launch

6. Test until your eyes and fingers bleed (joke, !joke)

Why continuous content migration

→ This allows code fixes to be rolled into place until very shortly (48 hours) before launch.

→ Why did we fail?▪ Tokens in content▪ Most in place but some had to be manually moved

→ Result -> migration was semi-continuous but we needed longer window for move (one week).▪ If “no-go called” -> redo the manual migration▪ Generated pressure

Continuous migration of content

$ ssh -i /home/{USERNAME}/{SSH_KEY} -f -L 1111:{DB_SERVER}:3306 {USER}@{WEB_SERVER} -N

→ Can you afford a hard cutover/downtime?→ Use SSH tunneling for maximum security

→ https://docs.acquia.com/articles/ssh-tunneling-server-side-applications

Now’s your chance

Take the migration to D8 as an opportunity to get rid of bad old habits!

→ PHP filter→ Full HTML→ Improve editorial Workflow

▪ Multiversion (must be in place prior to migration)▪ Workbench Moderation

However, Don’t fall down the rabbit hole!

Phase 5: LaunchWhere the rubber meets the road!

Plan for go/no-go decision

→ This is probably one of the hardest processes to plan.→ Complete buy-in from stakeholders is needed.→ Clearly record and communicate.→ Set a reasonable and firm timeline.→ Set the conditions for why to delay.→ If possible stick to the timeline when launching.→ Try to mitigate risk where possible

▪ sometimes this means delaying a launch▪ sometimes this means going and fixing post launch

Our plan: what worked

→ Content Freeze→ Window for migration→ Focusing on content as the priority→ Iterative development with Minimal Viable Product

▪ identifying key needs▪ addressing non-critical path issues after launch

→ Replaced a lot of older code just by using D8 core

Our plan: what didn’t work

→ Misaligned expectations▪ incomplete go/no-go plan

→ Working against a hard deadline (Drupal 6 EOL)→ Not enough time planned for encountering bugs in

custom migration→ Missing contrib modules→ Finding time to contribute to porting modules

Ensure assets consistencymysql> SELECT COUNT(fid) FROM file_managed;

$ find . ! -path "*styles*" ! -path "*ctools*" ! -path "*css*" ! -path "*js*" ! -path "*xmlsitemap*" -type f | wc -l

$ rsync -avz --progress --exclude=styles --exclude=ctools --exclude=css --exclude=js --exclude=xmlsitemap -e "ssh -i /home/{USERNAME}/.ssh/id_rsa" {SOURCE_SERVER}:{SOURCE_PATH} .

→ Not all files are necessarily managed by Drupal→ Use rsync when the Migrate API can’t help

We hope you learned about...

1. How to audit: bundles, fields, and entities 2. Planning steps: stay, go, and modify3. Migrate API and contrib migration modules4. Process: near-continuous migration of content to

minimize downtime5. Effectively creating, implementing a risk mitigation plan

Thank You - Questions?