migrating data into drupal using the migrate module
DESCRIPTION
Migrating data into Drupal using the migrate module presentation at Oxford Drupal Camp, 22 June 2012TRANSCRIPT
Oxford DrupalCampFriday 22 June, 2012
Migrating data into Drupal using the migrate module
Johan Gant
What's involved?
Migrate me an army of data!
Uh, oh!!
Planning/analysis
AcceptanceTech/dev work
What's involved?
Done!
Successful migrations
Move data from A to B in a planned, methodical manner.
RepeatableMeasurableWork with realistic data
Safe
Use the right tools for the task
The complexity of your solution should be driven bythe task at hand, not your tech
When stuff goes wrong...Un...
Safe
Reliable
Planned
Done
RAGEBALLS FOR EVERYONE
Upset...Processes
Clients
Developers
Drupal migrate module
Provides a great framework for moving content into Drupal
Encourages good habits
Drush support
migrate_ui adds a nice front end
The Drupal way – so other Drupal devs can
pick up your work without too much bother.
Let's look at some code
Not a one stop shop for migrations
Use the right tools, or combination of tools, to get the job done.
<?phpclass DemoMigration extends Migration { public function __construct() { parent::construct(); $this->description = t('A sample migration module');
// Define the map between your source and destination $this->map = new MigrateSQLMap( $this->machineName, // defaults to your migration class name array( 'id' => array( 'type' => 'int', .... ), ), MigrateDestinationNode::getKeySchema(); );
// Define the fields from your source $source_fields = array( 'id' => t('description'), 'field1' => t('Node title'), ..... );
$query = db_select('source_table_name', 'sql-alias') ->fields('alias', array_keys($source_fields)) ->orderBy('id', 'ASC');
$this->source = new MigrateSourceSQL($query);
// Define what sort of destination you have // - options include Nodes, Terms, Users and Comments $this->destination = new MigrateDestinationNode('node_machine_name');
// Define your field mappings - nice options include default values (can include PHP), // groupings, callbacks – see beer.inc example class for details $this->addFieldMapping('source_body', 'field_body') ->defaultValue('Wibble'), ..... }}
http://drupal.org/node/1528934
/** * Useful function to help 'massage' your awkward source data into shape */public function prepareRow($row) { // Convert ISO date to UNIX timestamp if ($row->created_at) { $row->created_at = strtotime($row->created_at); }}
Handling taxonomy terms
/** * Term syntax/format a bit awkward,depends on whether you're using tid or name. * Migrate module will match on tid or name and handle the rest for you :) */$this->addFieldMapping('source_col_name', 'terms') ->arguments(array('source_type' => 'name')), // use tid if you've got term ids ->separator('$$'); // Used to split long string of term names in source
Handling node reference fields
/** * Not documented when I used migrate, but expects a nid */if ($row->some-identifier) { $row->reference_nid = my_own_function_to_lookup_a_nid($row->some_identifier);}
Migration!
Rails/PostgreSQL > Drupal 6
Export data into CSV, import into Drupal db via table wizard module
Pre-migration script to create image nodes
Run migration
Rails/PostgreSQL > Drupal 6
Client demo and deployment
5 days from start to finish
Problems!
Steep(ish) learning curve
Documentation a bit sparse in places
Awkward handling of taxonomy,node reference, and domain data
Pre-migration fudges
Performance
Should I use Drupal migrate?
● Want to move a reasonable volume of data INTO Drupal from MySQL/XML/JSON/CSV
● Have complex data mappings that are best expressed programmatically
● Make best use of Drupal tools and existing code
● Want to recycle code between projects
● Low volume or low complexity
● Not familiar with coding
● Already got something that works well
● Trying to move data OUT of Drupal to somewhere else
Yes / absolutely / do it now Probably not
Drupal migrate - resources
● Migrate module page -http://drupal.org/project/migrate
● Economist migration (great summary) - http://drupal.org/node/915102
● Torchbox Team Drupal blog - http://drupalblog.torchbox.com/