off the treadmill: building a drupal platform for your organization

Off the Treadmill: Building a Drupal Platform for Your OrganizationRick Vugteveen (@rickvug)Pacific Northwest Drupal Summit 2013

Wednesday, 9 October, 13

Hello!

• @rickvug on Twitter, Drupal.org etc.

• Drupal and I have history: Dev, Business Dev, Technical Architect, Product manager.

• Currently work in Solutions Architecture at Acquia.


What will we be talking about?

• Code management strategies. Understanding your options and trade-offs.

• How to structure your code into re-usable Platform functionality vs. Site specific code.

• How to deal with problems with Features.

• Consistency and upgrade path issues.

• Whatever you want!


Things I like but don’t expect to talk about

• How branching works or other Git fundamentals

• Drush Make

• The basics of Features

• Strongarm

• Exportables


Setting the stage


• Duplication of content"

• Disparate sources"

• Disconnected reporting"

• Islands of excellence"

Inside Many Large Organizations, the Web Is Broken


Assemble solutions in Drupal

Drupal core! Drupal + recommended!modules!

Solution Distributions!


The Dream

Department sites!

Data portals!Primary site!

Microsites!

Collaboration Intranet !

Departmental Sites!


Does this sound familiar?

• “All of our Drupal sites have slight differences. It is difficult to keep track of the differences and maintain them all.”

• “I feel like we keep on developing the same functionality over and over again.”

• “Bug-fixing and fire-fighting never ends. We need to get back to creating new business value”.

• “We don’t have time to build things the right way”.


Code management & deployment strategies


Site (Yellow):/sites/foo/modules/*/sites/foo/themes/*/sites/bar/modules/*/sites/bar/themes/*

Platform (Brown):/profiles/foo/modules/*/profiles/foo/themes/*

Core (Blue):/modules/*/themes/*/includes... etc.

DrupalCore

Platform Theme, Modules & Features

Site specific Theme,

Modules & Features


Multi-site model

SiteB

SiteA

SiteC

SiteD

Git Repo


Vendor branching model

SiteA

DrupalCore

Features

Site Code

SiteB

SiteC

SiteD

Git Repos

Sites

Platform


SiteB

SiteA

SiteC

SiteD

SiteA

SiteB

SiteC

SiteD

DrupalCore

Features

Site Code

Drupal Multisite Architecture✓ Makes it easy to keep core and platform changes

in sync among many sites- Makes it really hard to test and assure quality on

core or platform changes that may a!ect tens or hundreds of sites

- Is di"cult to divide responsibility among di!erent teams of developers and testers

Independent Drupal codebases✓ Lets each site be tested, QA'd individually at their

own pace✓ Lets independent teams work on individual sites

without breaking others- Can lead to forking of core and platform code- Is di"cult to maintain

Lots of Code, Lots of Sites, Lots of Compromises


Mmmmm... Cake

Site Stuff

Platform Stuff

Core Stuff

▪ Segregate responsibility for site-speci!c development, platform development and QA

▪ Let dev teams use the tools they're used to (git, Github, Drupal)

▪ Combine the bene!ts of multisite and independent codebases


How does it do it?

Dev Stage Prod

Platform/SitePush to production

(manual and automated)

SiteA

SiteB

SiteC

SiteA

Site CodePublishingProcess

Hosting Environments

Git Repos

Git Repos

DrupalCore

Features

Site Code

Managed Cloud

Distribution Environment

Site A Setup

Tag

Platform 10Platform

1.4.3

Site Tag 4.3.6

TargetTag newfeaturebranchTarget

Repo Dev 3

Code Distribution Management

Platform 10

Platform 11

Platform 12

Platform 10


Who does Layer Cake work for?

Site DeveloperCommits themes,

modules to Site repo

Platform DeveloperCommits changes to

repos managing Drupal core, platform features and modules

Dev EnvironmentsDeveloper sandbox

environments

Production HostingProduction

environment(s)

Platform AdminManages available

deployment environments, setup

QA TeamTests and pushes approved

code to production, for 1 site or 100's


Developers!

DevelopersCommit to repos

they've been granted access to in Github

Developer's personal sandbox where they can test against replica of production stack

Use Layer Cake to deploy their repos

periodically to sandboxes


Admins!

Developer CSandbox

Production HostingProduction HA environments

Platform AdminSets holds the keys to

creating new mappings

Site A Setup

Tag

Platform 10Platform

1.4.3

Site Tag 4.3.6

TargetTag newfeaturebranchTarget

Repo DevCloud 3

Developer ASandbox

Developer DSandbox


Quality Assurance!

Production HostingProduction HA environments

QA TeamTests and pushes approved

code to production, for 1 site or 100's

Platform 10b

Keyword text

Filter Sites

Deploy Stage -> Prod

Site Version

4.5

7.2

3.1

1.0

134.2

Site

Site B

Site D

Platform Version

10b

Site C

Site E

10b

Site A

10b

10b

10b

Deploy Cancel


Site/Code management gotchas

• Testing the upgrade path on all sites. Shoot for eventual consistency but beware of the dangers of falling out of sync.

• Watch out for update.php, or what we call “the Upgrade Dance”.

• When using the Layer Cake approach beware Platform and Site compatibility issues.


Flexibility vs. Effort

• Multi-tenant: One database, one Drupal installation (ie. Domain Access).

• Hybrid: Multiple databases, one Drupal installation. (Multi-site, potentially a true Distribution)

• Multi-instance: Multiple databases, multiple Drupal installations. (potentially Vendor Branching)


But I’m already a Cowboy!


Baby steps

• Do an inventory. How many sites do we have? How similar are they? Are we looking at one platform or many?

• You can start just by moving modules around.

• Then start standardizing the list of modules you are using.

• Once modules are standardized, start capturing yourcommon configurations into Features.


Features. Love them. Hate them. Use them.


Where I’m coming from

30 magazines, up to 60 sites (long term), ~10 legacy content management systems.

Goal: One Drupal Platform.


Dennis Publishing Example

• Dennis Core:

• Base theme

• Media Management

• WYSIWYG

• Image cache settings

• Contexts (or Panels)

• Base Menus


Dennis Publishing Example

• Optional:

• Article

• Gallery

• Carousel

• Video player


Overridden Features

• Problem: Feature overridden due to configuration changes done in dev, stage or prod.

• Fix: You’re doing it wrong!

• Adopt a proper dev-> stage -> prod workflow.

• Automatically revert Features on deployments. Script drush fr.

• “drush fu” your workflow. (best Drush command ever!)


Overridden Features because of site differences

• Problem: I want to set defaults but I don’t need them enforced on each site.

• Example: Comment settings should have a default but may change on a per site basis.

• Potential Solution:

• Don’t be afraid of .install files, hook_update_N(), variable_set and friends. Only Strongarm variables you don’t want to change.

• Or structure your Features differently...


Structuring Features

• Make your Features granular with a proper dependency chain.

• Example: Blog Feature and Gallery Feature both depend on the Categories vocabulary. Create a Categories Feature with only that vocabulary. Have other Features depend on that.

• Capture your data structure in your Platform. Move display setting into a Site modules.

• Example:/profiles/distroname/features/distroname_blog_structure/sites/sitename/features/sitename_blog_display(or have shared base fields as their own Feature)


What the hell is a Skeleton Feature?

• Located at /profile/dennis_distro/modules/skel/*

• Naming convention: skel_blog_display, skel_gallery_display etc.

• 1. On distro install automatically copy over skel modules from profile/platformname/modules/skel/* to sites/sitename/modules/features/*.

• 2. Automatically find and replace “skel” with “sitename”. Eg. changes hook implementation from skel_blog_display_menu() to sitename_blog_display_menu().

• 3. hook_info_alter() any modules with the prefix “skel_”. You’ll never see them and Features will ignore.

• After install: /sites/sitename/modules/features/sitename_blog_display.Wednesday, 9 October, 13

Features upgrade path issues

• No silver bullet for data structure changes. Avoid at all costs. Get your hands dirty if needed.

• Start with multiple implementations of a Platform Feature to ensure the structure is generic enough.

• Push back on asks for site specific data structure. Get creative with Taxonomy.

• Adding to data structure on a per site basis is fine.


Wrapping up. What we (hopefully) learned

• Code management strategies. Understanding your options and trade-offs.

• How to structure your code into re-usable Platform functionality vs. Site specific code.

• How to deal with problems with Features.

• Consistency and upgrade path issues.


Additional Resources

• Phase2 Blog post on Features Base Fields: http://www.phase2technology.com/blog/new-field-bases-and-instances-in-features/

• Patch for Features Base Fields: https://drupal.org/node/1064472

• Multi-headed Drupal: http://palantir.net/blog/multi-headed-drupal

• The Drupal-Powered Enterprise White Paper: https://www.acquia.com/resources/whitepapers/drupal-powered-enterprise


http://www.phase2technology.com/blog/new-field-bases-and-instances-in-features/






https://drupal.org/node/1064472

https://drupal.org/node/1064472

http://palantir.net/blog/multi-headed-drupal

http://palantir.net/blog/multi-headed-drupal

https://www.acquia.com/resources/whitepapers/drupal-powered-enterprise




Thanks!

Rick Vugteveen@rickvug

[email protected]@acquia.com

PS - I’m hiring! Let’s talk.


mailto:[email protected]




off the treadmill: building a drupal platform for your organization

Technology

site c site d site

site tag

site developer

multisite model site

automated site

site yellow

primary site

site repo platform developer