keep drupal running with continuous integration
DESCRIPTION
Keeping a software project running is a difficult art. It's fine at first, when the site is in beta and you're happily flipping switches on Drupal's control panels. But soon the site goes live, and downtime costs both money and goodwill, and MySQL fills up with customer data that must not be lost or corrupted, and the code develops multiple branches that are developed and tested on multiple machines, and are edited by programmers who must avoid stepping on each other's work. This is an introduction to the principles of Continuous Integration (CI), a popular strategy for software projects that attempts to minimize the risk, uncertainty, and effort of releasing new code. We'll discuss: - Why managing a production site is so much more irritating than building a beta site from scratch, and how CI can ease the pain. - The central goal of CI: That every project should have a single main branch that is always ready to deploy, passing all tests, and synced with each developer's work. - Typical strategies for achieving this goal – dedicated integration servers; routine testing of each patch at several stages; frequent commits by all stakeholders; rapid, automated testing and deployment; testing on clones of the production stack – with notes on how these may apply to Drupal sites. This is a beginner to intermediate session, requiring no coding experience, though a familiarity with the dynamics of software maintenance will help to inspire you. Mike Booth, is a former devops engineer and the original release manager for Acquia Cloud Drupal hosting, and as such has had a meaningful relationship with Continuous Integration. Though his cardiologist cannot confirm it, Mike believes that CI may have saved his life. War stories will be told as time permits.TRANSCRIPT
Keep Drupal Running With Continuous
Integration
Mike BoothNYC Camp 2012
Saturday, July 21, 12
“Why can’t we edit our settings directly
on the production server?”
Saturday, July 21, 12
Why We Don’t Just Develop On Production Servers
Visitors dislike broken thingsHard downtimeData corruptionRollbacks are slow & they lose new dataNo audit trail
Hard to document changesHard to discuss changes
Saturday, July 21, 12
Hacking On Live Websites Is Awesome
Focus
One place to goOne set of knobs to turn
Short Cycle Time
Everyone looks for bugsBugs triage themselvesApplause arrives in real time
Saturday, July 21, 12
It’s So Awesome, We Maintain Dozens of Control Panels For It
Saturday, July 21, 12
Hacking On Live Websites Would Be Awesome
If Only It Didn’t CauseExpensive Disasters
Saturday, July 21, 12
“What’s the next best thing?”
Saturday, July 21, 12
What we actually do
Make copies of the site and develop with them
Copy the code
Copy or simulate DB and files
Run on a separate dev server
Saturday, July 21, 12
The Problem
Integration(The art of resolving
inconsistencies.)
Saturday, July 21, 12
Integration1. Everyone works on a copy.2. Announce a “freeze”.3. Collect copies and mix together.4. Test, find bugs, point fingers, fix bugs. 5. Iterate Step 4 until confident.6. Cross fingers. Upload result to production. 7. Fix more bugs, urgently.8. Go to Step 1.
Saturday, July 21, 12
Continuous Integration
Do this several times a day.
Saturday, July 21, 12
The Story of CI
1. Go to main branch & check out working code.2. Build a patch that passes all tests.3. Update from main branch, make tests pass.4. Commit to main branch ASAP.5. Make tests pass on an integration machine.6. [Optionally, release to production: see Continuous Deployment]
7. Go to Step 1.
Saturday, July 21, 12
Integration1. Everyone works on a copy.2. Announce a “freeze”.3. Collect copies and mix together.4. Test, find bugs, point fingers, fix bugs. 5. Iterate Step 4 until confident.6. Cross fingers. Upload result to production. 7. Fix more bugs, urgently.8. Go to Step 1.
Saturday, July 21, 12
Use version controlPut everything in there.
But don’t use too much version control:
Private branches are okay
There should be one main branch
It should live at a canonical source
Other branches should have brief lives
Saturday, July 21, 12
Make the main branch available to everyone
A friendly tarball is nice
Github can help you
A running server is nicer
Update this on every commit
Saturday, July 21, 12
Every Commit Must Pass Integration Tests
These are not run on your dev machine
Although those should’ve passed too
Run separate integration-testing machines
Your commit is not finished until the main branch passes these tests
Saturday, July 21, 12
Run Integration Tests on A Clone of Production
Same hardware.
Same OS and software stack.
Same versions of everything.
Database as similar to production as possible.
Scrubbed production DB
Script-generated test data
Virtualization is your friend.
Saturday, July 21, 12
Automate DeploymentYou’ll be deploying a lotConsistency is vitalAutomate every layer:
OS and web stack (e.g. Puppet)Code deployment (e.g. Capistrano)DB updates (Drupal update functions)
Acquia Cloud, Pantheon, etc. can help you
Saturday, July 21, 12
Automate Testing
You’ll be testing a lot
Every bit you do will help.
Smoke tests
Browser-simulation (e.g. Selenium)
Test the showstoppers
Test things which broke once before
Saturday, July 21, 12
“CI” ≠ “Jenkins”
But Jenkins can help(Or CruiseControl, Travis CI, Buildbot, etc.)
“Continuous Integration is an attitude, not a tool” - James Shore
Saturday, July 21, 12
Commit Very Often
Standard: Everyone commits once per day
Or more
Practice doing work in smaller pieces
Saturday, July 21, 12
Builds Must Be Fast
Seconds = superb
Five to Ten Minutes = fine
An hour = awful
Many hours = disastrous
Saturday, July 21, 12
OMG My Builds Take Three Days
Consider a deployment pipeline
Stage 1: minimalist tests, fast
Stage 2: slow
Stage 3: wicked slow
A commit to main branch:
Must pass Stage 1 in advance
Triggers Stage 2; failures must be fixed ASAP
Saturday, July 21, 12
Display the Results to Everyone
Status review during scrum meetings
Usable web dashboards
Control-room displays
Saturday, July 21, 12
Lava Lamps
http://www.pragmaticautomation.com/cgi-bin/pragauto.cgi/Monitor/Devices/BubbleBubbleBuildsInTrouble.rdoc
Saturday, July 21, 12
Rubber Chicken
http://jamesshore.com/Blog/Continuous-Integration-on-a-Dollar-a-Day.html
Saturday, July 21, 12
BLAME-SEEKING ENGINE OF DOOM
http://www.papercut.com/blog/chris/2011/08/19/who-broke-the-build/
Saturday, July 21, 12
You’re Gonna Love Continuous IntegrationReduced Risk
Process Visibility
Fewer Bugs
Easier Debugging
Easier Rollback
Less Wasted Effort
Saturday, July 21, 12
Credits
The Acquia Cloud team
Martin Fowler“Continuous Integration”www.martinfowler.com/articles/continuousIntegration.html
Saturday, July 21, 12