modxpo - continuous integration and deployment
DESCRIPTION
Just over a year ago we (Adido) decided to adopt MODX as our primary CMS in favour of our in-house platform Dub3 - one that we'd been using for almost 10 years. As a full-service Digital Agency servicing over 300 clients both in the UK and Europe we faced a number of challenges, whilst also giving ourselves the opportunity to revisit our methods of Continuous Integration & Deployment. We've learned a number of lessons along the way working with MODX, from continuously building new features within teams of multiple developers to deploying to scalable server farms. Jenkins, RunDeck and Rackspace; if you're a fan of these then you'll have an insight into what we'll be exploring in this session, and what our guys at Adido are up to.TRANSCRIPT
TomProject Manager
LauraDesigner
ChrisDeveloper
MarkPHP
Developer
ThomFrontend
Developer
30+ employees
Marketing, Web Build & Support
Bournemouth, UK
Creative Studio
1 x Project Manager1 x Designer3 x Developers
1 x Support Manager2 x Developers
Creative Studio
Support
ConfessionWe used to be a .NET agency
(and we built a CMS)
• Our primary platform for almost 10 years
• 300+ websites
Time to move on…
• Built in-house
• Community• Customisation• Easier for
clients
How we build websites
How we deploy websites
Opportunities to review
The .NET days
• SVN for version control (hosted internally)
• Local IIS or Visual Studio Server
• rsync to single ‘demo’ or ‘live’ site
rsync• .bat files initiate
sync• Diff local repo with remote before sync
• If local version out of date, stop and update• Checkout, merge (cross
fingers)• Possible to sync and never commit• Review sync.log on server to identify
offender• Edit .bat to change destination or source DB
Remote Desktop
• Direct access to files on server
• Quick fixes
• Sync would overwrite edits• Short term fix, long term
headaches
• The devil on your shoulder
• All developers had access
Dedicated Servers
• Expensive
• Not immediately scalable
This worked fine for a long time
Required an inherent knowledgeof our infrastructure
Room for human error
Not great for growth
Mark
New ideas!
@markwillis82
“create and configure lightweight, reproducibleand portable development environments”
Wrapper around VirtualBox, Chef, & Puppet(now also works with VMware)
Every developer works with exactly the sameenvironment, irrespective of their chosen OS
Centrally configured with Puppet
Files sync with host
> vagrant up
> vagrant up
/dev/Vagrantfile
> vagrant up
localhost:2000
localhost:2002
localhost:2004
> cd dev
CI Server
• Puppet
> vagrant up
localhost:2000
localhost:2002
localhost:2004
> cd dev
CI Server
• Puppet
Live Servers
Alternative MethodDNS wildcards
http://******.chris
nginx config
project1.chris
http://
project2.chris project3.chris project4.chris project5.chris
CI Server
• Puppet
Ok...What’s next ?...
Version ControlKeep your work safe
• (Potentially) more reliable than GitHub
• Free Private Repos
Local Environment
• git
• Vagrant (VirtualBox)
• IDE independent
Let’s use those……to go one step further
Continuous Integration
• Prevent integration problems
• Automated testing• Run periodically or after every
commit• …report results – email, HipChat
• Merging can be a scary thing
Continuous Integration
• Branch every feature• Write tests for that feature
• Commit and Push• CI server is waiting for
you…
Jenkins
Jenkins
• ‘Detects’ new commit
(actually, BitBucket tells him)
• runJobs() or createNewJobs()
https://bitbucket.org/adido/jenkins-bitbucket-connector/
• A set of jobs for Master, and every branch
• Parent jobs• Child jobs
Tests
• phplint
• unit tests
• behat
Behat
“Behat is a tool that makes behavior driven development (BDD) possible. With BDD, you write human-readable stories that describe the behavior of your application. These stories can then be auto-tested against your application. And yes,
it’s as cool as it sounds!”
Feature: Your first feature In order to start using Behat As a manager or developer I need to try
Scenario: Successfully describing scenario Given there is something When I do something Then I should see something
Behat
Behat
Feature: A MODX website with multiple contexts
Scenario: Successfully loading the correct context When am on "/" Then I should see "Venues & Centres" And I change to "littledown" context Then I should see "Membership" And I should see ”Swimming" Then I change to "web" context And I should see ”About Us"
Behat /** * @When /^I change to "([^"]*)" context$/ */ public function iChangeToContext($newContext) { $currentUrl = $this->getSession()->getCurrentUrl();
// store the current url as the correct context if($newContext !== "web") { $this->defaultContext = $currentUrl; $newUrl = str_replace("http://", "http://$newContext.", $currentUrl); } else { $newUrl = $this->defaultContext; // switch back to the original url }
$this->getSession()->visit($newUrl); }
Jenkins
• Not just for testing, but also…
• Serving multiple demo sites
Jenkins
• Browse every branch on the CI server• http://feature-1.project.ci-server.com
• http://feature-2.project.ci-server.com
• Requires wildcard DNS administration
What about databases?What can we do there?
Database
/core/config/config.inc.php
$database_type = 'mysql';$database_server = 'db.server';$database_user = ’db_user';$database_password = ’xxxx-xxxx-xxxx';$database_connection_charset = 'utf8';$dbase = ’modx_database_name';$table_prefix = 'modx_';$database_dsn ='mysql:host=db.server;dbname=modx_database_name;charset=utf8';
Database
• Sync database differences• Compare schema
• Compare data
• MySQL Workbench, among others
• Case-by-case basis
Merge & DeployLet’s push this feature live!
• Use BitBucket to create pull requests • Feature is reviewed• Branch is merged with master, and
closed
• Master tests run• On success, site is deployed using
Jenkins• RunDeck• Abstracts out deployment
This sounds like a lot of work…Can we automate this some more?
So, to FinishLet’s go back to the start…
New Projects
Node.js app: “toolbelt”
New Projects
• Clone base MODX repo to local machine
• Remove remote origin from local repo• Add new BitBucket origin to local repo
• Create new BitBucket repo (API)
• Create hook (API)• Push repo
• Jenkins creates Jobs
Thank you#MODXpo