automated deployment with phing

Automated Deployment With Phing - Daniel Cousineau

1

Automated Deployment With Phing

Or: How I Will Replace You With A Very Small Shell Script

ZendCon '09


2

Who

• Daniel Cousineau• Senior Software Applications Developer

Texas A&M UniversityDivision of Student AffairsDepartment of IT

• http://www.toosweettobesour.com/• @dcousineau

ZendCon '09

http://www.toosweettobesour.com/


3

THE PROBLEM

ZendCon '09


4

Human beings make mistakes

ZendCon '09


5

We aren’t as accurate each repetition

ZendCon '09


6

Machines don’t have this problem

ZendCon '09


7

They do the same thing every time (supposedly)

ZendCon '09


8

At Any Given Time

• New Code• New Configuration• New Database Schema• New Static Files

ZendCon '09


9

A Lot To Remember

• Did you remember to upload ALL new files?• Did you remember to update your DB?• Did you remember to correct your config?

ZendCon '09


10

Even Worse

• Did you clear your caches?• Did you delete that old file/plugin?• In the upload process, was your configuration

overwritten?• Did you upload ALL the changed files?• When did you last upload?

ZendCon '09


11

THE SOLUTION

ZendCon '09


12

Automation!

• Build scripts!– We are programmers after all…

ZendCon '09


13ZendCon '09

Don’t Do More Work Than You

Have To!


14ZendCon '09

Work Hard At Being Lazy!


15

What is Automation?

• Automated deployment means a single command– Locks your live site– Uploads changed files– Clears caches and temporary files– Updates the database schema– Runs other cron tasks– Unlocks your live site

ZendCon '09


16

Why Do We Automate?

• Deployment is tricky• Repetition degrades quality– She sells sea shells by the sea shore

ZendCon '09


17

When Is Automation Used?

• All the time!– Staging– Live

• Probably best to use it on your dev box too!

ZendCon '09


18

THE BASICS

ZendCon '09


19

Tools of the Trade

• Build System– Phing– Apache ANT– Capastrano– Plain PHP or BASH or BAT Files

• File Transfer– Rsync (*NIX Environments)– Xcopy (Windows Environments)

• Configuration Management• Database Migration

– DBDeploy– Doctrine

ZendCon '09


20

Phing Philosophy

• Build scripts contains "Targets"– Targets should be small and specialized.– Example Targets:• clean

– Clear temporary and cached files

• copy– Copy files to their intended destination

• migrate– Upgrade the database schema

ZendCon '09


21

Phing Philosophy

• Targets can have dependencies– Target "live" can depend on clean, copy, and

migrate– Meaning, when we run the "live" target, it first

runs clean, copy, then migrate• And any tasks they depend on

ZendCon '09


22

Installing Phing

• pear channel-discover pear.phing.info• pear install phing/Phing

– Want all the little dependencies?• pear config-set preferred_state alpha• pear install –alldeps phing/Phing• pear config-set preferred_state stable

ZendCon '09


23

Running Phing

• $> phing –v– Lists Phing version

• Phing expects to find a file "build.xml" in the current directory– build.xml defines the targets– You can use the "-f <filename>" flag to

specify an alternate build file like• $> phing -f build-live.xml

ZendCon '09


24

Syntax

• XML– If you don’t already know it, you have bigger

problems• Variables– ${variablename}– Conventions• Psuedo-Namespaces using periods• ${namespace.variable.name}

ZendCon '09


25

Basic Conventions

• build.xml– Central repository of your top-level tasks

• build-*.xml– Can be included into build.xml. – Usually for grouping by target (dev, staging, live)

or task (migrate, test, etc.)• build.properties– Technically an INI file that contains variables to be

included by build files.

ZendCon '09


26

Basic build.xml

ZendCon '09

<?xml version="1.0" encoding="UTF-8"?><project name="ZendCon" default="default" basedir="."> <property file="build.properties" /> <target name="default"> <echo message="Howdy World!" /> </target> </project>


27

Basic build.properties

ZendCon '09

source.directory = /src

## Database Configurationdb.user = usernamedb.pass = passworddb.host = localhostdb.name = database


28

FILE TRANSFER

ZendCon '09


29

Techniques• Built in Phing Copy Task

– Cross Platform– Slow– Not over network

• Version Control Checkout/Update– Syncs deleted files– User created files usually ignored

• Rsync– Great for *NIX environments.– Almost guaranteed to be installed on

• Linux, BSD, Mac OSX• Can be installed on Windows

• Xcopy– Good for Windows environments– Faster than Phing Copy– Not over network

ZendCon '09


30

Pitfalls

• Cleanup Deleted Source Files– Usually only a problem when you have * include

patterns

ZendCon '09


31

Pitfalls

• User created files are a HUGE pitfall– /htdocs/images/upload

• Sync programs usually will delete content your deployment machine doesn’t have

• What if user upload are mixed in with files you want to manage?

– Solutions• Keep user created content completely separated. The

further up the file tree and consolidated the better.• Separate static file server, Amazon S3, or other style

CDNs

ZendCon '09


32

Executing External Tools

• Nearly all file transfer tools will be external commands

• For this we need the Exec task

ZendCon '09

<exec command="cp file1 file2" />


33

Rsync

• So many different options you’re best off finding your own tutorials or reading MAN pages

• rsync -aCvz-e 'ssh -i /path/to/ssh-key'${project.basedir}/srcuser@domain: ${staging.deployment.directory}/

ZendCon '09


34

Rsync Options Of Note

• -a archive-mode– Recursive copy, preserve everything, etc.

• -C cvs-exclude– Ignore .cvs, .svn, and other version control artifacts

• -v verbose– Know EXACTLY what’s going on

• -z compress– Compress information transmitted

• -e alternative remote shell– Use a custom SSH command (namely use key-pair)

ZendCon '09


35

Xcopy

• Best when your live deployment process involves copying to a network share on Windows machines

• xcopy${project.basedir}${source.directory}\*${staging.deployment.directory}/D /I /Y /S /E/Exclude:${project.basedir}\staging.deployment.exclude

ZendCon '09


36

Xcopy Options Of Note

• /d– Update only those files that are different (timestamps)

• /i– Creates target directories if they don’t exist

• /y– Do not prompt

• /s– Copy all directories and sub-directories…

• /e– …Even if the directories are empty

• /exclude– Exclude files based on glob patterns

ZendCon '09


37

Xcopy Exclude File Example

ZendCon '09

\tmp\\uploads\


38

CONFIGURATION MANAGEMENT

ZendCon '09


39

Techniques

• Check incoming domain– www.domain.com means load production

configuration– localhost/domain.com means load

development configuration• Check for an environment file– Contents of DOCUMENT_ROOT/.env determine

configuration• Copy over during deployment

ZendCon '09


40

Check Incoming Domain

• Pros– No chance for error

• Cons– Detection is a weak link• Especially when development, staging, and live have

different web servers (e.g. Apache to IIS)

ZendCon '09


41

Check for an Environment File

• Pros– Consistent regardless of Server or OS changes– Easy to repair• Quick FTP or SSH

– Easy Testing• Just swap file contents

• Cons– Forget to copy changes?

ZendCon '09


42

Copy Over During Deployment

• Pros– Images and CSS now manageable– Reduce redundancy– Simple to repurpose regular push command

• Cons– Difficult to manage– Forget to copy changes?

ZendCon '09


43

Which one to choose?

• Depends on your application architecture• Depends on your environment• You’ll know what’s best for your project– I use a combination of Environment File and Copy

Over During Deployment

ZendCon '09


44

DATABASE MIGRATION

ZendCon '09


45

Database Deployment Philosophy

• “Versions” often referred to as “Deltas”• Each Delta has an UP and a DOWN script– UP makes the changes– DOWN reverts the changes

• Each change should be a non-destructive schema change– Unless of course you need data alteration

ZendCon '09


46

DBDeploy Philosophy

• Each delta is 2 native SQL scripts– Separated by --//@undo

• Database version is stored in a special table– Version X was applied at TIMESTAMP…– Beware corruption!

ZendCon '09


47

DBDeploy Under The Hood

• Connect to the database, read from changelog table, get the most current revision installed

• Retrieve all delta files newer that the current revision– Revision numbers are based off of alphabetic

sorting, name your files accordingly• Pull all the “UP” changes and concatenate

them to a migration script• Up to YOU to run the migration scriptZendCon '09


48

Installing DBDeploy

• Is Phing already installed? Great! So is DBDeploy…• Create the changelog table– Track the current version of your DB

ZendCon '09

CREATE TABLE changelog ( change_number BIGINT NOT NULL, delta_set VARCHAR(10) NOT NULL, start_dt TIMESTAMP NOT NULL, complete_dt TIMESTAMP NULL, applied_by VARCHAR(100) NOT NULL, description VARCHAR(500) NOT NULL );

ALTER TABLE changelog ADD CONSTRAINT Pkchangelog PRIMARY KEY (change_number, delta_set);


49

Basic Delta: 01_init.sql

ZendCon '09

--//

CREATE TABLE IF NOT EXISTS ùsers` ( ìd` int(10) unsigned NOT NULL auto_increment, `handle` varchar(25) NOT NULL default '', PRIMARY KEY (ìd`), UNIQUE KEY ùsers_handle_index` (`handle`) ) ENGINE=InnoDB DEFAULT;

--//@UNDO

DROP TABLE IF EXISTS ùsers`;

--//


50

Run Migration

• First add a task to your Phing build file

ZendCon '09

<target name="migrate">

</target>


51

Run Migration

• Generate the SQL migration file

ZendCon '09

<target name="migrate"> <dbdeploy url="mysql:host=${db.host};dbname=${db.name}" userid="${db.user}" password="${db.pass}" dir="${project.basedir}/db/deltas" outputfile="${project.basedir}/up.sql" undooutputfile="${project.basedir}/down.sql" /></target>


52

Run Migration

• Execute the SQL script

ZendCon '09

<target name="migrate"> <dbdeploy url="mysql:host=${db.host};dbname=${db.name}" userid="${db.user}" password="${db.pass}" dir="${project.basedir}/db/deltas" outputfile="${project.basedir}/up.sql" undooutputfile="${project.basedir}/down.sql" /> <exec command="/usr/bin/mysql -h${db.local.host} -u${db.local.user} -p ${db.local.pass} ${db.local.name} < ${project.basedir}/up.sql" dir="${project.basedir}" checkreturn="true"> </target>


53

Pitfalls

• Make sure you have a good CLI app for loading a SQL file– /usr/bin/mysql– mysql.exe

• Build script travelling across operating systems?– Write your own Phing task to execute the migration

output?• Was the changelog table created in the first

place?ZendCon '09


54

Doctrine Database Migrations

• Integrated into the Doctrine CLI tool– ./doctrine migrate

• Same philosophies– BUT, deltas are PHP objects that contain an UP

and a DOWN method• Run using the Phing exec task

ZendCon '09


55

Other Database Migration Tools

• http://www.liquibase.org/

ZendCon '09

http://www.liquibase.org/


56

CLOSING THOUGHTS

ZendCon '09


57

Further Resources

• The people around you– More people than you know automate their

deployment– There are many, many different techniques– My techniques may suck for your particular setup

• #phpc on freenode– Many of the speakers hang out there– Many smart people hang out there– I HANG OUT THERE!– Therefore I am smart

ZendCon '09


58

Further Resources

• Blogs– http://phpdeveloper.org

• Documentation– http://phing.info– http://dbdeploy.com

• Google• Experimentation– Pushing to a local directory– Pushing to a virtual machine

ZendCon '09

http://phpdeveloper.org/

http://phing.info/

http://dbdeploy.com/


59

Questions?

ZendCon '09

http://joind.in/937

automated deployment with phing

Technology