Continuous Integration For Real: The Perforce Java Platform Hamish Reid Perforce Software Inc.

OVERVIEW

•  What do we mean by… •  Agile? •  Continuous Integration?

•  Product line highlights: •  P4Eclipse + Mylin + MergeQuest •  P4Java •  P4Ant •  P4Maven •  (Hudson Perforce Plugin) •  (P4Java, P4D)

•  The Perforce Java Team CI Workflow

WHAT DO WE MEAN BY “AGILE”?

•  Adaptive processes and workflows: •  Feedback- and reality-driven •  Flexible goals

•  Continuous processes and workflows: •  No surprises! •  Small increments in planning •  Small increments in delivery •  Small increments in effort •  There’s always some version of the final product

•  Examples: •  Scrum, XP, Kanban

WHAT DOES “CONTINUOUS INTEGRATION” MEAN?

•  The heart of agile processes… (…but not just agile processes).

•  Martin Fowler’s essentials: •  Integration is a non-event. •  Any individual developer's work is only a few hours

away from a shared project state. •  Individual work can be integrated back into that state in

minutes. •  Any integration errors are found rapidly and can be fixed

rapidly. •  Trades up-front workflow and infrastructure effort for

integration simplicity. •  Not for every product or workflow.

CONTINUOUS INTEGRATION ELEMENTS (1)

•  Martin Fowler: •  Configuration management •  Build and test automation •  Self-testing •  Frequent commits •  Build and test on commit •  Fast build / test cycles •  Test in production environment clone •  Easy to get latest build artifacts •  Easy to get latest status •  Automated deployment

CONTINUOUS INTEGRATION ELEMENTS (2)

•  Additionally, from our own experience: •  Integration into issues tracker system •  Easy / safe / cheap branching…

…and integrate / merge / resolve cycle •  Low burden for developers:

•  Integration into IDE’s and other common tools •  Visualization •  “Culturally appropriate”

OUR JAVA TEAM WORKFLOW

•  We “eat our own dog food”… …but we also “eat our customers’ dog food”

•  Heavy emphasis on continuous integration •  (Very) test-driven development process •  Jobs-driven process •  Branch-centric process:

•  Dev / shared (team) / main / release •  Two modes:

•  Eclipse-centric development and dev testing •  Hudson-centric automation for shared branches

•  Open Source / off-the-shelf solutions: •  Eclipse, Mylin, Junit, Maven, Ant, Hudson, Cruise

control

KEY ELEMENTS IN OUR CI WORKFLOW

•  Eclipse + P4Eclipse •  P4Eclipse + MergeQuest + Mylin + more… •  Hudson + Perforce plugin •  Maven + P4Maven •  Ant + P4Ant •  P4Java •  (P4D)

ECLIPSE – THE DEVELOPER'S COMMAND CENTER

ECLIPSE + P4ECLIPSE

•  All code work done in Eclipse •  P4Eclipse supplies the Perforce functionality:

•  Import as project •  Add / edit / delete files •  Submit / changelist management •  Refactoring •  P4Grep, diffs, resolves, branches, merges

•  All dev testing done using Junit: •  Suite-based •  Right-click from Eclipse

•  Mylin – the task master •  MergeQuest, branch graph, timelapse… •  Mavenized development where needed

HUDSON + PERFORCE

•  Hudson + P4 extensions central to Perforce Java team workflow: •  Used to setup builds and tests •  Used to store results and raw report data back to SCM •  Triggered on successful merges and codeline changes

•  Uses existing third-party Open Source plugin •  Perforce will help develop and maintain •  Currently p4 command line based

•  Typically configured through Hudson control panel •  Works with Jenkins, too, of course…

MAVEN + P4MAVEN

•  Maven used extensively: •  Distribute shared team builds •  Cache external dependencies •  Off-the-shelf build lifecycle support •  Tie-ins to Eclipse •  Often run from Hudson

•  P4Maven: •  Perforce-developed and supported •  P4Java-based •  Released 11.1 •  Full Maven “scm” functionality •  MOJO-based extensions for jobs, etc. •  Non-core plugin

ANT + P4ANT

•  P4Java-based •  P4Ant targets:

•  The usual suspects, plus… •  …things like shelve / unshelve, fix, job-related tasks,

etc. •  Perforce usage:

•  Test setups •  Builds •  Distributions

•  Example: <taskdef resource="com/perforce/ant/tasks/P4Tasks.properties" /> …

<target name="p4tasktest" description="Perforce task test"> <p4jsubmit port="eng-p4java-vm:20101" user="p4jtestuser" passwd="p4jtestuser" client="p4TestUserWS" changelist="3391" /> </target>

P4JAVA

•  Makes all-Java implementations possible •  Native protocol implementation •  Clear object model •  Underpins:

•  Nearly all our Java-related CI platform •  Perforce Web Services Platform (P4WSP)

LOOKING BACK…

•  Tools integration (“it’s the tools, stupid!”): •  P4Eclipse + Mylin •  P4Maven, P4Ant, Hudson Plugin

•  Visualization: •  MergeQuest, P4Eclipse, Timelapse, Branch Graph •  Hudson + P4Hudson

•  Immediacy: •  P4Eclipse, Hudson

•  Flexibility: •  …all that and more.

SUMMARY

•  The Perforce Java product line now forms a platform •  Key components:

•  P4Eclipse •  P4Maven •  P4Ant •  P4Hudson •  (P4D, P4Java)

•  It makes agile continuous integration so easy… …that even the Perforce Java team can do it.