build automation for xpages - auslug 2015

Build Automation for XPages

Cameron Gregor | Senior Systems Engineer Jord International Pty Ltdwww.jord.com.au

June 11th & 12th, Melbourne, AustraliaMeet.Share.Learn.Connect @AusLUG #@Inform2015

Cameron Gregor

• Blog – gregorbyte.com

• Twitter - @gregorbyte

• Github - camac


Agenda

● What is Build Automation?

● Introduction to APACHE ANT

● Introduction to Jenkins CI Server

● Building NSF from Source with Headless Designer

● Building OSGi plugins with PDE Build

● Deployment Considerations


WHAT IS BUILD AUTOMATION?


What do we mean by build automation?• Automation of common tasks in software development

– Compiling Source Code

– Running Tests

– Generating Documentation

– Deploying to Production / Testing

• Repeatable– Avoid human error

– Avoid dependency on key personnel


Basic Tools involved for a Build ‘Job’

• A Script of Tasks to follow

• A Computer to run the script

• Some programs to execute the Tasks

• A mechanism to Trigger the Job


Example Build Job

Checkout Source Code

Ensure Dependencies

Met

Compile

Package into an Artifact


INTRODUCTION TO APACHE ANT


http://ant.apache.org/• ANT = Another Neat Tool

• Written in Java– Platform independent

• Mainly used for building Java applications– However: can be used for anything

• Easily Extensible via ‘antlibs’ (ant libraries)

• Run from command lineant


Why ANT?• It worked!

• Documentation was easy to follow

• Extending it was easy to follow

• Installation was easy

• Very Flexible (Some say this is a disadvantage though)

• Alternatives– Gradle: I hear this is pretty good but I don’t know much about it

– Maven: I tried to give this a go but It was too big a learning curve for too little gain

– others


ANT Concepts

Build File (build.xml)

Project (root node)

Target• Task• Task• Task

Target• Task• Task

Target• Task• Task• Task


Example Tasks

• Print Hello World to the Console<echo message=“Hello, world”/>

• Make a new Directory<mkdir dir=“lib”/>

• Compile java<javac srcdir="${src}" destdir="${build}" classpath="xyz.jar" debug="on" source="1.4" />

• Create a Zip File<zip destfile="${dist}/manual.zip“basedir="htdocs/manual"/>

• Execute a program<exec executable=“some.exe” failonerror=“true”/>


Example Build File


Example Build File Output

> ant

Buildfile: V:\Projects\SimpleBuildFile\build.xml

clean: [delete] Deleting directory V:\Projects\SimpleBuildFile\build

init: [mkdir] Created dir: V:\Projects\SimpleBuildFile\build

build: [touch] Creating V:\Projects\SimpleBuildFile\build\bar.txt

BUILD SUCCESSFULTotal time: 0 seconds


Properties• Similar to a ‘parameter’ or argument

• Provide a way to allow customisation of a build, – e.g. specify different directory locations, different options

• Reduce duplication of strings within the build file


Setting Properties within Build File• Set a property to a Value

– <property name=“propname” value=“propvalue”/>

• Set a property to a File or Directory Location (relative to base directory)– <property name=“propname” location=“someDir/someFile”/>

• Properties can be read from a ‘properties’ file from within build file<property file=“build.properties”/>


Setting Properties when Calling Ant• Properties can be passed in via command line when calling ant using –D flag

ant –Dproperty=value

• Or read from properties file from command lineant –propertyfile build.properties


Build File Using Property


Buildfile: V:\Projects\SimpleBuildFile\buildUsingProperty.xml

clean: [delete] Deleting directory V:\Projects\SimpleBuildFile\buildPropertyCmdLine

init: [mkdir] Created dir: V:\Projects\SimpleBuildFile\buildPropertyCmdLine

build: [touch] Creating V:\Projects\SimpleBuildFile\buildPropertyCmdLine\bar.txt

BUILD SUCCESSFULTotal time: 0 seconds

Build with Property Output

>ant –buildfile buildUsingProperty.xml -DbuildDir=buildPropertyCmdLine


INTRODUCTION TO JENKINS CI


Jenkins CI Server• An application that monitors executions of repeated jobs with focus on

– Building / Testing Software projects continuously

– Monitoring executions of externally run jobs such as cron jobs

• Easy installation, no database required

• Distributed builds, master / slave possible

• Extensible via Plugins, with a plugin directory ready to access.

• Web Interface with rest-like api as well

• Requires JRE of 1.6 or later

• FREE! As in Speech and Beer – MIT License, very permissive


Jenkins vs Hudson• https://jenkins-ci.org/

• Originally created at Sun Microsystems as ‘Hudson’

• Dispute between Project Team and Oracle

• ‘Jenkins’ created as a Fork

• Kohsuke Kawaguchi is Creator and went with Jenkins

• Hudson is still available too http://hudson-ci.org/@kohsukekawa


Note: the following Screen shots are

taken from a Jenkins installation with

additional plugins installed so may look

different to your install


Home Page


Status Icons• S = Last run status

– Grey = Disabled

– Blue = Success

– Red = Fail

• W = Health of Recent Builds (last 5)– Sunny - all succeeded

– Sunny with Cloud = 1 failed

– Cloud = 2 failed

– Rain 3 failed

– Lightning – all recent builds failed


Project Page


Project Workspace


Configure Project


Configure – Source Code Repo


Configure – Trigger


Configure – Build Steps


Configure – Post Build Actions


Build Page


Build – Console Output


Manage Jenkins


Global Properties


Manage Plugins


Chuck Norris


Jenkins - Conclusion• Price is right

• Good community – lots of plugins

• Easy to install and backup

• Works for us!


BUILDING NSF FROM SOURCE WITH HEADLESS DESIGNER


Domino Designer Source Control Enablement

NSF ODP


The .project file


Headless Designer• Not Really Headless though

• New feature since 9.0.1

• Allows for automated building of NSF from On Disk Project

• Still a bit clunky but it works

• http://www-10.lotus.com/ldd/ddwiki.nsf/dx/Headless_Designer_Wiki

• 2 Options for running– Single Command

– Command File (Multiple commands)

• Built NSF can only end up in the Notes Data Folder (not in subdirectory)


Requirements on machine• Domino Designer 9.0.1

• NOTES.INI entry – DESIGNER_AUTO_ENABLED=true

• Notes ID with NO PASSWORD

• Check Source Control preferences

• Maybe add Notes Program directory to PATH environment variable so you can run ‘designer’ from anywhere

• Disable any dialogs that prompt on start-up and shutdown


Setting No Password


Source Control Preferences


Disable Shutdown Prompts


Disable Replication on Startup / Shutdown


Single Command Example • Format:

designer –RPARAMS –vmargs –Dcom.ibm.designer.cmd=[singlecommand]

• [singlecommand] =“[clean],[exit],[file name],[job name],[file path],[file name]”

• designer –RPARAMS –vmargs -Dcom.ibm.designer.cmd=“true,true,Headless.nsf,importandbuild,C:\AUSLUG\Headless\.project,Headless.nsf’


DEMO HEADLESS DESIGNER


Designer Launches


Designer Minimizes


You can maximize it manually


Click Progress Icon to Monitor


Oh no – Designer doesn’t have latest plugins


INSTALL LATEST PLUGINSTRY AGAIN


Good – No problem now


Plugin management for Headless Designer• Wiki previously mentioned some ‘Headless’ commands that auto installed /

removed plugins

• Removed in version 27 of Wiki article

• Read version 26 to see them

• Probably still work?

• I haven’t tried yet


A Minor Problem

• Most build tasks will let you know when they finish and whether success or fail

• Headless Designer has short attention span

• designer.exe actually starts notes2.exe in parallel and then shuts down

• Need a way to monitor when notes2 shuts down

• Egor Margineanu @egmar came up with a Powershell script to do so– http://www.egmar.ro/blog/2014/02/10/using-continous-integration-in-xpages-projects/


Powershell script• Erases previous HEADLESS0.log file

• Launches Headless Designer

• Finds the ‘notes2’ process

• Waits until the notes2 process has exited


Jenkins Installation Notes• Jenkins can be installed as Service or started as normal application

• I had trouble running Headless Designer when installed as service– Others have said it works for them if they tick ‘Service can interact with desktop’

• Instead I launch Jenkins from a logged in user


BUILDING OSGI PLUGINS WITH PDE BUILD


Building Plugins != Building Java• OSGi plugins are complicated

• Bundles need to ‘resolve’ dependencies

• Eclipse has PDE Build– Can build using Eclipse UI

– Can also build Headless


Plugins, Features and Update Sites

PluginsPluginA

PluginB

PluginC

FeaturesFeature A• Plugin A

Feature B• Plugin A• Plugin B

Feature C• Plugin C


Building Normal Java • Java Runtime Environment

• Classpath

• javac!


Building OSGi Plugins• Java Runtime Environment

• Target Platform (Other OSGi plugins)

• Your Plugin(s) Source Code

• A Feature Project pointing to your plugins


Manually Building Plugins• Create Feature Project

– Add Plugins to feature

• Create Update Site Project– Add Features to Update Site

• Build Update Site


After running Build All


What do you need to build Domino OSGi Plugins• Eclipse for RCP and RAP Developers

– Eclipse.org, latest version is ‘Luna’

• Source Code of your plugins

• A Feature Project which specifies the plugins that are to be built

• Your Target Platform (all the other Plugins will be co-located)

• Your Target Java Runtime Environment (all the jars on the bootclasspath)– We run our headless builds on a development server that has a Domino instance

• A Build Configuration directory

• A Build Directory


Headless Eclipse

• http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.pde.doc.user%2Ftasks%2Fpde_customization.htm

http://help.eclipse.org/luna/index.jsp?topic=/org.eclipse.pde.doc.user/tasks/pde_customization.htm

http://help.eclipse.org/luna/index.jsp?topic=/org.eclipse.pde.doc.user/tasks/pde_customization.htm


Build Properties Template

<eclipse>/plugins/org.eclipse.pde.build_<version>/templates/headless-build


Automated Building of Features• Basic Setup

– Create a Build Configuration Directory in your project

– Copy the build.properties template file to your build configuration dir

– Modify build.properties as needed

• buildDirectory property specifies the directory to perform build in

• Before Each ‘Build’– Wipe BuildDirectory clean

– Copy Source of Plugins and feature into Build directory

– Structure is

• <buildDirectory>\plugins\<your plugin projects here>

• <buildDirectoyr>\features\<your feature project here>


Builder Configuration Properties

• http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.pde.doc.user%2Freference%2Fpde_builder_config.htm&anchor=fetchControl

http://help.eclipse.org/luna/index.jsp?topic=/org.eclipse.pde.doc.user/reference/pde_builder_config.htm&anchor=fetchControl




Build.Properties for Domino Plugins – in a Nutshell

• Please Build feature ‘X.Y.Z’

• Please do not to download Source from CVS

• We are not creating a ‘Product’ so don’t run the packager

• Please make plugins and features into Jars

• By the way this is our:– Runtime Environment ( jar’s that are on the boot class path),

• these are the jars in <domino/jvm/* subfolders

– Target Platform (Plugins that our plugins would like to play with)

• These are the plugins in

– <domino>/osgi/rcp/eclipse– <domino>/osgi/shared/eclipse– Any other plugins that are depended on


Running the build• Eclipse contains an embedded ‘ant’

• This command invokes the embedded ‘ant’ and tells it to use the pde build build.xml file– <eclipse> = Eclipse Directory

– <v> = Version number

java -jar <eclipse>/plugins/org.eclipse.equinox.launcher_<v>.jar -application org.eclipse.ant.core.antRunner -buildfile <<eclipse>/plugins/org.eclipse.pde.build_<v>/scripts/build.xml> -Dbuilder=<path to the build configuration folder>


Running build from Ant File


DEPLOYMENT CONSIDERATIONS


What to do with Built NSF and Plugins?• Copying NSF to Server

• Refreshing Designs– Set template name on built nsf

– Set template inheritance of target nsf

– Refresh design

• Single Copy Xpage Design

• Restarting Http


BuildXPages • Ant Library of tasks for use in deploying and building Xpages apps

• Uses Java Native Access to make calls to Notes C Api

• Only tested on Windows! But give it a try

• Tasks include– Setting Template Names (inherit from, is master, Single Copy Xpage)

– Copy NSF to server from Local Filesystem

– Deletensf

– Generate Site Xml for plugins


Generating a Site.xml• Site.xml file is needed to import plugins and features

• PDE Headless build only generates plugins and features, not site.xml

• Need to generate site.xml yourself


NSF Update Site• By Default you must manually import to NSF Update Site

• Karsten Lehmann made slight modification to allow ‘Headless’ import

• http://www.openntf.org/main.nsf/project.xsp?r=project/Open%20Eclipse%20Update%20Site

http://www.openntf.org/main.nsf/project.xsp?r=project/Open%20Eclipse%20Update%20Site




Conclusion

• If you want to automate some Build Processes you can

• It is fiddly to set up but once you have it there it works

• Contact me if you have any questions!


APPENDIX ANotes on Build Properties for OSGi Headless Build


Notes on Build.Properties

• topLevelElementId would correspond to the feature which defines the plugins to build

• <BuildDirectory>/features/com.your.feature.id/

Property Value

topLevelElementType feature

topLevelElementId com.your.feature.id



• The Packager is used when building a whole ‘Rich Client Platform’ product– E.g. Eclipse, IBM Notes, IBM Domino Designer are Rich Client Platform Products

• We are not building a ‘Product’ we are only building plugins

Property Value

runPackager false



• PDE build has an in-built mechanism to ‘fetch’ Source from CVS

• Keep these set to true to avoid any CVS operations

Property Value

skipFetch true

skipMaps true



• outputUpdateJars will generate the plugins and features as ‘.jar’ files– if set to false, features will not be made into ‘jar’ files

– if set to false, plugins with ‘unpack the plug-in archive after the installation’ will not be made into ‘jar’ files

– If you plan on loading into an Update site I recommened setting to ‘true’

• resolution.devMode may be needed to be used to allow for less strict validation of bundle dependencies

Property Value

outputUpdateJars true

resolution.devMode true



• PDE Build matches the Execution Environment in the Plugin Manifest, to the related property in build.properties

• These Jars form the ‘boot class path’

• skipBase tells

Property Value

JavaSE-1.6 List of all jars from <domino>/jvm and subdirectories

skipBase true



• pluginPath is a list of folders which contain features/ and plugins/– This is equivalent to your ‘Target Platform’

– By Default you will have

• <dominoProgDir>/osgi/rcp/eclipse

• <dominoProgDir>/osgi/shared/eclipse

– You could also include other 3rd party plugin dependencies

• <someotherDir>/someproject/eclipse

Property Value

pluginPath List of Eclipse Plugin folders



• If you are building plugins for production, then you may want javacDebugInfo to be false

• If you are building plugins and want to ‘Remote Java Debug’ them, you will need line numbers included, and should set javacDebugInfo to true

Property Value

javacDebugInfo true or false

compilerArg -g

build automation for xpages - auslug 2015

Software

build automation

inform2015 introduction

build job

inform2015 ant concepts

inform2015 http

inform2015 agenda

inform2015 basic tools

inform2015 cameron gregor