Buckminster

Building it all with Buckminster and p2Eclipse Con, March 2009

Henrik Lindberg, Cloudsmith Inc

EclipseCon 2009

AGENDA

• Overview of Buckminster• Show how easy it is to use Buckminster to materialize

and build an Eclipse RCP application in the IDE, and Headless.

EclipseCon 2009

AT A GLANCE

Buckminster:• Materializes components to a

location (e.g. workspace)• Produces new artifacts inside

or outside of the workspace (i.e. ‘build’)

A

A

EclipseCon 2009

TRANSITIVE MATERIALIZATION

• When working on A, components B and C are also needed, and they need D, and E, and…

• Buckminster materializes all pre-requisite components

• Can take action into account – dependency followed only when needed.

A

B C

D EF

G H

EclipseCon 2009

WHERE DOES BUCKMINSTER GET COMPONENTS?

B F GA HC D E

RMAP

EclipseCon 2009

WHERE DOES BUCKMINSTER MATERIALIZE COMPONENTS?

B F GA HC D E

workspace “eclipse” target pl. file system

using p2 in 3.5 – UM in earlier

EclipseCon 2009

Give me A.Give me A.

HOW DO YOU TELL BUCKMINSTER WHAT YOU WANT?

Component QUERY

EclipseCon 2009

AT BUCKY BURGER

Give me A.Give me A.ok

EclipseCon 2009

BUCKY DELICATESSEN

source or binary?

mutable or not?

for here or for to go?

we are out of binaries – do you want the

source instead?

Give me A,hold the mayo!

Give me A,hold the mayo!

override that broken provolone 2.0?

日本語 , or

English?

EclipseCon 2009

Give me A.Give me A.

TO SUMMARIZE…

B

FG

A

H

C D

E

EclipseCon 2009

COMPONENT – WHAT IS IT?

Buckminster:• A “Component Reader” gets

existing meta data from component and translates it “on the fly” to a Component Specification (CSPEC)

• If meta data does not exist – author the CSPEC

Benefit:• No roundtrip engineering!• Everything is maintained in

the original format (which is needed for other purposes).

requires contains provides includes

actions, build requires actions, source binary

meta datasource

documentation

requires contains provides includes

actions, build requires actions, source binary

meta datasource

documentation

EclipseCon 2009

The meta data translation can be decorated/extendedwith the use of a ComponentSpecification Extension (CSPEX)

META DATA UNDERSTOOD BY BUCKMINSTER

B

A

C

D

E

EclipseCon 2009

PB-MAPRMAP

ORGANIZING THE MAPS

B F GA HC D E

RMAP

RMAP

EclipseCon 2009

ROUTING

B F GA HC D E

EclipseCon 2009

DEPENDENCIES AND VERSIONS

• Describes; if a component is needed which version (or range of versions) is acceptable

• Uses “OmniVersion” – a pattern based version descriptor (contributed to p2).

A

B C

D EF

G H

1.01.0

[1.0.0,2.0.0)[1.0.0,2.0.0)

EclipseCon 2009

OMNI VERSION

• A “universal” canonical format• Backwards compatible with Buckminster < 3.5 – the version-type

simply selects a preconfigured version format.• OSGi is the default, all other formats must be specified with a

predefined or stated format/pattern.• Learn more at http://wiki.eclipse.org/Equinox/p2/Omni_Version

format(n[.n=0;[.n=0;[.S=[a-zA-Z0-9_-];]]]):1.0.0.r1234

format(n[.n=0;[.n=0;]][d?S=m;]):1.0.0

format(<[n:]a(d?a)*>[-n[dS=!;]]):33:1.2.3a-23/i386

osgi

triplet

rpm

EclipseCon 2009

ATTRIBUTES & ACTIONS

• Attributes represents artifact groups• Can have pre-requisites on other attributes

– Same or other component

• Attributes are static or computed (the result of actions)

Acompile

Blib

headers

EclipseCon 2009

ATTRIBUTES & ACTIONS

• Action attributes produce artifacts– Eclipse builds– ANT tasks

• Extension point– Maven– Your own

Acompile

Blib

headers<<private>>

makeLib

EclipseCon 2009

BUILT IN ACTIONS

• Buckminster includes a set of actions immediately available for bundles, features and products.

– builds all binaries– builds all source features/bundles– (steps available as separate actions too)

• Build update site – compiled, signed, pack-200 processed, and

packaged as a p2 update site!

• Build complete product– complete build to p2 update site

Asite.p2

EclipseCon 2009

LOOK MA, NO HEAD !

• Makes interactive and headless builds the same !

• Runs exactly the same builders the same way

• No need to maintain an extra set of build actions

EclipseCon 2009

Build me A product

Build me A product

BUCKMINSTER IN ONE SLIDE

B

FG

A

H

C D

E

Thank you!Thank you!

EclipseCon 2009

Demo

Build the “RCP mailapp” with support for p2 self management

Tutorial projects (org.eclipse.buckminster.tutorial.*) are found in Buckminster SVN repository:

http://dev.eclipse.org/svnroot/tools/org.eclipse.buckminster/trunk

EclipseCon 2009

Demo Projects Overview

projects: org.eclipse.buckminster.tutorial...

mailappThe “mailapp” example RCP app - but .product broken outThe “mailapp” example RCP app - but .product broken out

product.featureContains .product – refers only to this feature. Feature includes mailapp, and rcpp2.feature and everything required.

Contains .product – refers only to this feature. Feature includes mailapp, and rcpp2.feature and everything required.

rcpp2.featureContains everything needed to make mailapp p2 self managed.Contains everything needed to make mailapp p2 self managed.

relengA feature containing queries and definitions for Common Build Infrastructure CBI.

A feature containing queries and definitions for Common Build Infrastructure CBI.

.product

EclipseCon 2009

Materialize

releng

EclipseCon 2009

Build the site with everything

product.feature

EclipseCon 2009

Properties

# Where all the output should go

buckminster.output.root=${user.home}/bmturorial

# Where the temp files should go

buckminster.temp.root=${user.home}/tmp/bmtutorial.tmp

# How .qualifier in versions should be replaced

qualifier.replacement.*=generator:lastRevision

# How the qualifier (last revision) should be formatted

generator.lastRevision.format=r{0,number,00000}

# If a site should use pack200, turn off for faster build

# site.pack200=true

# If a site should be signed

# site.signing=true

# If a site should be signed locally, or remotely at Eclipse

# signing.type=eclipse.remote

# Should local signing be performed

# local.sign=true

# Should source bundles be built and included

cbi.include.source=false

# What platforms should be built, * for "all"

target.os=*

target.ws=*

target.arch=*

EclipseCon 2009

Install and Update

• We now have a p2 update site on our disk (under output root):o.e.b.tutorial.mailapp.product.feature_1.0.0-eclipse.feature/site.p2

• We need to install our mailapp to try it out using one of:– the p2.installer application (download from RT/provisioning > march 18)– headless director from buckminster download– the director.app from the command line (embedded in the IDE)– a launch configuration in the IDE that runs the director.app

• Install using the p2 installer to try it out• We need, the installer, and a modified installer.properties file

EclipseCon 2009

Installer properties

In a properties file (mailapp_install.properties):

eclipse.p2.metadata = file:<path to p2 site>

eclipse.p2.artifacts = file:<path to p2 site>

eclipse.p2.rootId = org.eclipse.buckminster.tutorial.mailapp.product

eclipse.p2.profileName = Mailapp

eclipse.p2.launcherName = eclipse

eclipse.p2.flavor = tooling

eclipse.p2.autoStart = true

Make installer use the properites:-vmargs

-Dorg.eclipse.equinox.p2.installDescription=file:<path to properties>

EclipseCon 2009

Adding categories

Define categories in the projects build properties.

Define the category “mailapp”

category.id.mailapp=Mailapp Demo

List the feature ids (,) that should be in this category

category.members.mailapp=\

org.eclipse.buckminster.tutorial.mailapp.product.feature

Add a description

category.description.mailapp=Mailapp BM build demo

Use qualified id such as:

category.id.org.myorg.main=My Main Product

category.id.org.myorg.options=My Main Product Options

EclipseCon 2009

The RMAP

Search path entries – for RELEASE, N, I, and M builds, and Galileo

<searchPath name="org.eclipse.platform.RBUILD">

<provider readerType="eclipse.import" componentTypes="osgi.bundle,eclipse.feature" mutable="false" source="false">

<uri format="http://download.eclipse.org/eclipse/updates/3.4?importType=binary"/>

</provider>

</searchPath>

Search path entry for Orbit

<searchPath name="orbit">

<provider readerType="eclipse.import" componentTypes="osgi.bundle" mutable="false" source="false">

<uri format="http://download.eclipse.org/tools/orbit/downloads/drops/S20090307012903/updateSite?importType=binary"/>

</provider>

</searchPath>

EclipseCon 2009

The RMAP - TBD

My project – built from source

<searchPath name="org.eclipse.buckminster”>

<provider readerType="svn" componentTypes="osgi.bundle,eclipse.feature,buckminster" mutable="true” source="true”>

<uri format="http://dev.eclipse.org/svnroot/tools/org.eclipse.buckminster/trunk/{0}? moduleAfterTag&amp;moduleAfterBranch"><bc:propertyRef key="buckminster.component" />

</uri>

</provider>

</searchPath>

Search order

<locator searchPathRef="org.eclipse.buckminster" pattern="^org\.eclipse\.buckminster(\..+)?"/>

<locator searchPathRef="org.eclipse.platform.${useBuild}" failOnError="false" />

<locator searchPathRef="org.eclipse.galileo" failOnError="false" />

<locator searchPathRef="orbit" />

EclipseCon 2009

The RMAP

Specify the default type of build to use

<property key="useBuild" value="RBUILD"/>

Override for some components in the query if needed

EclipseCon 2009

A headless build

Create and populate the workspace

buckminster import <path/url to cquery>

Build everything to the p2 update site

buckminster perform o.e.b.tutorial.mailapp.product.feature#site.p2 --properties <path to properties>

To use other workspace than “Workspace” in current dir, add:

--data <path to workspace>

EclipseCon 2009

Update

• Modify the version of the rcpp2 feature• Build ”site.p2” again• Run the application – use ”install new software” (bugs in

”update” at the moment).• You may have to add the local update site as a repo (due to

not yet fixed issues with installer).• See that there is a new version of the feature to install.• Select it, and the application is updated

EclipseCon 2009

TO RUN THE DEMO

• You need Eclipse 3.5 >= “N 0318”• Install Buckminster – follow instructions at

http://www.eclipse.org/buckminster/downloads.html • Checkout org.eclipse.buckminster.tutorial.mailapp.releng from

http://dev.eclipse.org/svnroot/tools/org.eclipse.buckminster/trunk

• Open the “developer.cquery”, and click “Resolve & Materialize” – this populates your workspace

• Now you have the full set of projects, and what you need to build as described in this presentation.

EclipseCon 2009

Build me A product

Build me A product

BUCKMINSTER IN ONE SLIDE

B

FG

A

H

C D

E

Thank you!Thank you!

EclipseCon 2009

VISIT US AT

http://www.eclipse.org/buckminster

http://wiki.eclipse.org/Buckminster_Project

EclipseCon 2009

More Links

• How to make RCP app self managed: http://wiki.eclipse.org/Equinox/p2/Adding_Self-Update_to_an_RCP_Application

• Download Buckminster: http://www.eclipse.org/buckminster/downloads.html

• Buckminster wiki: http://wiki.eclipse.org/Buckminster_Project