configure, pack and distribute: an rpm creation workshop

Configure, Pack, DistributeAn RPM Creation Workshop

Matthias G. EckermannSenior Product ManagerSUSE® Linux [email protected]

Bart WhiteleySenior Software EngineerNovell, [email protected]

mailto:[email protected]

mailto:[email protected]

© Novell, Inc. All rights reserved.2

Agenda

Part I: Challenges of software management

Part II: Introduction into RPM– Overview of RPM build process

– The RPM “Spec” file

– Dependencies

– Building RPMs

Part III: Hands-on– Building from source locally

– Building in a change-root environment using the “build” tool

– Building from a binary package

– Building in the BuildService

Part I:Challenges of Software Management


Challenges of Software Management

?


Requirements to a Sophisticated Software Management System• Systems management and consistency

– Easy to install and uninstall– Easy to determine what is installed, and where– Leverage package management tools

> ZENworks® Linux Management

• Dependency management• Life-cycle management

– Easy to create packages> Leverage openSUSE® Build Service

– Easy to update and upgrade• Security and reliability

– Easy to verify the installation


Applications

• Executable (binary), libraries, data files, icons, mime types

• Documentation – normally included, might be removed during installation due to space constraints

• Configuration – normally created after installation, can be pre-configured or defaults are pre-installed

Documentation

Libraries

Executable

Data

Configuration


Applications and Metadata

• Application• Meta-data

– Support for installation and management

– Description, version, requirements and dependencies – what needs to be installed on the system to be able to run the application or its parts

• Install, upgrade and remove process– Using system provided software management stack

– No interaction needed (useful for auto-installation, appliances)

– Clean remove – no leftovers, configuration files optionally


High Level Package Overview

Documentation

Libraries

Executable

Data

Configuration

Version

Description

Requirements

ApplicationMeta-data

Package


Build Overview - Theory

Metadata

Source File

Patch

Source FileApplication

PatchLocal changes

Tool

Archive

Source FileSource FileInstallable

Software Packs

Part II:Introduction into RPM


Build Overview – RPM

SPEC File

Source File

Patch

Source FileSource File

PatchPatch

rpmbuild-ba

Source RPM

Source FileSource FileBinary RPM


What Is it RPM?

• RPM Package Manager– Program to manipulate with packages (rpm files)

• Package file format specification– Files are named like: make-3.81-2.i586.rpm

– [name]-[version]-[subversion].[architecture].rpm or just file.rpm

• Library to manipulate (install, remove) rpm packages• Baseline of Linux Standard Base (LSB) packaging• The most common format in Enterprise Linux


RPM Philosophy

• Standard, uniform and clean way to build, install, remove, update and manage software

• Multi-architecture support• Non-interactive install process, easy to automate• Security – packages can be cryptographically signed• Reliability – package does not affect contents of other packages• Dependencies – package can be installed only if requirements

are already installed• Verification – all files included in a package can be verified for

permissions or contents modification• Building software

– reproducible– unattended


Inputs and Outputs

• Inputs– Spec file

> Directs RPM in the build process

– Source files> May be source archives (e.g. tarball or .zip) or single files

– Patches> Patches to the pristine source archive, usually created with 'diff -u'

• Outputs– One or more binary RPMS– Source RPM

> An archive of the inputs


The Spec File

• The preamble section

• The prep section

• The build section

• The install section

• Install and uninstall scripts

• The clean section

• File list


Preamble

• Required tags– Name

– Version

– Release

– Summary

– License

– Group

– %description

– BuildRoot

• Optional tags– Source– Patch– Url– Requires– Provides– Conflicts– Obsoletes– PreReq– BuildArch– Serial– BuildRequires


Preamble Example

# # Example spec file for cdplayer app... # Summary: A CD player app that rocks! Name: cdplayer Version: 1.0 Release: 1 License: GPL Group: Applications/Sound Source: ftp://ftp.gnomovision.com/pub/cdplayer/cdplayer-1.0.tgz URL: http://www.gnomovision.com/cdplayer/cdplayer.html Distribution: WSS Linux Vendor: White Socks Software, Inc. Packager: Santa Claus <[email protected]>

%description It slices! It dices! It's a CD player app that can't be beat. By using the resonant frequency of the CD itself, it is able to simulate 20X oversampling. This leads to sound quality that cannot be equaled with more mundane software...


The % Prep Section

• Prepares the build environment• Executed as a bash script

– Automatically removes remnants of a previous build

– Expand source archive

– Apply patches

• Macros– %setup

> Often all that is needed in %prep

– %patch> Apply a patch to the sources

%prep %setup


The % Build Section

• Compile the software

• Executed as a bash script

%build

./configure

make


The % Install Section

• Executed as a bash script• Runs during the package build• Does not run during package installation• Install the software into a staging area

– $RPM_BUILD_ROOT

%install %{__rm} -rf $RPM_BUILD_ROOT %{__mkdir} $RPM_BUILD_ROOT make DESTDIR=$RPM_BUILD_ROOT install


The % Clean Section

• This section is optional

• Can be used to clean up files that are not part of the application's normal build area

• Executed as a bash script


The % Files Section

• Lists the files that are part of the package• Declare file ownership and permissions• Special handling for configuration files

– %config

– %config(noreplace)

%files %defattr(-,root,root) %config(noreplace) /etc/foo.conf /usr/bin/* /usr/lib/*


Install/Uninstall Scripts

• %pre– Executed prior to package installation

• %post– Executed after package installation

• %preun– Executed prior to package deletion

• %postun – Executed after package deletion

• Best to do as much as possible in build scripts, and as little as possible in install/uninstall scripts


Scripts During Upgrade

• Scripts can complicate upgrades– %preun and %postun scripts also run during upgrade– The order of execution is not intuitive

> Run %pre of new package> Install new files> Run %post of new package> Run %preun of old package> Delete any old files not overwritten by newer ones> Run %postun of old package

– Important to get scripts right the first time> If you release a package with a bad %preun or %postun script,

you can't fix it with an update


Distinguishing Upgrade from Delete

• The first parameter passed to RPM scripts is the number of copies of the package that will be installed after the current package is installed or deleted

– In %preun and %postun, if $1 is 0, the package is being deleted

– In %preun and %postun, if $1 is > 0, the package is being upgraded

%postun if [ $1 -gt 0 ]; then # package being upgraded else # package being deleted; cleanup fi


Dependencies

• If anything listed in % files is a shared library, its soname is added to the capabilities of the package

• For all executables and shared libraries in %files, ldd is used to determine the package requirements

• Manual dependencies can be declared with Preamble tags

– Requires

– Provides

– Obsoletes

– Conflicts

– PreReq


RPMbuild

RPMbuild

-bp execute %prep

-bc execute %prep, %build

-bi execute %prep, %build, %install

-bb execute %prep, %build, %install (bin)

-ba execute %prep, %build, %install (bin, src)


Build Environment

• The RPM build directory structure– /usr/src/packages/SOURCES %{_sourcedir}

> Contains source files and patches

– /usr/src/packages/SPECS %{_specdir}> Contains Spec files

– /usr/src/packages/BUILD %{_builddir}> Sources are unpacked and patched here> Compile takes place here

– /usr/src/packages/RPMS/<arch> %{_rpmdir}> Binary RPMs end up here

– /usr/src/packages/SRPMS %{_srcrpmdir}> Source RPMs end up here


Customized Build Environment

• To build as a non-root user (good idea), define different paths for the build environment.

– In ~/.rpmmacros

– On the command line

$ cat ~/.rpmmacros %_topdir /home/jdoe/packages

$ cat ~/bin/myrpmbuild #!/bin/sh TOPDIR=${PWD}/rpmbuild mkdir -p ${TOPDIR}/{RPMS,SRPMS,SPECS,SOURCES,BUILD,tmp} rpmbuild --define "_topdir ${TOPDIR}" \ --define "_sourcedir ${PWD}" \ --define "_tmppath ${TOPDIR}/tmp" "$@"


Benefits of Using a Standard Package Format – Such as RPM• Software development more manageable

• Multi architecture support easier

• Software installation more secure

• Software deployment manageable

• Handling multiple version easy

• Consistent systems and compliance

Part III:Hands-on


Building from Source Locally (1)Source Code

/ / / / Copyr i ght ( c) 2010 SUSE Li nux Pr oduct s GmbH/ / Aut hor : Mat t hi as G. Ecker mann <mge@novel l . com>/ / Li cense: GPL v2/ /

#i ncl ude <QAppl i cat i on>#i ncl ude <QFont >#i ncl ude <QPushBut t on>#i ncl ude <QWi dget >

i nt mai n( i nt ar gc, char *ar gv[ ] ) {QAppl i cat i on hel l o_br ai nshar e( ar gc, ar gv ) ;QWi dget mai n_wi ndow;mai n_wi ndow. r esi ze( 512, 256 ) ;QPushBut t on wel come_but t on(

"Wel come t o Br ai nshar e 2010! \\ nNovel l - Maki ng I T Wor k as One. " , &mai n_wi ndow

) ;wel come_but t on. set Font (

QFont ( "Ar i al " , 16, QFont : : Bol d )

) ;

wel come_but t on. set Geomet r y( 32, 32, 448, 192) ;QObj ect : : connect (

&wel come_but t on, SI GNAL( cl i cked( ) ) ,&hel l o_br ai nshar e, SLOT( qui t ( ) )

) ;mai n_wi ndow. show( ) ;r et ur n hel l o_br ai nshar e. exec( ) ;

}


1. Building from Source Locally (2).spec file

#Name: hel l o_br ai nshar eLi cense: GPL v2Ver si on: 1Rel ease: 1. mgeSummar y: Hel l o Br ai nshar e 2010Sour ce0: hel l o_br ai nshar e. cppGr oup: Pr oduct i vi t yBui l dRoot : %{_t mppat h}/ %{name}- bui l dBui l dRequi r es: l i bqt 4- devel

%descr i pt i on Hel l o Br ai nshar e 2010 pr ogr am f or demonst r at i onpur poses.

%pr ep %set up - c - Tcp - a %{S: 0} .

%bui l dg++ - I / usr / i ncl ude/ Qt Gui - l Qt Cor e - l Qt Gui \

$RPM_OPT_FLAGS \- s - o hel l o_br ai nshar e hel l o_br ai nshar e. cpp

%i nst al li f [ - n "$RPM_BUI LD_ROOT" ] ; t hen

[ "$RPM_BUI LD_ROOT" ! = " / " ] &&r m - r f $RPM_BUI LD_ROOT &&mkdi r $RPM_BUI LD_ROOT

f imkdi r - p \

$RPM_BUI LD_ROOT/ usr / bi ni nst al l hel l o_br ai nshar e \

$RPM_BUI LD_ROOT/ usr / bi n/

%f i l es%def at t r ( - , r oot , r oot )/ usr / bi n/ hel l o_br ai nshar e

%cl eani f [ - n "$RPM_BUI LD_ROOT" ] ; t hen

[ "$RPM_BUI LD_ROOT" ! = " / " ] && r m - r f $RPM_BUI LD_ROOT

f i

%changel og* Mon Mar 08 2010 - mge@novel l . com- i ni t i al ver si on


Challenges and RisksWhen Building from Source Locally

• Software-build might interfere with the buildhost– Security and Consistency:

Risk of polluting the host with freshly built software– Resource Constraints:

a build process might need lots of disk space, memory, compute power

• Software-build might depend on special settings of the local build-host

– violates goal of reproducible software builds• Multi-User and Multi-Architecture goal not easy

to implement – even building 32bit on 64bit hosts is not always easy


Alternativesto Building from Source Locally

• Change root environment– Create a build system within the host system; same kernel,

but userland is separated– Mitigates: security, consistency, reproducible– Controls: Multi-architecture requirements.– Does not mitigate: Resource constraints– Technology used by the “build” tool, see #2 below

• Virtualization– Create a build system within the host system;

completely separated– Mitigates: security, consistency, reproducible– Controls: Resource constraints, Multi-architecture requirements– See #4 below: openSUSE® Buildservice


2. Building in a Change-root Environment Using the “Build” Tool (1)• “build” is available on the SUSE® Linux Enterprise 11

Software Development Kit and on openSUSE®

• Build installs a minimal SUSE Linux as build system into some directory and will chroot to this system to compile the package.

• Special .spec file options to support build

# norootforbuild

# needsrootforbuild


Building in a Change-root Environment Using the “Build” Tool (2)

Functionality overviewbuild [--clean|--no-init] \

[--no-checks] \[--repository PATH] \[--rpms path1:path2:...] \[--arch arch1:arch2:...] \[--root buildroot] \[specfile|srcrpm]

build --verify

• Environment variablesBUILD_RPMSBUILD_ROOTBUILD_RPM_BUILD_STAGE


3. Building from a Binary Package (1)

• Differences– No compile stage necessary– Can be used for nearly any type of existing software

• Challenges– Binary packages might need an “installer” to run– system specific configuration– Installation paths not following the Linux Filesystem

Hierarchy Standard (FHS)• Caveats

– Legal aspects– Licensing


Building from a Binary Package (2)Binary Package

Example: Mendeley Desktop (freeware, non-opensoure)• prepackaged tar, no rpm• Directories:mendeleydesktop-0.9.6.1-linux-x86_64/binmendeleydesktop-0.9.6.1-linux-x86_64/libmendeleydesktop-0.9.6.1-linux-x86_64/lib/mendeleydesktopmendeleydesktop-0.9.6.1-linux-x86_64/lib/mendeleydesktop/libexecmendeleydesktop-0.9.6.1-linux-x86_64/lib/mendeleydesktop/pluginsmendeleydesktop-0.9.6.1-linux-x86_64/lib/mendeleydesktop/plugins/sqldriversmendeleydesktop-0.9.6.1-linux-x86_64/sharemendeleydesktop-0.9.6.1-linux-x86_64/share/applicationsmendeleydesktop-0.9.6.1-linux-x86_64/share/docmendeleydesktop-0.9.6.1-linux-x86_64/share/doc/mendeleydesktopmendeleydesktop-0.9.6.1-linux-x86_64/share/iconsmendeleydesktop-0.9.6.1-linux-x86_64/share/icons/hicolormendeleydesktop-0.9.6.1-linux-x86_64/share/icons/hicolor/128x128mendeleydesktop-0.9.6.1-linux-x86_64/share/icons/hicolor/128x128/appsmendeleydesktop-0.9.6.1-linux-x86_64/share/icons/hicolor/16x16mendeleydesktop-0.9.6.1-linux-x86_64/share/icons/hicolor/16x16/appsmendeleydesktop-0.9.6.1-linux-x86_64/share/icons/hicolor/22x22mendeleydesktop-0.9.6.1-linux-x86_64/share/icons/hicolor/22x22/appsmendeleydesktop-0.9.6.1-linux-x86_64/share/icons/hicolor/32x32mendeleydesktop-0.9.6.1-linux-x86_64/share/icons/hicolor/32x32/appsmendeleydesktop-0.9.6.1-linux-x86_64/share/icons/hicolor/48x48mendeleydesktop-0.9.6.1-linux-x86_64/share/icons/hicolor/48x48/appsmendeleydesktop-0.9.6.1-linux-x86_64/share/icons/hicolor/64x64mendeleydesktop-0.9.6.1-linux-x86_64/share/icons/hicolor/64x64/appsmendeleydesktop-0.9.6.1-linux-x86_64/share/mendeleydesktopmendeleydesktop-0.9.6.1-linux-x86_64/share/mendeleydesktop/citationStylesmendeleydesktop-0.9.6.1-linux-x86_64/share/mendeleydesktop/citationStyles/defaultmendeleydesktop-0.9.6.1-linux-x86_64/share/mendeleydesktop/generated-svm-modelsmendeleydesktop-0.9.6.1-linux-x86_64/share/mendeleydesktop/openOfficePluginmendeleydesktop-0.9.6.1-linux-x86_64/share/mendeleydesktop/word-lists

Goal: install to /opt/mendeleydesktop


Building from a Binary Package (3).spec file

Name: mendel eydeskt opLi cense: Commer ci al ( Mendel ey Lt d. )Ver si on: 0. 9. 6. 1Rel ease: 1. mgeSummar y: Mendel ey i s a f r ee r esear chmanagement t oolBui l dAr ch: x86_64

%def i ne _f name %{name}- %{ver si on}- l i nux- %{bui l dar ch}

Sour ce0: %{_f name}. t ar . bz2Gr oup: Pr oduct i vi t yURL: ht t p: / / www. mendel ey. com/ downl oad-mendel eydeskt op/Bui l dRoot : %{_t mppat h}/ %{name}- bui l d

%descr i pt i on Mendel ey i s a f r ee r esear ch management t ool f ordeskt op & web

%pr ep %set up - c - T

%bui l d

%i nst al li f [ - n "$RPM_BUI LD_ROOT" ] ; t hen

[ "$RPM_BUI LD_ROOT" ! = " / " ] &&r m - r f $RPM_BUI LD_ROOT &&mkdi r $RPM_BUI LD_ROOT

f imkdi r - p $RPM_BUI LD_ROOT/ opt

t ar - C $RPM_BUI LD_ROOT/ opt - xsj pf %{S: 0}

mv $RPM_BUI LD_ROOT/ opt / %{_f name} \$RPM_BUI LD_ROOT/ opt / %{name}

%f i l es%def at t r ( - , r oot , r oot )/ opt / %{name}

%cl eani f [ - n "$RPM_BUI LD_ROOT" ] ; t hen

[ "$RPM_BUI LD_ROOT" ! = " / " ] &&r m - r f $RPM_BUI LD_ROOT

f i

%changel og* Mon Mar 08 2010 - mge@novel l . com- i ni t i al ver si on


4. Building in the Buildservice

• Open and complete distribution development platform• Create RPMs for multiple distributions and architectures• Create online package repositories• openSUSE® Build Service is open source

– Deploy your own internal build service• User Interfaces

– Webinterface http://build.opensuse.org/– Commandline interface: “osc”– Fat-clients– Integration into other tool-chains (qt-creator, kde4)

http://build.opensuse.org/


Building in the Buildservice (2)

• “osc” command line interface• Very similar to “cvs” and “svn” command line tools:

familiar for developers• Overview

– Checking

– Commit

– Add

– importsrcpkg

– Build

– Rebuild


Next Steps

Learn more at:Brainshare® Session BOF 102

en.opensuse.org/Build_Servicebuild.opensuse.org

Go, and build your packages!

Join the openSUSE®

Buildservice online!

Appendix


References

http://rpm.org/max-rpm/

http://docs.fedoraproject.org/drafts/rpm-guide-en/

http://en.opensuse.org/SUSE_Build_Tutorial

http://en.opensuse.org/Packaging/SUSE_Package_Conventions

SUSE® software management stack:

http://en.opensuse.org/Libzypp

http://www-128.ibm.com/developerworks/linux/library/l-rpm3.html

http://www.ibm.com/developerworks/library/l-rpm1/

http://rpm.org/max-rpm/

http://docs.fedoraproject.org/drafts/rpm-guide-en/

http://en.opensuse.org/SUSE_Build_Tutorial

http://en.opensuse.org/Packaging/SUSE_Package_Conventions

http://en.opensuse.org/Libzypp

http://www-128.ibm.com/developerworks/linux/library/l-rpm3.html

http://www.ibm.com/developerworks/library/l-rpm1/

Unpublished Work of Novell, Inc. All Rights Reserved.This work is an unpublished work and contains confidential, proprietary, and trade secret information of Novell, Inc. Access to this work is restricted to Novell employees who have a need to know to perform tasks within the scope of their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of Novell, Inc. Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability.

General DisclaimerThis document is not to be construed as a promise by any participating company to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. Novell, Inc. makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for Novell products remains at the sole discretion of Novell. Further, Novell, Inc. reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All Novell marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners.

configure, pack and distribute: an rpm creation workshop

Documents

rpm overview of rpm

installation5 novell

build section

build process source

rpm spec file dependencies

binary rpms source rpm

management description

install section