open source software solutions and best practices for aixidh.ch/ibm_tu_2013/opensource on...

2013 IBM Corporation

Dr. Michael Perzl - IBM Migration Factory, GermanyOctober 2013

Open Source Software Solutions and Best Practices for AIX(pOX669)

pOX669 - Open Source Software Solutions and Best Practices for AIX

2013 IBM Corporation2

TrademarksThe following are trademarks of the International Business Machines Corporation in the United States, other countries, or both.

Not all common law marks used by IBM are listed on this page. Failure of a mark to appear does not mean that IBM does not use the mark nor does it mean that the product is not actively marketed or is not significant within its relevant market.

Those trademarks followed by are registered trademarks of IBM in the United States; all others are trademarks or common law marks of IBM in the United States.

For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml:

*, AS/400, e business(logo), DBE, ESCO, eServer, FICON, IBM, IBM (logo), iSeries, MVS, OS/390, pSeries, RS/6000, S/30, VM/ESA, VSE/ESA, WebSphere, xSeries, z/OS, zSeries, z/VM, System i, System i5, System p, System p5, System x, System z, System z9, BladeCenter

The following are trademarks or registered trademarks of other companies.

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office.IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce.

* All other products may be trademarks or registered trademarks of their respective companies.

Notes: Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.



Important reminder

Please note:The opinions given in this presentation are my own and do not necessarily represent IBMs position.

PS: I still like my job...



Agenda

Open source software what are we talking about ?Customer expectation and a look at the competitionA historical view 12 years ago...Current open source software availability for AIXMy personal repository: http:///www.perzl.org/aix

Technical Part:AIX 32-bit physical address spaceRecommended compile environmentShared libraries on AIXGCC on AIXCompatibility issue with the original AIX Toolbox for Linux ApplicationsAIX V6.1 and V7.1 specifics

Summary

http:///www.perzl.org/aix



5 2013 IBM Corporation

Open source software what are we talking about ?



Open source software what are we talking about ?

Description of Open Source Software (OSS):Not only literally availability of source code but also defines a boundary on the usage, modification and redistribution of open source softwareSoftware licenses grant rights (on usage, modification and redistribution) to users which would otherwise be prohibited by copyright

Properties of open source software:Licensed through one of the many available open source licensesMay be licensed under more than one license type, commercial and open source

Good example here is Nokias (formerly Trolltech) QT GUI-frameworkUsed license may restrict usage for certain typically commercial purposes

For our considerations:Software where the source code is (freely) available and licensed through one of the open source licenses

http://en.wikipedia.org/wiki/Software_licensehttp://en.wikipedia.org/wiki/Copyrighthttp://trolltech.no/



Selected open source licenses

Public domain vs. licensed software

From: http://en.wikipedia.org/wiki/Comparison_of_free_software_licencesApache LicenseBerkeley Database LicenseBSD licenseEclipse Public LicenseIBM Public LicenseGNU General Public License (GPL)GNU Lesser General Public License (LGPL)MIT license / X11 licenseMozilla Public LicenseSun Public Licenseand many more...

http://en.wikipedia.org/wiki/Comparison_of_free_software_licenceshttp://en.wikipedia.org/wiki/Apache_Licensehttp://en.wikipedia.org/wiki/Berkeley_Database_Licensehttp://en.wikipedia.org/wiki/BSD_licensehttp://en.wikipedia.org/wiki/Eclipse_Public_Licensehttp://en.wikipedia.org/wiki/IBM_Public_Licensehttp://en.wikipedia.org/wiki/GNU_General_Public_Licensehttp://en.wikipedia.org/wiki/GNU_Lesser_General_Public_Licensehttp://en.wikipedia.org/wiki/MIT_Licensehttp://en.wikipedia.org/wiki/Mozilla_Public_Licensehttp://en.wikipedia.org/wiki/Sun_Public_License



Some prominent examples of open source software

Apache - many more besides web serverEmacsFirefoxGaim instant messengerGangliaGCC GNU Compiler CollectionGimp image manipulation programGNOME (alternative GUI)gzip - file compression utility JBOSSKDE (alternative GUI)LibreOffice (OpenOffice fork)Net-SNMPMuninMySQL

OpenOffice.orgPHPPostgreSQLRDist file distribution RSyncSambaSquid proxy server ThunderbirdVIM (VI improved)VLC Media PlayerVNCGNU wgetWireshark (previously called ethereal)XEmacs.... and many more...

http://www.apache.org/http://www.gnu.org/software/emacs/http://www.mozilla.org/http://gaim.sourceforge.net/http://www.ganglia.info/http://gcc.gnu.org/http://www.gimp.org/http://www.gnome.org/http://www.gzip.org/http://www.jboss.org/http://www.kde.org/http://www.libreoffice.org/http://net-snmp.sourceforge.net/http://munin.projects.linpro.no/http://www.mysql.com/http://www.openoffice.org/http://www.php.net/http://www.postgresql.org/http://www.magnicomp.com/rdist/http://rsync.apache.org/http://www.samba.org/http://www.squid-cache.org/http://www.mozilla.com/en-US/thunderbird/http://www.vim.org/http://www.videolan.org/vlc/http://www.realvnc.com/http://www.uk.research.att.com/vnc/index.htmlhttp://www.wireshark.org/http://www.xemacs.org/




Customer expectation anda look at the competition



Why are customers using open source software ?

Some reasons:Because most of this software is pretty goodBecause it provides additional functionality not present in the native UNIX implementationDifferent native implementations behave differently (e.g., differences in the Solaris, HP-UX and AIX implementation)They want to use the same tools across platforms



Customer expectation

Customers who have used open source software before have a certain expectation:

They are used to having open source tools easily and readily available (just download and install)

For our UNIX competitors (Oracle, HP) this is possible Especially customers who also use Linux fall into that category

Typically they want the latest version or a rather up-to-date version of that software

Linux and its software availability is clearly the measuring stick for UNIX here !



A look at the UNIX competition** Not really a competition anymore when looking at Unix market share!

Oracle URL: http://www.sunfreeware.com/ Claims:

19 years as the #1 site for Free Software Packages for Solaris Currently offering more than 60 Gigabytes of packages for Solaris 2.5 2.10 Used by over 528,000 unique IPs in 220 countries/territories

My personal remarks: Huge selection of Open Source Software Very much up-to-date

No official SUN support but private project of Steven M. Christensen (and Associates, Inc.)

HP Porting and Archive Centre for HP-UX

URL UK: http://hpux.connect.org.uk/ My personal remarks:

Huge selection of Open Source Software Very much up-to-date

No official HP support but private project of Connect Internet Solutions Limited

http://www.sunfreeware.com/http://hpux.connect.org.uk/http://www.connectinternetsolutions.com/




A historical view 12 years ago...



Do you still remember back in 2001 ?

Next generation of AIXAIX 5L is the new nameNew AIX logo signals:

Leadership technology advancement Openness and strong Linux affinity



AIX 5L: Linux affinity, why ?

BenefitsExpands the AIX application portfolioAllows Linux developers to use familiar tools on AIX

Common source code, tools, development environmentScalability and enhanced RAS features of AIX for Linux applicationsAllows Linux developers to easily reach the AIX install base

AIX the easiest and best UNIX platform to port toIncreases the commonality between Linux and AIX

AIXAIX

LinuxLinux



AIX 5L: Linux affinity implementation

Implemented in two Phases

First phase:Release of the AIX Toolbox for Linux ApplicationsPopular "Linux - GNU" tools, utilities, look & feel on AIXFor AIX 4.3.3 and AIX 5L 5.1 (shipped with every media)

Second phase:Integration of Linux compatible APIs and header files into AIX 5L (header files and libraries)Application Programming Interface (API) approach to provide Linux source application interoperability ( actually not more than Linux Affinity on Application Level !Not an environment or additional layer/wrapper to run Linux applicationsApplication recompile necessary, recompiled application is native AIX applicationTighter integration of application to OS than with Application Binary Interface (ABI) approach possible

AIXAIX

LinuxLinux



AIX 5L: Linux affinity API approach (1/3)

Definition of a common Linux base:Linux Distributions which were considered:

RedHat 6.2 SuSE Linux 6.4 Debian GNU/Linux 2.25 Caldera OpenLinux 2.4

and compared to: AIX 4.3.3 which included a development environment

Determination of differences based upon:Common Linux files missing from AIXCommon Linux library symbols missing from AIX




Linux system calls:Employed documents

AIX 4.3.3 Linux kernel 2.2.14 GNU libc 2.1.3 Open Group Single UNIX Specification V2 (SUSv2) Linux Standards Base specification 0.2pre

Taken from the system call table (syscalls.h) in the i386 port of Linux 2.2.14Determine use frequency of incompatible Linux system calls and LSB subroutines

compared against the external symbols referenced during the building of over 500 RPM packages on RedHat 6.2

Determine use frequency of non-AIX Linux system calls and LSB subroutines compared against the external symbols referenced during the building of over 500

RPM packages on RedHat 6.2




Different groups of APIs:Linux APIs compatible with AIX

most system calls fall into this categoryLinux APIs not available on AIX

typically Linux specific system calls, e.g., vm86()

Linux affinity on application source code level ! Linux APIs introduced in AIX 5L Version 5.1 see Redbook "Linux Applications on pSeries" for a detailed list

Linux APIs available on AIX but not 100% compatible (less than 10) Dual-semantic functions, plenty of #ifdef and typedef additions Use preprocessor macro -D_LINUX_SOURCE_COMPAT flag to obtain wanted

functionalityRedbook resource:

"Linux Applications on pSeries" (SG24-6033)web site: http://www.redbooks.ibm.com

http://www.redbooks.ibm.com/



AIX Toolbox for Linux Applications

Application Development gcc, g++, gdb, rpm, cvs, automake, autoconf, libtool, bison, flex, gettext

Desktop Environments Gnome, KDE

GNU base utilities gawk, m4, indent, sed, tar, diffutils, fileutils, findutils, textutils, grep, sh-utils

Programming Languages guile, python, Tcl/Tk, rep-gtk

System Utilities emacs, vim, bzip2, gzip, git, elm, ncftp, rsync, wget, lsof, less, samba, zip, unzip, zoo

Graphics Applications ImageMagick, transfig, xfig, xpdf, ghostscript, gv, mpage

Libraries ncurses, readline, libtiff, libpng, libjpeg, slang, fnlib, db, gtk+, Qt

System Shells bash2, tcsh, zsh

Window Managers enlightenment, sawfish



IBM support for AIX Toolbox Applications

IBM provides the GNU tools and other software in the AIX Toolbox for Linux Applications on an "as-is" basis.Linux Affinity version of GNU tools usually better integrated into AIX than if you downloaded the GNU software and compiled it yourself !All patches made to the source code available

Support via the Linux Affinity web sites (back then):http://www.ibm.com/servers/aix/products/aixos/linux/

AIX Toolbox for Linux Applications homepagehttp://www.ibm.com/servers/aix/products/aixos/linux/faq.html

Frequently Asked Questions about the AIX Toolbox for Linux Applicationshttp://oss.software.ibm.com/developerworks/projects/aixtoolbox/

Project homepage within the IBM DeveloperWorks web hierarchy Most useful site with regards to the AIX Toolbox for Linux Applications

http://www.ibm.com/servers/aix/products/aixos/linux/http://www.ibm.com/servers/aix/products/aixos/linux/faq.htmlhttp://oss.software.ibm.com/developerworks/projects/aixtoolbox/



AIX Toolbox for Linux Applications over the years...

The AIX Toolbox for Linux Applications was intended to give customers exactly what they want.In the beginning and first years this worked really well and was well received by our customers.However, legal issues have significantly limited the usefulnesss of the AIX Toolbox for Linux Applications the last 7 to 8 years.Current situation:

Only bug fixes (security issues) are added to the AIX Toolbox packages Hardly any new packages have been released the last couple of years Hardly any new versions of packages have been released the last couple of years Important packages, e.g., Samba, are only available in pre-historic versions

In its current state the AIX Toolbox is not very usable anymore !

http://www.ibm.com/servers/aix/products/aixos/linux/



The AIX Toolbox for Linux Applications is outdated...

As of 08/2004:

Please also check the posting of David Clissold (back then IBM maintainer of the Toolbox):http://sourceforge.net/p/aixtoolbox/mailman/message/6711692/

The issue with our distributing open source packages really isn't the building or packaging of the images; it is a legal question. We cannot distribute open source packages -- whether just a version update or (even rarer) a new package -- without extensive review and approval from the legal group. So having it built externally doesn't help get it onto our own site, but thanks for the offer.

Note that this doesn't stop anyone from building packages themselves and sharing them with each other directly (though I know that's not what you were asking). Or even making their own images publicly available somewhere, from an independent site of their own.

Bull AIX freeware site: http://www.bullfreeware.com/UCLA AIX freeware site: http://aixpdslib.seas.ucla.edu/

http://sourceforge.net/p/aixtoolbox/mailman/message/6711692/http://www.bullfreeware.com/http://aixpdslib.seas.ucla.edu/




Current open source softwareavailability for AIX



Open source repositories for AIX (1/4)

AIX Toolbox for Linux Applications URL: http://www-03.ibm.com/systems/power/software/aix/linux/index.html

Bull open source software archive for AIX URL: http://www.bullfreeware.com/

Bull's continued version of the AIX Toolbox for Linux Applications URL: http://www.bullfreeware.com/aixtoolbox/index.html

Darren Tuckers OpenSSH for AIX website: Older versions of OpenSSH for AIX URL: http://www.zipworld.com.au/~dtucker/openssh/

http://www-03.ibm.com/systems/power/software/aix/linux/index.htmlhttp://www.bullfreeware.com/http://www.bullfreeware.com/aixtoolbox/index.htmlhttp://www.zipworld.com.au/~dtucker/openssh/




Up to date web browsers for AIX, based on GTK+ and GTK2 Firefox, Seamonkey and Thunderbird for AIX URL: http://www.linkitup.de/ForAIX/ No Firefox beyond version 3.6.25!

Alternative desktop environments (instead of default AIX CDE environment) KDE v3.4.3

Unfortunately not downloadable from the KDE FTP Server anymore was: ftp://ftp.kde.org/pub/kde/stable/3.4.3/AIX/

You can still download a copy from my website: http://oss4aix.org/download/KDE/

GNOME for AIX5L (v2.14, v2.16): Downloadable from Bull Open Source Software Archive for AIX

URL: http://www.bullfreeware.com/ Direct URL: http://gnome.bullfreeware.com/new_index.html

http://www.linkitup.de/ForAIX/ftp://ftp.kde.org/pub/kde/stable/3.4.3/AIX/http://oss4aix.org/download/KDE/http://www.bullfreeware.com/http://gnome.bullfreeware.com/new_index.html




RRDtool versions by Zoran Majcenic http://oss.oetiker.ch/rrdtool/download.en.html http://www.inet.hr/zmp/ibm/rrdtool

Recent Python versions for AIX V6.1 by Sascha Thomas Spreitzer https://spreitzer.name/python-2-6-5-for-ibm-aix-6-1

http://oss.oetiker.ch/rrdtool/download.en.htmlhttp://www.inet.hr/zmp/ibm/rrdtoolhttps://spreitzer.name/python-2-6-5-for-ibm-aix-6-1




Hudson Valley Community College open source software for AIX Provides packages in BFF format URL: http://pware.hvcc.edu/ Required AIX levels:

AIX 5.3 TL-09-06 or AIX 5.3 TL-10-04 or AIX 5.3 TL-11-04 or AIX 5.3 TL-12-02 AIX 6.1 TL-03-06 or AIX 6.1 TL-04-06 or AIX 6.1 TL-05-02 or AIX 6.1 TL-06-01 AIX 7.1 TL-00-01

Notes: AIX 5.2 not supported anymore The AIX 5.3 code base no longer supported since April 30, 2012

Samba for AIX http://us1.samba.org/samba/ftp/Binary_Packages/AIX/ Note: They just link to Hudson Valley Community College Just get it from my website!

http://pware.hvcc.edu/http://www-01.ibm.com/software/support/aix/lifecycle/index.htmlhttp://us1.samba.org/samba/ftp/Binary_Packages/AIX/




My personal website:http://www.perzl.org/aix

http://www.perzl.org/aix



My personal AIX open source repository (1/3)

URL: http://www.perzl.org/aix/Started 12/2005 because I was clearly frustrated with the current situation of the AIX Toolbox for Linux ApplicationsGoal:

Completely replace the AIX Toolbox for Linux Applications Not much missing anymore

Stay 100% upgrade-compatible to the AIX Toolbox for Linux Applications Sometimes difficult

Software availability: Currently more than 920 different packages available Total number of downloadable packages 16000 Total downloadable volume: 40+ GB of RPM packages Growing constantly

This is the de-facto AIX Toolbox now!

http://www.perzl.org/aix/




So far user feedback has been superb!

Number #1 ranking on Google (7 years in a row now!)Just enter aix open source on Google

Available download protocols: FTP, HTTP, RSYNC

I do this strictly in my free time No relation to IBM on purpose (please see next slide)

If you have a request for a specific package let me know and well see what we can do




http://www.perzl.org/aix runs on an Intel x86_64 VServer (CentOS 6.2)Hosted by Strato (German ISP) with 100 GB disk spaceUnlimited traffic (otherwise I would go bankrupt)

I personally pay for the ISP servicesWebsite uses PmWiki as CMS (Content Management System)

Personal plea:If this website has been

useful to you convinced you to stay on AIX convinced you to buy even more AIX systems etc.

please let me or IBM know. Feedback is always very welcome !

http://www.perzl.org/aixhttp://www.pmwiki.org/



Some download statistics from http://www.perzl.org/aix

http://www.perzl.org/aix




Technical part



Technical part Agenda

Best practices:AIX 32-bit Physical Address SpaceRecommended Compile EnvironmentShared Libraries on AIXGCC on AIXCompatibility issue with the original AIX Toolbox for Linux ApplicationsAIX V6.1 and V7.1 Specifics




AIX 32-bit physical address space



AIX 32-bit physical address space (1/3)

Virtual memory:hardware provides continuous range of virtual memory to access data from

0x0000000000000 to 0xFFFFFFFFFFFFF

total addressable space of more than 1,000 TB52-bit representation requiredmemory access instructions generate 32-bit addresses

Consequence:segment::offset mechanism implemented to provide mapping of virtual to physical addresses




segment offset

offsetsegment

segment register

28

2824

4

32-bit program address

52-bit virtual memory address




segment::offset mechanismaddressing scheme provides access to 16 segments of up to 256 MB(= 228 bytes) eacheach segment register contains a 24-bit segment ID that becomes a prefix to the 28-bit offset, which together form the virtual memory address

Process space is 32-bit address spaceprograms use 32-bit pointerseach process or interrupt handler can address only the system-wide virtual memory space (segment) whose segment IDs are in the segment register



AIX 32-bit address space layout (1/3)

Segment number Use Size

Available number

0x0 Kernel 256 MB 10x1 Process text 256 MB 10x2 Process private 256 MB 10x3 : : : :0xC

shmat / mmap

(attached with explicit compiler options)

2.56 GB 10

0xD Shared library text 256 MB 10xE shmat / mmap 256 MB 10xF Shared library data 256 MB 1




Remarks:the kernel (0x0) and shared-library (0xD) text segment are shared by all processes and have restricted access when executing in the user protection domainthe process text segment is held by segment 1the process private segment is held by segment 2 and contains:

user block (ignored from now on, small and fixed-size) kernel stack (ignored from now on, small and fixed-size) user stack user data regions

the segments 3 through 12 may be loaded (using interfaces such as shmat or mmap) to provide access to files or shared memory datathese conventions constrain the size of the user data and user stack regions (single shared 256 MB segment #2)




user blockkernel stackuser stack

user data

uninitialized variables

malloc allocated space

unused space

u.ulimit[CUR].data

u.ulimit[MAX].data

u.ulimit[MAX].stack

u.ulimit[CUR].stack

0x2000 0000

0x2FF2 30000x2FF3 B400

DATA

BSS

HEAP

in /etc/security/limits:fsize =core =cpu =data =rss =stack =

initialized variables & constants



AIX 32-bit large address space layout (1/3)

Large program address space modelis supported to handle programs with requirements for large amounts of uninitialized datathe process user data is relocated to start at segment 3 for as much data as the program needs up to a maximum of eight segments

the program can therefore have up to 2 GB of data maximum attachable shared memory: 11 segments (2.75 GB)

to enable the large address space model, use the -bmaxdata flag of the ldcommand

Other aspects of the program address space remain unchanged:the user stack, kernel stack and user block continue to reside in segment 2as a result of this organizational scheme, the user stack is still limited by the size of segment 2




user blockkernel stackuser stack

unused space

u.ulimit[MAX].stack

0x2000 0000

0x2FF2 30000x2FF3 B4000xF

0x00x10x20x30x40x50x60x70x80x90xA0xB0xC0xD0xE

kernelprocess text

process private user data

shared library textshmat / mmap

shared library data

shmat/mmap

segment 2:




-bmaxdata option of the ld commandto link a program that will have the maximum eight segments reserved to it, the following command line could be used:

cc sample.o -bmaxdata:0x80000000

the number 0x80000000 is the number of bytes, in hex format, equal to eight 256 MB segmentsuse -bmaxdata option if the program needs large data areasuse only multiples of 0x10000000the maxdata value is stored in the o_maxdata field of the XCOFF (Extended Common Object File Format) header (AIX uses XCOFF as binary format)



Check and set the o_maxdata field (1/2)

Check the setting of the o_maxdata field

root@aix:/> dump -ov

.....

maxSTACK maxDATA SNbss magic modtype

0x00000000 0x80000000 0x0003 0x010b 1L

Script to set the o_maxdata field to 0x80000000 for any binary:

# patch 32-bit binary to use the maximum number of 8 x 256MB data segments

f=

if [[ "`/usr/bin/file ${f} | awk '{ print $2 }'`" = "executable" ]] ; then

/usr/bin/echo '\0200\0\0\0' | \

/usr/bin/dd of=${f} bs=4 count=1 seek=19 conv=notrunc

fi

Alternatively, set the "LDR_CNTRL=MAXDATA=0x80000000" environment variable



Check and set the o_maxdata field (2/2)

You can also modify an existing program so that it will use the large or very large address-space model.To set the maxdata value in an existing 32-bit XCOFF program, a.out, to 0x80000000, use the following command:

/usr/ccs/bin/ldedit -bmaxdata:0x80000000 a.out




Recommended compile environment



My compile environments (1/2)

Compiling on AIX5L v5.1 AIX 5.1 ML 04 plus some selected ML 05 filesets IBM XL C/C++ compiler version 7 with all applicable patches for AIX 5.1 ML 04

Compiling on AIX5L v5.2 AIX 5.2 ML 01 IBM XL C/C++ compiler version 7 with all applicable patches for AIX 5.2 ML 01

Compiling on AIX5L v5.3 AIX 5.3 ML 01 IBM XL C/C++ compiler version 7 with all applicable patches for AIX 5.3 ML 01

Compiling on AIX v6.1 AIX 6.1 TL 00 IBM XL C/C++ compiler version 9 with all applicable patches for AIX 6.1 TL 00

Compiling on AIX v7.1 AIX 7.1 TL 00 IBM XL C/C++ compiler version 11.1 with all applicable patches for AIX 7.1 TL 00



My compile environments (2/2)

Remarks:I always use the BASH shell if possibleMain development system is AIX5L v5.1For the different AIX versions I use the environment settings listed on the following slidesIf any different settings are required they are explicitely specified in the RPM spec file

Strategy:Try to build every library as a shared library

You can always statically link a shared library in AIX!Build always 32-bit and 64-bit versions

http://gnu.org/software/bash/



Environment settings for IBM XL C/C++ (1/3)

AIX5L v5.1:

export CONFIG_SHELL=/opt/freeware/bin/bashexport CONFIG_ENV_ARGS=/opt/freeware/bin/bash

export CC=ccexport CFLAGS="-qmaxmem=16384 -DSYSV -D_AIX -D_AIX32 -D_AIX41 -D_AIX43 -D_AIX51 \

-D_ALL_SOURCE -DFUNCPROTO=15 -O -I/opt/freeware/include"

export CXX=xlCexport CXXFLAGS=$CFLAGS

export F77=xlfexport FFLAGS="-qmaxmem=16384 -O -I/opt/freeware/include"

export LD=ldexport LDFLAGS="-L/opt/freeware/lib -Wl,-bmaxdata:0x80000000"

export PATH=/usr/bin:/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/vac/bin: \/usr/vacpp/bin:/usr/ccs/bin:/usr/dt/bin:/usr/opt/perl5/bin:/opt/freeware/bin: \/opt/freeware/sbin:/usr/local/bin:/usr/lib/instl




AIX5L v5.2:Same as for AIX5L v5.1 except the definition of $CFLAGS

export CFLAGS="-qmaxmem=16384 -DSYSV -D_AIX -D_AIX32 -D_AIX41 -D_AIX43 \-D_AIX51 -D_AIX52 \-D_ALL_SOURCE -DFUNCPROTO=15 -O -I/opt/freeware/include"


export CFLAGS="-qmaxmem=16384 -DSYSV -D_AIX -D_AIX32 -D_AIX41 -D_AIX43 \-D_AIX51 -D_AIX52 -D_AIX53 \-D_ALL_SOURCE -DFUNCPROTO=15 -O -I/opt/freeware/include"




AIX v6.1:Same as for AIX5L v5.1 except the definition of $CFLAGS

export CFLAGS="-qmaxmem=16384 -DSYSV -D_AIX -D_AIX32 -D_AIX41 -D_AIX43 \-D_AIX51 -D_AIX52 -D_AIX53 -D_AIX61 \-D_ALL_SOURCE -DFUNCPROTO=15 -O -I/opt/freeware/include"


export CFLAGS="-qmaxmem=16384 -DSYSV -D_AIX -D_AIX32 -D_AIX41 -D_AIX43 \-D_AIX51 -D_AIX52 -D_AIX53 -D_AIX61 -D_AIX71 \-D_ALL_SOURCE -DFUNCPROTO=15 -O -I/opt/freeware/include"



Environment settings for GCC (1/3)

AIX5L v5.1:

export CONFIG_SHELL=/opt/freeware/bin/bashexport CONFIG_ENV_ARGS=/opt/freeware/bin/bash

export CC=gccexport CFLAGS="-DSYSV -D_AIX -D_AIX32 -D_AIX41 -D_AIX43 -D_AIX51 \

-D_ALL_SOURCE -DFUNCPROTO=15 -O -I/opt/freeware/include"

export CXX=g++export CXXFLAGS=$CFLAGS

export F77=g77export FFLAGS="-O -I/opt/freeware/include"

export LD=ldexport LDFLAGS="-L/opt/freeware/lib -Wl,-bmaxdata:0x80000000"

export PATH=/usr/bin:/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/vac/bin: \/usr/vacpp/bin:/usr/ccs/bin:/usr/dt/bin:/usr/opt/perl5/bin:/opt/freeware/bin: \/opt/freeware/sbin:/usr/local/bin:/usr/lib/instl





export CFLAGS="-DSYSV -D_AIX -D_AIX32 -D_AIX41 -D_AIX43 -D_AIX51 \-D_AIX52 \-D_ALL_SOURCE -DFUNCPROTO=15 -O -I/opt/freeware/include"


export CFLAGS="-DSYSV -D_AIX -D_AIX32 -D_AIX41 -D_AIX43 -D_AIX51 \-D_AIX52 -D_AIX53 \-D_ALL_SOURCE -DFUNCPROTO=15 -O -I/opt/freeware/include"





export CFLAGS="-DSYSV -D_AIX -D_AIX32 -D_AIX41 -D_AIX43 -D_AIX51 \-D_AIX52 -D_AIX53 -D_AIX61 \-D_ALL_SOURCE -DFUNCPROTO=15 -O -I/opt/freeware/include"


export CFLAGS="-DSYSV -D_AIX -D_AIX32 -D_AIX41 -D_AIX43 -D_AIX51 \-D_AIX52 -D_AIX53 -D_AIX61 -D_AIX71 \-D_ALL_SOURCE -DFUNCPROTO=15 -O -I/opt/freeware/include"



Large file ( 2GB) support for 32-bit AIX binaries

Must define the following CPP (C preprocessor) symbol:

-D_LARGE_FILES

Sometimes in source code you find also defines like

-D_FILE_OFFSET_BITS=64-D_LARGEFILE_SOURCE

but they are only Linux-centric and not necessary for AIX!

Recommended settings:

export CC="xlc_r -D_LARGE_FILES"




Shared libraries on AIX



Some facts about shared libraries on AIX (1/5)

Objects files are defined by XCOFF (eXtended Common Object File Format) on AIX.AIX provides two different development environments:

32-bit 64-bit

Object files on AIX have two different bit modes: 32-bit object files 64-bit object files

To build executable files, all object files and archive members must be in the same bit mode, either 32 or 64.To distinguish the bit mode of object files, use the file command:

$ file *.ofoo1.o: executable (RISC System/6000) or object module not strippedfoo2.o: executable (RISC System/6000) or object module not strippedfoo3.o: 64-bit XCOFF executable or object module not stripped

In this example, foo3.o is a 64-bit object, whereas foo1.o and foo2.o are 32-bit.

AIX object files are always position independent code (PIC), i.e., no absolute jumps, only relative jumps within the code segment.The GCC "fPIC" option is not necessary on AIX.




Multiple object files can be archived into a single library archive file (libfoo.a).

Hybrid mode library archivesAn AIX library archive can contain both the 32- and 64-bit object modules as its members.This is called a hybrid mode library archive. Most AIX system libraries are hybrid mode.

Utility commandsUtility commands that deal with object files, such as ar, dump, and nm, have been enhanced with the -X option in order to support the 64-bit XCOFF format object format.Possible options (with or without space):-X32-X64-X32_64

Alternatively set the OBJECT_MODE variable




Difference between shared and static objects on AIXOn many UNIX operating systems, a shared object file normally has a file name extension ".so" (stands for shared object) and a static object file has normally a file name extension ".o" (stands for object).However, an object normally has a file name extension ".o" regardless of shared or static on AIX.Therefore, you cannot determine whether an object is shared or regular static object from the file name extension on AIX.To determine whether the object file is shared or static, use the dumpcommand.

Note:AIX also supports shared objects with a file name extension ".so" like other UNIX operating systems do, which are used for run-time linking.




Difference between the terms shared library and shared object on AIX:Shared object

A shared object is a single object file that has the SHROBJ flag in the XCOFF header.

A shared object normally has a name of the form name.o on AIX.Shared library

A shared library refers to an ar format archive library file, where one or more of the archive members is a shared object.

Note that the library can also contain regular, non-shared object files, which are handled in the normal way by the linker.

A shared library normally has a name of the form libname.a on AIX. The magic number of the file is used by the linker to determine whether the file is

valid object file or not.

Note:Shared libraries (also often referred to as dynamic link libraries or DLL) on other UNIX operating systems are actually shared objects, whereas multiple shared objects can be contained in a single library on AIX.




Shared objects comparison with Linux:

AIX: /usr/lib/libfoo.a contains the shared object foo.so.1 For different versions of foo.so the library archive /usr/lib/libfoo.a must

contain these different versions, e.g., foo.so.1, foo.so.2, foo.so.3

Linux: /usr/lib/libfoo.so.1 is the shared object, compiled with "-fPIC" Different versions of foo.so are just part of the file system, i.e.,

/usr/lib/libfoo.so.1/usr/lib/libfoo.so.2/usr/lib/libfoo.so.3/usr/lib/libfoo.so > libfoo.so.3

/usr/lib/libfoo.a contains the statically compiled (without "-fPIC") members of foo



Hardcoded shared library search path

To check use the dump -Hv command:

root@aix51:/> dump -Hv ......***Import File Strings***INDEX PATH BASE MEMBER0 /opt/freeware/lib:/usr/vac/lib:/usr/lib:/lib1 libc.a shr.o2 /opt/freeware/lib libintl.a libintl.so.1

To hardcode it in your 32-bit binary use:

export LDFLAGS="-L/opt/freeware/lib \-Wl,-blibpath:/opt/freeware/lib:/usr/lib:/lib \-Wl,-bmaxdata:0x80000000"

To hardcode it in your 64-bit binary use:

export LDFLAGS="-L/opt/freeware/lib64 -L/opt/freeware/lib \-Wl,-blibpath:/opt/freeware/lib64:/opt/freeware/lib:/usr/lib:/lib \-Wl,-bmaxdata:0x80000000"



Recommended further reading

IBM Redbook SG24-5674:Developing and Porting C and C++ Applications on AIXURL: http://www.redbooks.ibm.com/abstracts/sg245674.html

IBM Redbook SG24-6034:AIX 5L Porting GuideURL: http://www.redbooks.ibm.com/abstracts/sg246034.html

http://www.redbooks.ibm.com/abstracts/sg245674.htmlhttp://www.redbooks.ibm.com/abstracts/sg246034.html




GCC on AIX



GCC availability chart (from my website) (1/2)

AIX5L V5.1 AIX5L V5.2 AIX5L V5.3 AIX V6.1 AIX V7.1

GCC V4.8.2 yes yes yes

GCC V4.8.1 yes yes yesGCC V4.8.0 yes yes yesGCC V4.7.3 yes yes yesGCC V4.7.2 yes yes yesGCC V4.7.1 yes yes yesGCC V4.7.0 yes yes yesGCC V4.6.4 soon yes yes yes

GCC V4.6.3 yes yes yes yes

GCC V4.6.2 yes yes yes yesGCC V4.6.1 yes yes yes yesGCC V4.6.0 yes yes yes

continued...



GCC availability chart (from my website) (2/2)

AIX5L V5.1 AIX5L V5.2 AIX5L V5.3 AIX V6.1 AIX V7.1

GCC V4.5.4 yes yes yes yes yes


GCC V4.5.2 yes yes yes











GCC error message

"gcc -g ..." fails with an error message similar like that:

ld: 0711-596 SEVERE ERROR: Object xxx.o

An RLD for section 2 (.data) refers to symbol 111,

but the storage class of the symbol is not C_EXT or C_HIDEXT.

The source file contains implicitly initialized global symbols.

Interim fix:

# cd /usr/ccs/bin

# mv as as.ORIG

# cp as.61TL00# ln -s as.61TL00 as



List of APARs with fixes available now:

AIX V7.1 TL00: IZ99107 IZ99107: ASSEMBLER GENERATES INVALID OBJECT FILE WITH GCC-PRODUCED FILES fixed in AIX V7.1 TL01 SP0, i.e., must upgrade to TL01 (7100-01-00-1140, bos.adt.base.7.1.1.0) until AIX V7.1 TL00 SP4 is available

AIX V6.1 TL06: IZ98226 IZ98226: ASSEMBLER GENERATES INVALID OBJECT FILE WITH GCC-PRODUCED FILES fixed in AIX V6.1 TL06 SP6 (6100-06-06-1140, bos.adt.base.6.1.6.16)

AIX V6.1 TL05: IZ98732 IZ98732: ASSEMBLER GENERATES INVALID OBJECT FILE WITH GCC-PRODUCED FILES fixed in AIX V6.1 TL05 SP7 (6100-05-07-1140, bos.adt.base.6.1.5.3)

AIX V6.1 TL04: IV00150 IV00150: ASSEMBLER GENERATES INVALID OBJECT FILE WITH GCC-PRODUCED FILES fixed in AIX V6.1 TL04 SP11 (6100-04-11-1140, bos.adt.base.6.1.4.4)

AIX V6.1 TL03: IZ98861 IZ98861: ASSEMBLER GENERATES INVALID OBJECT FILE WITH GCC-PRODUCED FILES as AIX V6.1 TL03 is not supported anymore, no fix for TL03 is available, instead you have to upgrade to a later TL

AIX5L V5.3 TL12: IZ98134 IZ98134: ASSEMBLER GENERATES INVALID OBJECT FILE WITH GCC-PRODUCED FILES APPLIES TO AIX 5300-12 fix available at https://www-304.ibm.com/support/docview.wss?uid=isg1fixinfo123037 fix contained in AIX5L V5.3 TL12 SP4



https://www-304.ibm.com/support/docview.wss?uid=isg1IZ99107https://www-304.ibm.com/support/docview.wss?uid=isg1IZ98226https://www-304.ibm.com/support/docview.wss?uid=isg1IZ98732https://www-304.ibm.com/support/docview.wss?uid=isg1IV00150https://www-304.ibm.com/support/docview.wss?uid=isg1IZ98861https://www-304.ibm.com/support/docview.wss?uid=isg1IZ98134https://www-304.ibm.com/support/docview.wss?uid=isg1fixinfo123037https://www-304.ibm.com/support/docview.wss?uid=isg1fixinfo123037https://www-304.ibm.com/support/docview.wss?uid=isg1fixinfo123037https://www-304.ibm.com/support/docview.wss?uid=isg1IZ98477https://www-304.ibm.com/support/docview.wss?uid=isg1IZ98385




Compatibility issue with the originalAIX Toolbox for Linux Applications



Compatibility issue with the original AIX Toolbox for Linux Applications (1/3)

I strive to keep my RPM packages 100%-compatible with the RPM packages of the AIX Toolbox for Linux Applications.However, there are some things beyond my control. Whenever the rpm.rte AIX LPP gets updated for whatever reason, e.g., a technology level update problems may arise if you have installed newer versions of the following packages from my website:

bzip2 gettext info popt zlib

http://www.perzl.org/aix/index.php?n=Main.Bzip2http://www.perzl.org/aix/index.php?n=Main.Gettexthttp://www.perzl.org/aix/index.php?n=Main.Infohttp://www.perzl.org/aix/index.php?n=Main.Popthttp://www.perzl.org/aix/index.php?n=Main.Zlib




This update of rpm.rte blindly overwrites any newly installed versions of those RPM packages in /opt/freeware/lib. I consider this behavior broken but there is nothing in my power to change that. Newer versions of rpm.rte at least issue a statement during installation like that:

Warning: the contents of the following RPM packages will be overwritten.

If these packages did not come from the AIX Toolbox for Linux Applications,

it may be necessary to reinstall them.

bzip2-1.0.5-1

gettext-0.17-1

info-4.12-1

zlib-1.2.3-5




Most problems arose with updated versions of the gettext package.Therefore, if you get the following error message:

...

0509-150 Dependent module /opt/freeware/lib/libintl.a(libintl.so.8) could not be loaded.

0509-152 Member libintl.so.8 is not found in archive

...

you have two options (number #1 is recommended):1. Please update to a later version of the RPM package as I have since then

recompiled all my packages to only use the basic versions of the AIX Toolbox.2. Please re-install the current gettext-0.17 RPM from my website again with the

following command as root user: rpm -Uvh --force gettext-0.17*rpm

Note: Please do not update the gettext RPM!

http://www.perzl.org/aix/index.php?n=Main.Gettext




AIX V6.1 and V7.1 specifics



AIX V6.1 and V7.1 specifics (1/2)

Situation:The configure script does not detect the shared library capability on AIX V6.1/7.1

Check the following items:Check that your have autoconf, automake, libtool, m4 versions are not older than January, 2008!

Older versions still have explicit checks like aix4* | aix5*) in these scripts, thus neglecting AIX V6.1/7.1 while the newer versions have checks like [[aix4-9]]*)

Check that the config.guess script knows about AIX V6.1 and V7.1! config.guess tries to guess the correct value for the OS

Wrong: rs6000-ibm-aix Correct: powerpc-ibm-aix6.1.0.0

A grep AIX config.guess must return the following output: Wrong: *:AIX:*:[45]) Correct: *:AIX:*:[4567]) or *:AIX:*:[4-9])

Check if the situation after an autoreconf has improved?Still no solution in sight? contact me



AIX V6.1 and V7.1 specifics (2/2)

AIX V6.1 and V7.1 specific versions are not necessary i.e., all RPM packages compiled on previous AIX versions should be fine due to the excellent AIX binary compatibility except when:

newer X11 libraries are required, as AIX V6.1/7.1 has rather recent X11 librariese.g., libXcursor, libXpm, libXrender etc.

certain AIX features e.g., AME (Active Memory Expansion) are required. platform-specific and OS level-specific header files are required, e.g., for GCC, lsof.




Summary



Summary

Open Source Software is important to AIX as our clients are using it

A viable alternatives to the AIX Toolbox for Linux Applications existsplease use my personal website now

Building Open Source Software on AIX has become rather easy nowadays The AIX APIs are more "mainstream" than in earlier years The Open Source community in the meantime has learnt do deal with

the "quirks" and peculiarities of AIX



Questions ?

Thank you for your attention !Thank you for your attention !




Backup charts




The GNU build system aka autotools



Building open source software yourself

Advantages:It is in many cases really easy to do, e.g., GNU software only requires

$ ./configure$ make$ make install

Disadvantages:Requires ressources, e.g., application development environment and compilerNo automatic software dependency checkingYou have to do the "regression testing"Not easily transferable from one system to another without using a package management system



The GNU build system aka autotools (1/2)

Properties of the GNU build system aka autotoolsA suite of programming tools produced by the GNU projectThese tools are designed to assist in making various source code packages portable to many Unix-like systems.The GNU build system is part of the GNU toolchain and is widely used in many free software and open source packages.While the tools comprising the GNU build system themselves are GPL-ed free software, there are no restrictions in using them in making non-free software portable.



The GNU build system aka autotools (2/2)

Tools included in the GNU build systemAutoconfAutomakeLibtool

Other related tools frequently used with the GNU build system areGNU makeGNU gettextpkg-configGNU Compiler Collection, also called GCC



GNU autoconf

Autoconf processes files (configure.in or configure.ac) with the help of GNU m4 to generate a configure script.When running the generated configure script, other template files, customarily ending in ".in", such as Makefile.in, are processed to create their final output, in this case a Makefile.Autoconf is used to attempt to work around the quirks found in various Unix-like operating systems.

For example, some Unix-like systems may have certain facilities that are known to be broken or missing entirely.

Autoconf creates a shell script which can detect these, and the program can work around them.

The final output of the Autoconf utility is the configure script.Autoconf comes with several auxiliary programs designed to ease the creation of configure.ac:

autoheader, which is used to help manage C header files autoscan, which can create an initial input file for autoconf ifnames, which can list C pre-processor identifiers used in the program.



GNU automake and libtool

GNU automake:Automake aims to allow the programmer to write a makefile in a higher-level language, rather than having to write the whole makefile manually.Automake helps to create portable Makefiles, which are in turn processed with the make utility. It takes its input as Makefile.am and turns it into Makefile.in, which is used by the configure script to generate the file Makefile output.

GNU libtool:Libtool helps manage the creation of static and dynamic libraries on various Unix-like operating systems.Libtool accomplishes this by abstracting the library creation process, hiding differences between various systems (e.g., Linux vs. Solaris, AIX etc.).



What if my software has no configure script ?

One option is to execute a series of commands at a shell prompt:

$ aclocal$ autoconf$ autoheader$ automake

Another common convention is to provide a script, often named autogen.sh, that runs all the needed pre-build tools.

In some cases, one can also use:

$ autoreconf

which automatically invokes the above commands as needed.



Flow diagram of autoconf and automake

autoheader

autoscan

aclocal automake

autoconf

make

editor

configure.ac Makefile.am

aclocal.m4 config.h.in Makefile.in

Makefile

configure

Developer

User




Building RPM packages on AIX



Building RPM packages on AIX

The RPM package managerIntroduction and overviewBest practices

General documentation about RPM available at:http://wiki.rpm.org/Docshttp://www.rpm.org/RPM-HOWTO/

http://wiki.rpm.org/Docshttp://www.rpm.org/RPM-HOWTO/




Building RPM packages on AIX the RPM package manager



The RPM package manager

RPM is the RPM Package Manager.It is an open packaging system available for anyone to use.It maintains a database of all packages and their files that can be used for verifying packages and querying for information about files and/or packages.Originally developed by Red Hat, Inc., it has become the package manager for most Linux distributions.

RPM on AIX:RPM on AIX is installed via the BFF package rpm.rteAIX still uses version 3 of RPM while all Linux distributions are now at version 4RPM version 4 has split the rpm command into rpm and rpmbuild.The directory prefix for all RPM software on AIX is /opt/freeware.



Directory structure of RPM

Directory structure under /opt/freeware/src/packages:

BUILD here the RPM package is extracted and compiledRPMS

noarch generated architecture-independent RPMsppc generated binary RPMs for AIX

SOURCES all sources and patchesSPECS all spec filesSRPMS generated source RPMs



Useful RPM commands (1/2)

Query RPM information:Show all installed RPM filesrpm -qa

List contents of a specfic RPM filerpm -qpl

Show information about an RPM filerpm -qpi

Show package requirements of an RPM filerpm -qp --requires

To which RPM file belongs the file /usr/bin/abc123 ?rpm -qf /usr/bin/abc123

Show pre/post install scripts of an RPM filerpm -qp --scripts

Show changelog history of an RPM filerpm -qp --changelog



Useful RPM commands (2/2)

Installation and Removal of RPM files:Install an RPM filerpm -ivh

Update an RPM file (typically better than the -ivh" option)rpm -Uvh

Update (Freshen) only installed RPM filesrpm -Fvh *.rpm

Remove an RPM filerpm -e

Verification of RPM files:Verify complete RPM database (did you erase a file by accident ?)rpm -Va




Building RPM packages on AIX introduction and overview



Building RPM packages

Basic procedure to build an RPM:Get the source code you are building the RPM for to build on your system.Make a patch of any changes you had to make to the sources to get them to build properly.

RPM by design uses the pristine sources from the original web site Any necessary changes are to be supplied via patch files

Make a spec file for the package.Make sure everything is in its proper place.Build the package using RPM.



The Spec File dissected"

The spec file contains the build instructions for the package:Header%prep

%build

%install

%clean

optional pre and post install/uninstall scripts %pre %post %preun %postun

%files

%changelog



The RPM header (1/5)

Header standard fields that must be filled:Summary:

This is a one line description of the package.Name:

This must be the name string from the rpm filename you plan to use.Version:

This must be the version string from the RPM filename you plan to use.Release:

The release number for a package of the same version (i.e., if we make a package and find it to be slightly broken and need to make it again, the next package would be release number 2).

Copyright: This line tells how a package is copyrighted. You should use something like GPL, BSD, MIT, public domain, distributable, or

commercial.




Group: This line is used to tell high level installation programs where to place this particular program in

its hierarchical structure. The group tree currently looks something like this:

Development/DebuggersDevelopment/LanguagesDevelopment/LibrariesDevelopment/SystemDevelopment/ToolsDocumentationSystem Environment/BaseSystem Environment/DaemonsSystem Environment/KernelSystem Environment/LibrariesSystem Environment/ShellsUser Interface/DesktopsUser Interface/XUser Interface/X Hardware Support

Amusements/GamesAmusements/GraphicsApplications/ArchivingApplications/CommunicationsApplications/DatabasesApplications/EditorsApplications/EmulatorsApplications/EngineeringApplications/FileApplications/InternetApplications/MultimediaApplications/ProductivityApplications/PublishingApplications/SystemApplications/Text




Source: This line points at the HOME location of the pristine source file It is used if you ever want to get the source again or check for newer versions Note:

The filename in this line must match the filename you have on your system, i.e., don't download the source file and change its name

You can also specify more than one source file using lines like:

Source0: foo0.tar.gzSource1: bar1.tar.gzSource2: abc.tar.gz

These files would go in the SOURCES directory




Patch: This is the place you can find the patch if you need to download it again Note:

The filename here must match the one you use when you make your patch. You may also want to note that you can have multiple patch files much as you

can have multiple sources. You would have something like:

Patch0: foo0.patchPatch1: bar1.patchPatch2: abc.patch

These files would go in the SOURCES directory




BuildRoot: This line allows you to specify a directory as the "root" for building and installing the

new package. You can use this to help test your package before having it installed on your

machine.

%description It's not really a header item, but should be described with the rest of the header. You need one description tag per package and/or subpackage. This is a multiline field that should be used to give a comprehensive description of

the package.



%prep (1/3)

Second section in the spec fileUsed to get the sources ready to buildHere you need to do anything necessary to get the sources patched and setup like they need to be setup to do a make.

Things to note:Each of these sections is really just a place to execute shell scripts.You could simply make an sh script and put it after the %prep tag to unpack and patch your sources.Macros exist to help you.



%prep (2/3)

The %setup macroIn its simplest form (no command line options), it simply unpacks the sources and cd's into the source directory.Options:

n name will set the name of the build directory to the listed name. The default is %{name}%{version}.

c will create and cd to the named directory before doing the untar.

b # will untar Source# before cd'ing into the directory (and this makes no sense with c so don't do it). Only useful with multiple source files

a # will untar Source# after cd'ing into the directory

T This option overrides the default action of untarring the source and requires a b 0 or a 0 to get the main

source file untarred You need this when there are secondary sources

D Do not delete the directory before unpacking Only useful where you have more than one setup macro Should only be used in setup macros after the first one (but never in the first one)



%prep (3/3)

The %patch macroThis macro helps automate the process of applying patches to the sources.Options:#

will apply Patch# as the patch filep #

specifies the number of directories to strip for the GNU patch(1) commandp

The default action is to apply Patch (or Patch0). This flag inhibits the default action and will require a 0 to get the main source file untarred. This option is useful in a second (or later) %patch macro that required a different number than the first macro.

b extension will save originals as filename.extension before patching.

You can also do %patch# instead of doing the real command: %patch # p

After those macros you can do any other setup you need to do via sh type scripting. Anything you include up until the %build macro (discussed in the next slide) is executed via sh.



%build

There aren't really any macros for this section.You should just put any commands here that you would need to use to build the software once you had untarred the source, patched it, and cd'ed into the directory.This is just another set of commands passed to sh, so any legal sh commands can go here (including comments).Your current working directory is reset in each of these sections to the toplevelof the source directory, so keep that in mind.

You can cd into subdirectories if necessary.The variable RPM_OPT_FLAGS is set using values in /usr/freeware/lib/rpm/rpmrc.

Look there to make sure you are using values appropriate for your system

or

simply don't use this variable in your spec file as it is optional. Instead define your compile settings.



%install

There aren't really any macros for this section.You basically just want to put whatever commands here that are necessary to install.If you have make install available to you in the package you are building, put that here.If not, you can either patch the makefile for a make install and just do a make install here, or you can hand install them here with sh commands.You can consider your current directory to be the toplevel of the source directory.The variable RPM_BUILD_ROOT is available to tell you the path set as the BuildRoot: in the header.

Using build roots are optional but are highly recommended because they keep you from cluttering your system with software that isn't in your RPM database.



%clean

It's a good idea to always make sure there is a clean build root before building a package a second time on a system.The %clean macro will help with that.Simply put the proper commands there to remove a former build root.It is suggested to test that RPM_BUILD_ROOT wasn't set to / before doing something this volatile:

%clean[ "${RPM_BUILD_ROOT}" != "/" ] && rm -rf ${RPM_BUILD_ROOT}



Optional pre and post Install/Uninstall Scripts

You can put scripts in the spec file that are run before and after the installation and uninstallation of binary packages.

Macros for each of the scripts: %pre is the macro to do preinstall scripts %post is the macro to do postinstall scripts %preun is the macro to do preuninstall scripts %postun is the macro to do postuninstall scripts

The contents of these sections should just be any sh style script, though you do not need the #!/bin/sh.



%files (1/2)

Here you must list the files for the binary package.RPM has no way to know what binaries get installed as a result of make install.

Available macros:%docused to mark documentation in the source package that you want installed in a binary install

The documents will be installed in /opt/freeware/doc/%{name}%{version}%{release}.

You can list multiple documents on the command line with this macro, or you can list them all separately using a macro for each of them.

%configused to mark configuration files in a package

This includes files like sendmail.cf, passwd, etc. If you later uninstall a package containing config files, any unchanged files will be removed and

any changed files will get moved to their old name with a .rpmsave appended to the filename. You can list multiple files with this macro as well.



%files (2/2)

%dirmarks a single directory in a file list to be included as being owned by a package

By default, if you list a directory name WITHOUT a %dir macro, EVERYTHING in that directory is included in the file list and later installed as part of that package.

%defattrallows you to set default attributes for files listed after the defattr declaration

The attributes are listed in the form (mode, owner, group) where the mode is the octal number representing the bit pattern for the new permissions (like chmod would use), owner is the username of the owner, and group is the group you would like assigned.

You may leave any field to the installed default by simply placing a in its place.%files f allows you to list your files in some arbitrary file within the build directory of the sources

This is nice in cases where you have a package that can build it's own filelist. You then just include that filelist here and you don't have to specifically list the files.

The biggest caveat in the file list is listing directories. If you list /usr/bin by accident, yourbinary package will contain every file in /usr/bin on your system.



%changelog

Log of what changes occurred when the package is updatedIf you are modifying an existing RPM it is a good idea to list what changes you made here.

Simple format:Start each new entry with a line with a '*' followed by the date, your name, and your email address.The date should appear in the same format that is output by:

date +"%a %b %d %Y"

The rest of the section is a free text field, but should be organized in some coherent manner.



Build the Package with RPM (1/2)

Once you have a spec file, you are ready to try and build your package.The most useful way to do it is with a command like the following:

rpm ba mypackage1.01.spec

Other useful options with the b switch:p just run the %prep section of the spec filel do a list check on %filesc do a %prep and %buildi do a %prep, %build, and %installb do a %prep, %build, %install, and %build a binary package onlya build it all (both source and binary packages)



Build the Package with RPM (2/2)

There are several modifiers to the b switch:shortcircuit

will skip straight to a specified stage (can only be used with c and i)clean

removes the build tree when donekeeptemps

will keep all the temp files and scripts that were made in /tmp You can see what files were created in /tmp using the v option

test does not execute any real stages, but does keeptemp




Building RPM packages on AIX Best Practices



Best Practices

First, compile the software before starting with the SPEC fileEliminate any compile errors, i.e., produce appropriate AIX patchesIf the software permits most GNU software does run a "make check"Do a "test-installation" into some temporary directory

Do a "./configure prefix="Then, start with a SPEC file templateFor libraries, do a 32-bit build first

Once this works, try the 64-bit build Then combine the two

From the "test-installation" you can already determine what gets installed and where

Put this information into the %files sectionCheck if a "rpm bb " is successful ?

If yes, then also build the source RPM



Some more hints...

Useful RPM macro: %ifos possible values include: aix5.1, aix5.2, aix5.3, aix6.1, aix7.1

%ifos aix5.1%define buildhost powerpc-ibm-aix5.1.0.0Requires: AIX-rpm >= 5.1.0.0%endif

Sometimes GNU make is required instead of AIX make

%{_tmppath} on AIX resolves to /var/tmp

How to extract the abc-1.2.3-1.src.rpm source RPM file into a local directory tmp ?

mkdir tmp cd tmp rpm2cpio /abc-1.2.3-1.src.rpm | /opt/freeware/bin/cpio idm

You need GNU cpio as AIX cpio produces I/O errors.



Spec File Example (1/7)

Name: t1libVersion: 5.1.2Release: 1Summary: PostScript Type 1 font rasterizerGroup: Applications/PublishingLicense: LGPLURL: ftp://sunsite.unc.edu/pub/Linux/libs/graphicsSource0: ftp://sunsite.unc.edu/pub/Linux/libs/graphics/%{name}%{version}.tar.gz# Patches taken from Debian and adapted for AIXPatch0: %{name}-5.1.1-manpages.patchPatch1: %{name}-5.1.1-xglyph-env.patchPatch2: %{name}-5.1.1-t1libconfig.patchBuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-rootBuildRequires: patch

%descriptionT1lib is a rasterizer library for Adobe Type 1 Fonts. It supportsrotation and transformation, kerning underlining and antialiasing. Itdoes not depend on X11, but does provides some special functions for X11.

AFM-files can be generated from Type 1 font files and font subsettingis possible.

The library is available as 32-bit and 64-bit.




%package develSummary: Header files and static libraries for %{name}Group: Development/LibrariesRequires: %{name} = %{version}-%{release}

%description develThis package contains header files and static libraries for %{name}.

If you are compiling a 32-bit program, no special compiler options areneeded.

If you are compiling a 64-bit program, you have to compile and link yourapplication with "cc -q64" or "gcc -maix64".

%prep%setup -qexport PATH=/opt/freeware/bin:$PATH%patch0 -p0%patch1 -p1%patch2 -p0




%build# setup environment for 32-bit and 64-bit buildsexport AR="ar -X32_64"export NM="nm -X32_64"

# first build the 64-bit versionexport CC="cc -q64"export CXX="xlC -q64"export CFLAGS="-O2"./configure \

--prefix=%{_prefix} \--enable-shared --enable-static

make %{?_smp_mflags} without_doc

cp lib/.libs/libt1.so.5 .cp lib/.libs/libt1x.so.5 .make distclean

# now build the 32-bit versionexport CC="cc"export CXX="xlC"./configure \

--prefix=%{_prefix} \--enable-shared --enable-static

make %{?_smp_mflags} without_doc




# add the 64-bit shared objects to the shared library containing already the# 32-bit shared objects${AR} -q lib/.libs/libt1.a ./libt1.so.5${AR} -q lib/.libs/libt1x.a ./libt1x.so.5

%install[ "${RPM_BUILD_ROOT}" != "/" ] && rm -rf ${RPM_BUILD_ROOT}make DESTDIR=${RPM_BUILD_ROOT} install

/usr/bin/strip ${RPM_BUILD_ROOT}%{_bindir}/* || :

# setup environment for 32-bit and 64-bit buildsexport AR="ar -X32_64"

# add the 64-bit shared objects to the shared library containing already the# 32-bit shared objects${AR} -q ${RPM_BUILD_ROOT}%{_libdir}/libt1x.a ./libt1x.so.5

mkdir -p ${RPM_BUILD_ROOT}%{_sysconfdir}/t1libcp examples/FontDataBase ${RPM_BUILD_ROOT}%{_sysconfdir}/t1lib/cp lib/t1lib.config ${RPM_BUILD_ROOT}%{_sysconfdir}/t1lib/chmod 644 ${RPM_BUILD_ROOT}%{_sysconfdir}/t1lib/*




mkdir -p ${RPM_BUILD_ROOT}%{_sbindir}cp debian/t1libconfig ${RPM_BUILD_ROOT}%{_sbindir}/chmod 755 ${RPM_BUILD_ROOT}%{_sbindir}/t1libconfig

mkdir -p ${RPM_BUILD_ROOT}%{_mandir}/man1/mkdir -p ${RPM_BUILD_ROOT}%{_mandir}/man5/mkdir -p ${RPM_BUILD_ROOT}%{_mandir}/man8/cp debian/type1afm.1 ${RPM_BUILD_ROOT}%{_mandir}/man1/cp debian/xglyph.1 ${RPM_BUILD_ROOT}%{_mandir}/man1/chmod 644 ${RPM_BUILD_ROOT}%{_mandir}/man1/*cp debian/FontDataBase.5 ${RPM_BUILD_ROOT}%{_mandir}/man5/chmod 644 ${RPM_BUILD_ROOT}%{_mandir}/man5/*cp debian/t1libconfig.8 ${RPM_BUILD_ROOT}%{_mandir}/man8/chmod 644 ${RPM_BUILD_ROOT}%{_mandir}/man8/*

(cd ${RPM_BUILD_ROOT}for dir in bin include libdo

mkdir -p usr/${dir}cd usr/${dir}ln -sf ../..%{_prefix}/${dir}/* .cd -

done)




%clean[ "${RPM_BUILD_ROOT}" != "/" ] && rm -rf ${RPM_BUILD_ROOT}

%files%defattr(-,root,system,-)%doc Changes LGPL LICENSE README.t1lib-5.1.2%dir %{_sysconfdir}/t1lib%config %{_sysconfdir}/t1lib/t1lib.config%config %{_sysconfdir}/t1lib/FontDataBase%{_bindir}/*%{_sbindir}/*%{_mandir}/man*/*%{_libdir}/*.a/usr/bin/*/usr/sbin/*/usr/lib/*.a

%files devel%defattr(-,root,system,-)%doc doc/t1lib_doc.pdf%{_includedir}/*%{_libdir}/*.la/usr/include/*/usr/lib/*.la




%changelog* Fri Feb 01 2008 Michael Perzl - 5.1.2-1- updated to version 5.1.2* Thu Jan 03 2008 Michael Perzl - 5.1.1-2- included both 32-bit and 64-bit shared objects* Tue Aug 14 2007 Michael Perzl - 5.1.1-1- updated to version 5.1.1* Wed Jan 04 2006 Michael Perzl - 5.1.0-1- first version for AIX V5.1 and higher

TrademarksImportant reminderAgendaOpen source software what are we talking about ?Open source software what are we talking about ?Selected open source licensesSome prominent examples of open source softwareCustomer expectation anda look at the competitionWhy are customers using open source software ?Customer expectationA look at the UNIX competition*A historical view 12 years ago...Do you still remember back in 2001 ?AIX 5L: Linux affinity, why ?AIX 5L: Linux affinity implementationAIX 5L: Linux affinity API approach (1/3)AIX 5L: Linux affinity API approach (2/3)AIX 5L: Linux affinity API approach (3/3)AIX Toolbox for Linux ApplicationsIBM support for AIX Toolbox ApplicationsAIX Toolbox for Linux Applications over the years...The AIX Toolbox for Linux Applications is outdated...Current open source softwareavailability for AIXOpen source repositories for AIX (1/4)Open source repositories for AIX (2/4)Open source repositories for AIX (3/4)Open source repositories for AIX (4/4)My personal website:http://www.perzl.org/aixMy personal AIX open source repository (1/3)My personal AIX open source repository (2/3)My personal AIX open source repository (3/3)Some download statistics from http://www.perzl.org/aixTechnical partTechnical part AgendaAIX 32-bit physical address spaceAIX 32-bit physical address space (1/3)AIX 32-bit physical address space (2/3)AIX 32-bit physical address space (3/3)AIX 32-bit address space layout (1/3)AIX 32-bit address space layout (2/3)AIX 32-bit address space layout (3/3)AIX 32-bit large address space layout (1/3)AIX 32-bit large address space layout (2/3)AIX 32-bit large address space layout (3/3)Check and set the o_maxdata field (1/2)Check and set the o_maxdata field (2/2)Recommended compile environmentMy compile environments (1/2)My compile environments (2/2)Environment settings for IBM XL C/C++ (1/3)Environment settings for IBM XL C/C++ (2/3)Environment settings for IBM XL C/C++ (3/3)Environment settings for GCC (1/3)Environment settings for GCC (2/3)Environment settings for GCC (3/3)Large file ( 2GB) support for 32-bit AIX binariesShared libraries on AIXSome facts about shared libraries on AIX (1/5)Some facts about shared libraries on AIX (2/5)Some facts about shared libraries on AIX (3/5)Some facts about shared libraries on AIX (4/5)Some facts about shared libraries on AIX (5/5)Hardcoded shared library search pathRecommended further readingGCC on AIXGCC availability chart (from my website) (1/2)GCC availability chart (from my website) (2/2)GCC error messageList of APARs with fixes available now:Compatibility issue with the originalAIX Toolbox for Linux ApplicationsCompatibility issue with the original AIX Toolbox for Linux Applications (1/3)Compatibility issue with the original AIX Toolbox for Linux Applications (2/3)Compatibility issue with the original AIX Toolbox for Linux Applications (3/3)AIX V6.1 and V7.1 specificsAIX V6.1 and V7.1 specifics (1/2)AIX V6.1 and V7.1 specifics (2/2)SummarySummaryQuestions ?Backup chartsThe GNU build system aka autotoolsBuilding open source software yourselfThe GNU build system aka autotools (1/2)The GNU build system aka autotools (2/2)GNU autoconfGNU automake and libtoolWhat if my software has no configure script ?Flow diagram of autoconf and automakeBuilding RPM packages on AIXBuilding RPM packages on AIXBuilding RPM packages on AIX the RPM package managerThe RPM package managerDirectory structure of RPMUseful RPM commands (1/2)Useful RPM commands (2/2)Building RPM packages on AIX introduction and overviewBuilding RPM packagesThe Spec File dissected"The RPM header (1/5)The RPM header (2/5)The RPM header (3/5)The RPM header (4/5)The RPM header (5/5)%prep (1/3)%prep (2/3)%prep (3/3)%build%install%cleanOptional pre and post Install/Uninstall Scripts%files (1/2)%files (2/2)%changelogBuild the Package with RPM (1/2)Build the Package with RPM (2/2)Building RPM packages on AIX Best PracticesBest PracticesSome more hints...Spec File Example (1/7)Spec File Example (2/7)Spec File Example (3/7)Spec File Example (4/7)Spec File Example (5/7)Spec File Example (6/7)Spec File Example (7/7)

open source software solutions and best practices for aixidh.ch/ibm_tu_2013/opensource on...

Documents