Solving the Package Problem

Joe Brockmeierjzb@redhat.comTwitter: @jzbhttp://dissociatedpress.net/Open Source Community @ Red HatTwitter: @RedHatOpenhttp://community.redhat.com/

Who's this guy?

Solving the Package Problem

The Packaging Problem We Face

Software Collections

rpm-ostree

Dockah, Dockah, Dockah

Questions

In the Beginning...

In the beginning, all we had was source, and it was a pain to work with. But we didn't know better, and we compiled, and compiled, and compiled...

Then there were package managers... and software installation became easier, and we looked to a New Hope...

Distributions as the Center of the Universe

For a long time, Linux distributions were essentially the center of the universe for getting open source software. You installed Red Hat, Debian, Fedora, or whatever and took the packages that were current because it was the easiest way to consume FOSS projects.

It also ensured that you'd get updates and bug fixes, and life was good...

Developers do not want to be limited to system versions of software

Surely a variation of Murphy's Law must be this:

The system version of Python, PHP, Ruby, Perl, Node.js, or whatever is not the version that your application needs or the version you want to use to build a new application.

It will invariably be too old or too new, or have some minor bug that breaks your application.

(e.g., CloudStack and Tomcat bug...)

Developers want easier ways to deploy complex software from desktop to server

Automate ALL THE THINGS

Software Collections, rpm-ostree, and Docker (oh my)

Developers have a lot of choices, not least of which is just grabbing their packages from upstream (e.g., Ruby).

That way lies madness, and bad things.

Containers, especially Docker, are the new hotness... but 1) not ready for prime time just yet, 2) may not be suitable for many instances, and 3) where do you get the software for Docker?

PaaS see above.

Software Collections...

Let's talk about Software Collections

Do not require changes to RPM

Software Collections are not just a different version packaged for your OS

Let's say you're running Fedora 19 or CentOS 6.x and want to run a different version of LibreOffice. So you go to LibreOffice.org and decide to download 4.2.1. Then you remove the old packages and install the new packages.

Yeah, that's not going to work for system software... so that's not how SCLs work.

Do not overwrite system files

Software Collections are distributed as a set of several components, which provide their full functionality without overwriting system files.

Example: PHP 5.4

On CentOS 6.xPHP 5.4 package is php54This pulls in:php54-php-cli.x86_64php54-php-common.x86_64php54-php-pear.noarchphp54-php-process.x86_64php54-php-xml.x86_64php54-runtime.x86_64

Lives in: /opt/rh/php54/root

Avoid conflicts with system files

Software Collections make use of a special file system hierarchy to avoid possible conflicts between a single Software Collection and the base system installation.

Require minor changes to your existing spec files

To convert a conventional package to a single Software Collection, you only need to make minor changes to the package spec file.

With a single spec file, you can build both the conventional package and the Software Collection package.

Do not conflict with updates on your system

Nifty: Can depend on other SCLs

Let's talk about using SCLs

Software Collections Currently

Red Hat Software Collections went GA on 12 Sept 2013

CentOS announced SCLs 19 February 2014

Fedora has SCLs on Fedora Hosted since 2013:

https://fedorahosted.org/SoftwareCollections/

What's Packaged for Software Collections

Ruby 1.9.3

Python 2.7

Python 3.3

PHP 5.4

Perl 5.16.3

Tech preview Node.js 0.10

MariaDB 5.5

MySQL 5.5

PostgreSQL 9.2

Getting Started

Assuming using a SLC with CentOSyum install centos-release-SCL

yum install php54 (or whatever...)

scl enable php54 application --option

Your application now uses PHP 5.4 ... the rest of the system ignores it.

Python & Django with SCL (by Langdon White):http://red.ht/scldjango

Find packages for CentOS here:
http://mirror.metrocast.net/centos/6.5/SCL/x86_64/

Packaging SCLs

Grab the necessary packages (CentOS or Fedora or RHEL 6.5):
yum install scl-utils scl-utils-build

Instructions on converting an existing package: http://bit.ly/scl-spec-file

For Conversion: spec2scl

General instructions on packaging SCLs: http://bit.ly/pkging-scls

SoftwareCollections.org

What is SoftwareCollections.org?

Upstream community for development of SCLs.

Build and hosting services for collections.

Resources (documentation, forums, mailing lists) for developers/packagers.

An index of packaged software for users of CentOS, Fedora, RHEL, and other RPM-based distributions.

The Lifecycle of Collections

SCLs can be used to provide newer software support on older releases, or (going forward) to provide legacy support on newer releases:Example: Application using Ruby on Rails N deployed on CentOS 6, via SCL. Same application deployed on CentOS 7 (when released) using SCL.

OpenShift leverages SCLs for its cartridges using RHEL supported and non-supported components.

SCLs can be used inside Docker containers to simplify container deployment.

rpm-ostree

The Problem with Packages

RPM (and dpkg) are designed to go one way: forward

Upgrades are difficult to roll back in the event something goes wrong

Switching between two distinct OSes / versions is more or less impossible

What is rpm-ostree?

Derived from ostreeInitially conceived of as a way to parallel install multiple UNIX-like OSes (e.g., Fedora Rawhide and Fedora 20)

git for operating system binaries

Creates an installable tree from RPMs

Not a package manager, but does take on some of the role from package managers

What rpm-ostree Enables

Install one or more operating system trees to a system

Gives atomic updates An update is, essentially, one unit it succeeds or fails

An update can be rolled back

Allows switching between trees

Provides tools for creating tree composes

Current Limitations

Currently, an rpm-ostree tree is an immutable system Doesn't allow for adding packages to a system w/out rebuilding the tree

Build tools are still being developed, but moving quickly

So, anybody heard of this Docker thing?

The Problem with Packages

Deploying complex services / applications is difficult with packages

Packages aren't as portable as we'd likeApplication is developed on CentOS 6, but production is using CentOS 7?

Packaging guidelines can be ... difficult

Packages don't provide any solution for running containerized applications...

Docker: It's Like Deluxe Super Awesome Packaging

Docker is application-centric

Docker containers are portable

Supports versioning for an entire container

Components can be re-used

Allows for supplying ready-to-run services rather than half-configured packages

Buzzword compliant

Docker to the Rescue?

Docker containers: relatively easy to work with

Provide far more services than package systemsApplication isolation

Image format, sharing, API

Allows layering of applicationsOne group provides base image

Another group provides base image + framework/service (e.g., Apache)

Another group provides base image + framework + finished application ready to deploy

Thanks!

Joe Brockmeierjzb@redhat.comTwitter: @jzb

Click to edit the title text format

Click to edit the outline text format

Click to edit the outline text formatSecond Outline LevelThird Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline LevelSeventh Outline LevelEighth Outline LevelNinth Outline Level