abdullah&hiatham 05/02/20111. introduction history of boxgrinder how does boxgrinder work ?...

BoxGrinder and OpenStack

CS526 Semester Project Presentation

Abdullah Sheneamer and Hiatham Ennajah

Department of Computer ScienceUniversity of Colorado at Colorado Springs

Abdullah&Hiatham 05/02/2011 1

• Introduction

• History of BoxGrinder

• How does Boxgrinder work ?

• Installation and Configuration of BoxGrinder

• BoxGrinder of Plugins

• OpenStack

• References

Outline of the Talk


Introduction

• BoxGrinder is an open source suite for building virtual appliances (also known as virtual machines) that are ready to run on various virtualization and Cloud providers or another definition that it is a set of tools used for building appliances (virtual machines) on various platforms (KVM, Xen, VMware, EC2).

• It supports many platforms such as EC2, Xen, KVM and VMware.

• BoxGrinder is able to create appliances based on Fedora, Red Hat Enterprise Linux or CentOS, and is easy extendable to meet any requirements.


Introduction (Cont.)

• KVM (for Kernel-based Virtual Machine) is a full virtualization solution for Linux on x86 hardware containing virtualization extensions (Intel VT or AMD-V).

• Xen is a virtual-machine monitor for IA-32, x86-64, Itanium and ARM architectures. It allows several guest operating systems to execute on the same computer hardware concurrently

• VMware's desktop software runs on Microsoft Windows, Linux, and Mac OS X.

• Appliance definition files are written in YAML


History

• History:

- Who:

-Marek Goldmann

- When:

-BoxGrinder Build 0.6.5 in 04 December 2010.

-BoxGrinder Build 0.7.0 in 19 December 2010

-BoxGrinder Build 0.7.1 in 03 January 2011

-BoxGrinder Build 0.8.0 in 25 January 2011

-BoxGrinder Build 0.8.1 released 22 February 2011

-BoxGrinder Build 0.9.0 Released 09 march 2011

-BoxGrinder Build 0.9.1 in 28 April 2011


How does BoxGrinder work?

• BoxGrinder is creating appliances (images/virtual machines) from simple plain text Appliance Definition Files.

• There are only two simple steps to create an appliance:

1-Create Appliance Definition File

2-Run BoxGrinder

Appliance Definition

BoxGrinder Appliance


How does BoxGrinder work? (Cont.)

• BoxGrinder consists of three projects:

1- BoxGrinder Build - a fast and easy to use command line

tool that is responsible for actually building the appliances.

2- BoxGrinder REST - a server with farm of builder nodes.

Build tasks are distributed to nodes where BoxGrinder

Build is used to build the appliance.The resulting appliance is transferred to the configured destination.

3- BoxGrinder Studio - a web front-end to BoxGrinder

REST designed to provide a friendly graphical user

experience on top of BoxGrinder REST.(Still working on it) BOXGRINDER BUILD

BOXGRINDER REST

BOXGRINDER STUDIO


Installation and Configuration of BoxGrinder

• There are three phases to create boxgrinder:

1- Preparing environment to start building appliances.

- Fedora 13 and 14 doesn’t require preparing anything before installing BoxGrinder – use BoxGrinder RPMs – this will pull all required packages.

2- Installation. - Before you install BoxGrinder make sure you have prepared your

environment or use meta appliance

3- Building appliances

- Before you start make sure you have installed BoxGrinder Build correctly.


Installation

• The easiest way to obtain BoxGrinder Build is to simply add BoxGrinder repo to your environment and use yum.

- You have to execute this command to install BoxGrinder in Fedora.

- “ yum install rubygem-boxgrinder-build” Command

- Other Systems use this command to install BoxGrinder on it.

- “gem install boxgrinder-build “ Command


Building appliances

• Create appliances directory such as “ mkdir appliances “ command.

- The creating is option.• Simple Example:

name (required)This is a name for your appliance. It must be unique in all of your appliances

summaryA small (one sentence or such) summary of the appliance; purpose, goals, etc.

osThis section specifies operating system information for the appliance.Members: name – OS name, default: fedora, version – OS version; this could be a number or string: 1, rawhide, default: 12. password – this is the root password, default: boxgrinder.

hardwareThis section gives the ability to specify virtual hardware requirements.Members: cpus – virtual CPU count, default: 1, memory – memory quantity in MB, default: 256, partitions – partitioning scheme with root mounts and size in GB

packagesThis section contains all of the packages you wish to add to your appliance.

Repos: you can specify additional repositories in order to add packages to your appliance that are sourced from these repositories.


Boxgrinder Plugins

• BoxGrinder Build has a plugin architecture. We can distinguish three types:

- Operating System plugins – generating base appliance for

selected OS.

- Platform plugins – creating converted base appliance for

selected platform.

- Delivery plugins – designed to deliver your new appliance to a specfied location. For example as a tar file to a remote server or just reqister as AMI on EC2.

Appliance Definition

BoxGriinder

Repository

OS Plugin

Platform Plugin

Delivery Plugin

Base Appliance

Converted Appliance

Appliance


• Operating system plugins:

• The goal of this kind of plugin is to create a base image for the selected operating system. Each plugin must inherit the BaseOperatingSystemPlugin class:

• Supported Operating Systems:

Fedora plugin

CentOS plugin

RHEL plugin

• Fedora Operating System Plugin Configuration:

• The configuration of the plugin would have the following three declarations at the begigning of the file:

• Plugins: # Plugin name.

• fedora: # OS supported.

• format: # Disk format to use. Default: raw.

BoxGrinder of Plugins ( Operating System plugins)


Operating System plugins(Cont.)

Example of Fedora-14.appl plugin file:

name:

fedora-14 # Name of Plugin.

os:

name: fedora # Name of OS.

version: 14 # OS Version.

Fedora Operating System Plugin Usage:

In order to run the Fedora Plugin, the following command has to be used;

boxgrinder-build fedora-14.appl


Platform plugins (Cont.)

Platform plugins convert intermediary deliverables produced by the operating system plugin into a selected platform. A platform could be VMware vSphere or Amazon EC2 for example.

There are three kinds of such plugin;

VMware plugin

VirtualBox Plugin

EC2 plugin

EC2 Platform Plugin:

This plugin creates a EC2 disk image. Note that created image isn’t a bundled AMI, it is a disk image prepared to be bundled and delivered by the S3 plugin.

Supported operating Systems:

Fedora – all versions

RHEL – all versions

CentOS – all versionsAbdullah&Hiatham 05/02/2011 14

Platform plugins (Cont.)

EC2 Platform Plugin (Continue):

EC2 Platform Plugin Configuration:

Configuration for this kind of plugins is not required

Ec2 Platform Plugin Usage:

boxgrinder-build applince-name.appl –p ec2


Delivery plugins

Delivery plugins:

A delivery plugin moves the deliverables from a platfrorm or operating system plugin to a selected location type. This could be a local directory, SFTP server, Amazon CloudFront or an Amazon S3 bucket.

There are Four types of delivery Plugins;

Local plugin

SFTP plugin

S3 plugin

EBS Plugin

*The focus will be on the third type “S3 Plugin” as we had dealt with this term when we

used Amazon Web Services.


Delivery plugins(Cont.)

S3 Delivery Plugin:

This plugin delivers artifacts to a S3 bucket. The plugin is able to deliver artifact in three types:

S3: a packaged (.tgz) image with metadata – good for distribution,

Cloudfront: a packaged image with metadata (same as for s3 type) for public download using CloudFront – great for distribution, you need to have CloudFront enabled for your account,

Ami: creates an AMI from selected image and registers it in Amazon EC2. After that the AMI will be visible for you as a private image and ready to run. This type is only available for images in EC2 format (converted using “-p ec2” switch).

Supoerted Operating Systems:


RHEL – all versions

CentOS – all versions

S3 Delivery Plugin Supported Platforms:

EC2



S3 Delivery Plugin Configuration:

This is how would the configuration section of this plugin look like:

In our Example:

plugins:

s3:

access_key: AWS_ACCESS_KEY # (required)

secret_access_key: AWS_SECRET_ACCESS_KEY # (required)

bucket: BoxGrinder # (required)

account_number: 6641-7087-8440 # (required)

path: /images # default: / The path were the image created is located.

cert_file: cert-4NR4SZJMIQM7N4VKKVDOFIDMY6L5QLUP.pem

key_file: pk-4NR4SZJMIQM7N4VKKVDOFIDMY6L5QLUP.pem

host: http://s3.amazonaws.com



S3 Delivery Plugin (Continue):

S3 Delivery Plugin Example:

EC2 AMI for appliance-name.appl:

“boxgrinder-build jeos.appl -p ec2 -d ami” Command


EBS Delivery plugins

This plugin delivers appliance as EBS-based AMI to AWS. Note that only appliances converted to EC2 format using EC2 platform plugin can be delivered as EBS AMI’s.

*This plugin can only be used on instances running on EC2. This plugin will not work on your local host because we need to mount EBS volume to copy the data and we cannot do a remote mount. You can use meta appliance AMI to create EBS AMI’s.

Supported Operating Systems:


RHEL – Ver. 6

EBS Delivery Plugin Supported Platforms:

EC2

EBS Delivery Plugin Configuration:

This is how would the configuration section of this plugin look like:


EBS Delivery plugins(Cont.)

plugins:

ebs:

access_key: AWS_ACCESS_KEY # required

secret_access_key: AWS_SECRET_ACCESS_KEY # required

account_number: 6641-7087-8440 # required

delete_on_termination: false # default: true

(Note that the delete_on_termination flag is used to specify if the root volume should be deleted after the instance is terminated)

EBS Delivery Plugin Example:

EBS-based AMI for jeos.appl:

“boxgrinder-build appliance-name.appl -p ec2 -d ebs” CommandAbdullah&Hiatham 05/02/2011 21

What is OpenStack?OpenStack is a collection of open source technologies delivering a massively

scalable cloud operating system. OpenStack is currently developing two interrelated projects: OpenStack Compute and OpenStack Object Storage.

OpenStack Compute is software to provision and manage large groups of virtual private servers, and OpenStack Object Storage is software for creating redundant, scalable object storage using clusters of commodity servers to store terabytes or even petabytes of data.

Backed by Rackspace, NASA, Dell, Citrix, Cisco, Canonical and over 50 other organizations, OpenStack has grown to be a global software community of developers, technologists, researchers and corporations collaborating on a standard and massively scalable open source cloud operating system.

History:

In July 2010, Rackspace Hosting and NASA jointly launched a new open source cloud initiative known as OpenStack. The mission of the OpenStack project is to enable any organization to create and offer cloud computing services running on standard hardware. The community's first official release, code-named Austin, was made available just 4 months later with plans to release regular updates of the software every few months.


Why OpenStack?

Control and Flexibility:

Open source platform means you’re never locked to a proprietary vendor, and modular design can integrate with legacy or third-party technologies to meet your business needs. Hypervisor support for Microsoft Hyper-V, Citrix XenServer, Xen, KVM, VMWware ESX, LXC, QEMU, and UML.

Industry Standard:

More than 60 leading companies from over a dozen countries are participating in OpenStack, including Cisco, Citrix, Dell, Intel and Microsoft, and new OpenStack clouds are coming online across the globe.

Proven Software:

Running the OpenStack cloud operating system means running the same software that today powers some of the largest public and private clouds in the world.

Compatible and Connected:

Compatibility with public OpenStack clouds means enterprises are prepared for the future—making it easy to migrate data and applications to public clouds when conditions are right—based on security policies, economics, and other key business criteria.


Why OpenStack?(Cont.)

The OpenStackCloud:

OpenStack is a collaborative software project designed to create freely available code,badly needed standards, and common ground for the benefit of both cloud providers and cloud customers. OpenStack is currently three projects;

*OpenStack Compute

*OpenStack Object Storage

*OpenStack Image Service

OpenStack Compute:

OpenStack Compute is open source software designed to provision and manage large networks of virtual machines, creating a redundant and scalable cloud computing platform. It gives you the software, control panels, and APIs required to orchestrate a cloud, including running instances, managing networks, and controlling access through users and projects.


OpenStack Compute (Cont.):

Popular Use Cases:

* Service providers offering an IaaS compute platform

* IT departments provisioning compute resources to teams and projects

* Processing big data with tools like Hadoop

* Scaling compute up and down to meet demand for web resources and applications

Key Features:

*Complete multi-tenant cloud computing platform scaling to tens of thousands of compute nodes.

*Rich, well documented native OpenStack API with strong ecosystem and support for the Amazon EC2 API.

*Control panels that make it easy to move these workloads between data centers, and even different service providers.


OpenStack Compute (Cont.):OpenStack Compute Control Panel View:


OpenStack Object Storage:

OpenStack Object Storage (code-named Swift) is open source software for creating redundant, scalable object storage using clusters of standardized servers to store petabytes of accessible data. It is not a file system or real-time data storage system, but rather a long-term storage system for a more permanent type of static data that can be retrieved, leveraged, and then updated if necessary.

Primary examples of data that best fit this type of storage model are virtual machine images, photo storage, email storage and backup archiving.

Popular Use Cases: * Service providers offering IaaS storage platform * Integrates with OpenStack Compute to store server images * Document storage * Back-end for Microsoft SharePoint * Archive platform for log files and other data with long retention periods * Store web images and thumbnails


OpenStack Object Storage (Cont.)

Key Features:

* Complete multi-tenant cloud storage platform

* Massive scalability to multi-petabyte size and billions of objects

* Able to store objects of unlimited size

OpenStack Image Service:

OpenStack Image Service (code-named Glance) provides discovery, registration, and delivery services for virtual disk images. The Image Service API server provides a standard REST interface for querying information about virtual disk images stored in a variety of back-end stores, including OpenStack Object Storage. Clients can register new virtual disk images with the Image Service, query for information on publicly available disk images, and use the Image Service's client library for streaming virtual disk images.


OpenStack Image Service (Cont.)

A multi-format image registry, OpenStack Image Service allows uploads of private andpublic images in a variety of formats, including:

* Raw

* Machine (kernel/ramdisk outside of image, a.k.a. AMI)

* VHD (Hyper-V)

* VDI (VirtualBox)

* qcow2 (Qemu/KVM)

* VMDK (VMWare)

* OVF (VMWare, others)

Popular Use Cases:

* Service providers offering an IaaS virtual machine image registry

* Enterprises creating a low cost DR/BCP platform to backup their virtual server images

* ISVs offering pre-configured optimized images of their softwareAbdullah&Hiatham 05/02/2011 29

OpenStack Image Service (Cont.)

Key Features:

*Configured to use standard command line and configuration option processing, making use of paste.deploy configuration

*Registry database is under version control, with migration files allowing upgrade and downgrade of the registry database.

*Versatile combination of "disk format" and "container format" fields, allowing more than just EC2-style image formats to be stored in the registry.


References

• http://cs.uccs.edu/~cs526/cloudcomputing/intro.html

• http://fedoraproject.org/wiki/Features/BoxGrinder

• http://boxgrinder.org/

• http://www.openstack.org/index.php


http://cs.uccs.edu/~cs526/cloudcomputing/intro.html

http://fedoraproject.org/wiki/Features/BoxGrinder

http://boxgrinder.org/

http://www.openstack.org/index.php

Questions ??


abdullah&hiatham 05/02/20111. introduction history of boxgrinder how does boxgrinder work ?...

Documents

boxgrinder build

boxgrinder abdullahhiatham

introduction boxgrinder

boxgrinder rest

boxgrinder work

boxgrinder use boxgrinder

boxgrinder studio

virtual appliances