![Page 1: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/1.jpg)
Cobbler, Func and Puppet:Cobbler, Func and Puppet:Tools for Large Scale EnvironmentsTools for Large Scale Environments
Francesco CrippaFrancesco Crippafcrippa @ fedoraproject,orgfcrippa @ fedoraproject,org
This presentation is licensed under a Creative CommonsAttribution-NonCommercial-ShareAlike (BY-NC-SA) 3.0 license.
![Page 2: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/2.jpg)
what is a“Large Scale Environment”?
![Page 3: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/3.jpg)
From this...
Google first WebFarm
![Page 4: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/4.jpg)
...to this!
CC by Johnnie Walker: http://flickr.com/photos/johnniewalker/359440369
![Page 5: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/5.jpg)
Large scale environments
● Huge number of machines● Huge number of environments
– Production
– Pre-production
– Test
– Develop
● Huge number of people● Need to scale!!!
![Page 6: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/6.jpg)
All is complex
● An example: make a web server on line● In theory it should be easy:
– Buy hardware
– Install operating system (next, next, next...)
– Install a http server (yum install httpd)
– Configure http (vi /etc/httpd/conf/httpd.conf)
– That's all
● ...but...●
![Page 7: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/7.jpg)
All is complex
● In a LSE you need:– 2 web server to maintain service in high availability
– 2 geographical sites to support disaster recovery
– A testing environment to test configurations
– A development environment to setup machines
– Backup
– Monitoring
– Clustering● Shared storage (SAN) required for each cluster
![Page 8: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/8.jpg)
A lot of web servers
1... ...7 servers!
![Page 9: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/9.jpg)
Where is the complexity?
● Web server configuration is simple● But you need to configure:
– 7 different web server (with exactly the same configuration!!!)
– Keep all configurations in sync
– 3 clusters (with different configurations)
– Shared storages (maybe with shared filesystems)
● The complexity is not on the “business”, but on the “infrastructure”
![Page 10: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/10.jpg)
Typical work flow
● Installing OS to a new hardware● Configure machines as you need● Make (sometimes) unconventional tasks
InstallingInstallingO.S.O.S. ConfigurationsConfigurations UnconventionalUnconventional
TasksTasks
![Page 11: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/11.jpg)
Typical work flow
InstallingInstallingO.S.O.S. ConfigurationsConfigurations UnconventionalUnconventional
TasksTasks
COBBLERCOBBLER PUPPETPUPPET FUNCFUNC
![Page 12: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/12.jpg)
All under your control!
Copyright by WarnerBras (Matrix)
![Page 13: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/13.jpg)
Installing OS...COBBLER
![Page 14: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/14.jpg)
OS Installation
● Usually KickStart driven● Two main targets for a new installation:
– Real hardware
– Virtual Machine
● Usually expensive because require some manual human tasks– Put booting cd-rom into reader, configure network
and location of kickstart...
![Page 15: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/15.jpg)
Cobbler
● Cobbler is a Linux provisioning server that allows for rapid setup of network installation environments.
● It keeps in sync:– DHCP
– TFTP
– RPM Repositories
– KickStarts
![Page 16: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/16.jpg)
Importing repositories
● Check configuration files● Import repositories
[root@a~]# cobbler check[root@a~]# cobbler import
mirror=rsync://servergoeshere/path/to/distroname=fedora9
![Page 17: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/17.jpg)
Add profiles and systems
● Distros● Profiles● Systems
[root@a~]# cobbler distro add[root@a~]# cobbler profile add[root@a~]# cobbler system add
![Page 18: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/18.jpg)
Keep in sync
● Generate output in:– /tftpboot
– /var/www/cobbler
– /etc/dhcp.conf
[root@a~]# cobbler sync
![Page 19: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/19.jpg)
KickStart templates
● Kickstart is a template● You can pass values for all your variables
through “cobbler system add”
[root@a~]# cobbler system add ksmeta=”KEY=us”
...keyboard $KEY...
![Page 20: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/20.jpg)
Koan
● Client side● Provisioning for virtual machines ●
● koan --server=server.expl.org --profile=RHEL5-i386
●
● koan -–server=server.expl.org --profile=RHEL5-i386 --virt
![Page 21: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/21.jpg)
Configure all your systemsPUPPET
![Page 22: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/22.jpg)
Configuration Management
– Centralize all configurations into a single place
– Defining “templates” to generalize similar configurations (make one, apply hundreds)
– Manage exceptions
– Make possible for a lot of people to work all together
![Page 23: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/23.jpg)
Working all together
![Page 24: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/24.jpg)
Working all together
![Page 25: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/25.jpg)
Working all together
![Page 26: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/26.jpg)
Working all together
![Page 27: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/27.jpg)
Configuration Items
– Files
– Packages
– Services
– Users
– Groups
– Mount points
– ...
– You can define your own items :-)
– Subscribe
– Require
– Notify
– ...
– Arrays
– Conditions
– ...
![Page 28: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/28.jpg)
A formal definition for HTTPDClass http inherits security-hardening {
file {“/etc/httpd/httpd.conf”:
owner => apache,mode => 644,require => Package[“httpd”]
}
package {“httpd”:
ensure => “latest”}
service {“httpd”:
ensure => “running”,enabled => “true”,subscribe => [
File[“/etc/httpd/httpd.conf”],Package[“httpd”]
]}
}
![Page 29: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/29.jpg)
A formal definition for HTTPD
node 'www.example.com' inherits default {import httpd
}
node 'default' {import baculaimport nagiosimport bacula-backup
}
![Page 30: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/30.jpg)
Puppet architecture
– Client (the puppet) polls the server (the puppet-master) to know if there are new configuration files to download (every 30 minutes by default)
– Client is recognized through its hostname
– Server try to find a right configuration for that hostname, else it uses a “default” config.
– ...caching, md5, checksum and ssl are part of infrastructure
●
![Page 31: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/31.jpg)
Python API for your networkFUNC
![Page 32: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/32.jpg)
Goals
● Manage your hosts as python objects– Groups, ACLs, SSL certificates, ...
[root@a~]# func “*.com” call service restart “httpd”[root@a~]# func “*.lan” call command run “df h”
>>> from func.overlord import client>>> client1 = client.Client(“*.lan”)>>> client1.service.restart(“httpd”)>>> client1.command.run(“df h”)
![Page 33: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/33.jpg)
Modules and new features
● Func based on modules architecture● A module support new stuff● 20 modules (libVirt, jboss, info, process,
command, iptables, nagios, etc)● Writing a new module is simple. ● When you write a module, it works on both CLI
and PyScripting, no modification on func are needed.
![Page 34: Cobbler, Func and Puppet: Tools for Large Scale Environments](https://reader033.vdocuments.us/reader033/viewer/2022052822/554f9faeb4c905ad218b4974/html5/thumbnails/34.jpg)
THANK YOU.
Credits: Byte-Code, Tango Icons, Crystals Icons